我有以下YAML:
paths:
patha: /path/to/root/a
pathb: /path/to/root/b
pathc: /path/to/root/c
我如何“正常化”它,通过从三个路径中删除/path/到/root/,并将其作为自己的设置,类似于:
paths:
root: /path/to/root/
patha: *root* + a
pathb: *root* + b
pathc: *root* + c
显然这是无效的,我瞎编的。真正的语法是什么?这能做到吗?
使用OmegaConf
OmegaConf是一个基于yaml的分层配置系统,在变量插值功能下支持此功能。使用OmegaConf v2.2.2:
创建YAML文件路径。Yaml如下:
paths:
root: /path/to/root/
patha: ${.root}a
pathb: ${.root}b
pathc: ${.root}c
然后我们可以读取可变路径的文件:
from omegaconf import OmegaConf
conf = OmegaConf.load("test_paths.yaml")
>>> conf.paths.root
'/path/to/root/'
>>> conf.paths.patha
'/path/to/root/a'
>>> conf.paths.pathb
'/path/to/root/b'
>>> conf.paths.pathc
'/path/to/root/c'
深度和交叉裁判
可以定义更复杂的(嵌套的)结构,使用变量的相对深度来引用其他变量:
创建另一个文件nested_paths.yaml:
data:
base: data
sub_dir_A:
name: a
# here we note that `base` is two levels above this variable
# hence we will use `..base` two dots but the `name` variable is
# at the same level hence a single dot `.name`
nested_dir: ${..base}/sub_dir/${.name}/last_dir
sub_dir_B:
# add another level of depth
- name: b
# due to another level of depth, we have to use three dots
# to access `base` variable as `...base`
nested_file: ${...base}/sub_dir/${.name}/dirs.txt
- name: c
# we can also make cross-references to other variables
cross_ref_dir: ${...sub_dir_A.nested_dir}/${.name}
我们可以再次检查:
conf = OmegaConf.load("nested_paths.yaml")
# 1-level of depth reference
>>> conf.data.sub_dir_A.nested_dir
'data/sub_dir/a/last_dir'
# 2-levels of depth reference
>>> conf.data.sub_dir_B[0].nested_file
'data/sub_dir/b/dirs.txt'
# cross-reference example
>>> conf.data.sub_dir_B[1].cross_ref_dir
'data/sub_dir/a/last_dir/c'
在无效引用的情况下(例如错误的深度,错误的变量名),OmegaConf将抛出错误OmegaConf .errors. interpolationresolutionerror。在Hydra中,它还用于配置复杂的应用程序。
我用Python编写了自己的库来扩展从目录中加载的变量,其层次结构如下:
/root
|
+- /proj1
|
+- config.yaml
|
+- /proj2
|
+- config.yaml
|
... and so on ...
这里的关键区别是,扩展必须在所有配置完成后才应用。Yaml文件被加载,其中来自下一个文件的变量可以覆盖来自前一个文件的变量,所以伪代码应该是这样的:
env = YamlEnv()
env.load('/root/proj1/config.yaml')
env.load('/root/proj1/proj2/config.yaml')
...
env.expand()
作为一个附加选项,xonsh脚本可以将结果变量导出到环境变量中(请参阅yaml_update_global_vars函数)。
脚本:
https://sourceforge.net/p/tacklelib/tacklelib/HEAD/tree/trunk/python/cmdoplib/cmdoplib.yaml.xsh
https://sourceforge.net/p/tacklelib/tacklelib/HEAD/tree/trunk/python/tacklelib/tacklelib.yaml.py
优点:
简单,不支持递归和嵌套变量
可以将未定义的变量替换为占位符(${MYUNDEFINEDVAR} -> *$/{MYUNDEFINEDVAR})
可以从环境变量(${env:MYVAR}
${env:MYVAR:path}可以替换所有的\\ to / in路径变量
缺点:
不支持嵌套变量,因此不能展开嵌套字典中的值(例如${MYSCOPE. xml)。MYVAR}未实现)
不检测展开递归,包括递归后的占位符放
使用OmegaConf
OmegaConf是一个基于yaml的分层配置系统,在变量插值功能下支持此功能。使用OmegaConf v2.2.2:
创建YAML文件路径。Yaml如下:
paths:
root: /path/to/root/
patha: ${.root}a
pathb: ${.root}b
pathc: ${.root}c
然后我们可以读取可变路径的文件:
from omegaconf import OmegaConf
conf = OmegaConf.load("test_paths.yaml")
>>> conf.paths.root
'/path/to/root/'
>>> conf.paths.patha
'/path/to/root/a'
>>> conf.paths.pathb
'/path/to/root/b'
>>> conf.paths.pathc
'/path/to/root/c'
深度和交叉裁判
可以定义更复杂的(嵌套的)结构,使用变量的相对深度来引用其他变量:
创建另一个文件nested_paths.yaml:
data:
base: data
sub_dir_A:
name: a
# here we note that `base` is two levels above this variable
# hence we will use `..base` two dots but the `name` variable is
# at the same level hence a single dot `.name`
nested_dir: ${..base}/sub_dir/${.name}/last_dir
sub_dir_B:
# add another level of depth
- name: b
# due to another level of depth, we have to use three dots
# to access `base` variable as `...base`
nested_file: ${...base}/sub_dir/${.name}/dirs.txt
- name: c
# we can also make cross-references to other variables
cross_ref_dir: ${...sub_dir_A.nested_dir}/${.name}
我们可以再次检查:
conf = OmegaConf.load("nested_paths.yaml")
# 1-level of depth reference
>>> conf.data.sub_dir_A.nested_dir
'data/sub_dir/a/last_dir'
# 2-levels of depth reference
>>> conf.data.sub_dir_B[0].nested_file
'data/sub_dir/b/dirs.txt'
# cross-reference example
>>> conf.data.sub_dir_B[1].cross_ref_dir
'data/sub_dir/a/last_dir/c'
在无效引用的情况下(例如错误的深度,错误的变量名),OmegaConf将抛出错误OmegaConf .errors. interpolationresolutionerror。在Hydra中,它还用于配置复杂的应用程序。