我希望提供一个结构化的配置文件,它对于非技术用户来说尽可能容易编辑(不幸的是它必须是一个文件),所以我想使用YAML。然而,我找不到任何方法从Unix shell脚本解析这个。
当前回答
perl -ne 'chomp; printf qq/%s="%s"\n/, split(/\s*:\s*/,$_,2)' file.yml > file.sh
其他回答
如果你需要一个单一的值,你可以使用一个工具将你的YAML文档转换为JSON并提供给jq,例如yq。
sample.yaml的内容:
---
bob:
item1:
cats: bananas
item2:
cats: apples
thing:
cats: oranges
例子:
$ yq -r '.bob["thing"]["cats"]' sample.yaml
oranges
yq是一个轻量级、可移植的命令行YAML处理器
这个项目的目标是yaml文件的jq或sed。
(https://github.com/mikefarah/yq #自述)
作为示例(直接从文档中窃取),给出一个示例。Yaml文件:
---
bob:
item1:
cats: bananas
item2:
cats: apples
then
yq eval '.bob.*.cats' sample.yaml
将输出
- bananas
- apples
使用Python的PyYAML或YAML::Perl等库最容易进行复杂的解析。
如果您希望将所有YAML值解析为bash值,请尝试此脚本。这也可以处理注释。参见下面的示例用法:
# pparse.py
import yaml
import sys
def parse_yaml(yml, name=''):
if isinstance(yml, list):
for data in yml:
parse_yaml(data, name)
elif isinstance(yml, dict):
if (len(yml) == 1) and not isinstance(yml[list(yml.keys())[0]], list):
print(str(name+'_'+list(yml.keys())[0]+'='+str(yml[list(yml.keys())[0]]))[1:])
else:
for key in yml:
parse_yaml(yml[key], name+'_'+key)
if __name__=="__main__":
yml = yaml.safe_load(open(sys.argv[1]))
parse_yaml(yml)
test.yml
- folders:
- temp_folder: datasets/outputs/tmp
- keep_temp_folder: false
- MFA:
- MFA: false
- speaker_count: 1
- G2P:
- G2P: true
- G2P_model: models/MFA/G2P/english_g2p.zip
- input_folder: datasets/outputs/Youtube/ljspeech/wavs
- output_dictionary: datasets/outputs/Youtube/ljspeech/dictionary.dict
- dictionary: datasets/outputs/Youtube/ljspeech/dictionary.dict
- acoustic_model: models/MFA/acoustic/english.zip
- temp_folder: datasets/outputs/tmp
- jobs: 4
- align:
- config: configs/MFA/align.yaml
- dataset: datasets/outputs/Youtube/ljspeech/wavs
- output_folder: datasets/outputs/Youtube/ljspeech-aligned
- TTS:
- output_folder: datasets/outputs/Youtube
- preprocess:
- preprocess: true
- config: configs/TTS_preprocess.yaml # Default Config
- textgrid_folder: datasets/outputs/Youtube/ljspeech-aligned
- output_duration_folder: datasets/outputs/Youtube/durations
- sampling_rate: 44000 # Make sure sampling rate is same here as in preprocess config
需要YAML值的脚本:
yaml() {
eval $(python pparse.py "$1")
}
yaml "test.yml"
# What python printed to bash:
folders_temp_folder=datasets/outputs/tmp
folders_keep_temp_folder=False
MFA_MFA=False
MFA_speaker_count=1
MFA_G2P_G2P=True
MFA_G2P_G2P_model=models/MFA/G2P/english_g2p.zip
MFA_G2P_input_folder=datasets/outputs/Youtube/ljspeech/wavs
MFA_G2P_output_dictionary=datasets/outputs/Youtube/ljspeech/dictionary.dict
MFA_dictionary=datasets/outputs/Youtube/ljspeech/dictionary.dict
MFA_acoustic_model=models/MFA/acoustic/english.zip
MFA_temp_folder=datasets/outputs/tmp
MFA_jobs=4
MFA_align_config=configs/MFA/align.yaml
MFA_align_dataset=datasets/outputs/Youtube/ljspeech/wavs
MFA_align_output_folder=datasets/outputs/Youtube/ljspeech-aligned
TTS_output_folder=datasets/outputs/Youtube
TTS_preprocess_preprocess=True
TTS_preprocess_config=configs/TTS_preprocess.yaml
TTS_preprocess_textgrid_folder=datasets/outputs/Youtube/ljspeech-aligned
TTS_preprocess_output_duration_folder=datasets/outputs/Youtube/durations
TTS_preprocess_sampling_rate=44000
使用bash访问变量:
echo "$TTS_preprocess_sampling_rate";
>>> 44000
另一种选择是将YAML转换为JSON,然后使用jq与JSON表示进行交互,从其中提取信息或编辑信息。
我写了一个简单的bash脚本,包含这个胶水-见Y2J项目在GitHub上
我已经用python编写了shyaml,用于从shell命令行查询YAML。
概述:
$ pip install shyaml ## installation
示例的YAML文件(具有复杂的功能):
$ cat <<EOF > test.yaml
name: "MyName !!"
subvalue:
how-much: 1.1
things:
- first
- second
- third
other-things: [a, b, c]
maintainer: "Valentin Lab"
description: |
Multiline description:
Line 1
Line 2
EOF
基本的查询:
$ cat test.yaml | shyaml get-value subvalue.maintainer
Valentin Lab
更复杂的循环查询复杂的值:
$ cat test.yaml | shyaml values-0 | \
while read -r -d $'\0' value; do
echo "RECEIVED: '$value'"
done
RECEIVED: '1.1'
RECEIVED: '- first
- second
- third'
RECEIVED: '2'
RECEIVED: 'Valentin Lab'
RECEIVED: 'Multiline description:
Line 1
Line 2'
以下几个要点:
all YAML types and syntax oddities are correctly handled, as multiline, quoted strings, inline sequences... \0 padded output is available for solid multiline entry manipulation. simple dotted notation to select sub-values (ie: subvalue.maintainer is a valid key). access by index is provided to sequences (ie: subvalue.things.-1 is the last element of the subvalue.things sequence.) access to all sequence/structs elements in one go for use in bash loops. you can output whole subpart of a YAML file as ... YAML, which blend well for further manipulations with shyaml.
更多的示例和文档可以在shyaml github页面或shyaml PyPI页面上找到。
推荐文章
- 如何从查找“类型d”中排除此/ current / dot文件夹
- 只使用md5sum获取哈希值(没有文件名)
- 使用sh shell比较字符串
- 在Bash中测试非零长度字符串:[-n "$var"]或["$var"]
- 如何创建Bash别名?
- 如何设置ssh超时时间?
- 将所有变量从一个shell脚本传递到另一个?
- 只列出UNIX中的目录
- 如何删除shell脚本中文件名的扩展名?
- 'find -exec'是Linux中的shell函数
- 临时更改bash中的当前工作目录以运行命令
- 重定向复制的标准输出到日志文件从bash脚本本身
- Shell脚本for循环语法
- Docker入口点运行bash脚本被“拒绝权限”
- Chmod递归