我希望提供一个结构化的配置文件,它对于非技术用户来说尽可能容易编辑(不幸的是它必须是一个文件),所以我想使用YAML。然而,我找不到任何方法从Unix shell脚本解析这个。
当前回答
我已经用python编写了shyaml,用于从shell命令行查询YAML。
概述:
$ pip install shyaml ## installation
示例的YAML文件(具有复杂的功能):
$ cat <<EOF > test.yaml
name: "MyName !!"
subvalue:
how-much: 1.1
things:
- first
- second
- third
other-things: [a, b, c]
maintainer: "Valentin Lab"
description: |
Multiline description:
Line 1
Line 2
EOF
基本的查询:
$ cat test.yaml | shyaml get-value subvalue.maintainer
Valentin Lab
更复杂的循环查询复杂的值:
$ cat test.yaml | shyaml values-0 | \
while read -r -d $'\0' value; do
echo "RECEIVED: '$value'"
done
RECEIVED: '1.1'
RECEIVED: '- first
- second
- third'
RECEIVED: '2'
RECEIVED: 'Valentin Lab'
RECEIVED: 'Multiline description:
Line 1
Line 2'
以下几个要点:
all YAML types and syntax oddities are correctly handled, as multiline, quoted strings, inline sequences... \0 padded output is available for solid multiline entry manipulation. simple dotted notation to select sub-values (ie: subvalue.maintainer is a valid key). access by index is provided to sequences (ie: subvalue.things.-1 is the last element of the subvalue.things sequence.) access to all sequence/structs elements in one go for use in bash loops. you can output whole subpart of a YAML file as ... YAML, which blend well for further manipulations with shyaml.
更多的示例和文档可以在shyaml github页面或shyaml PyPI页面上找到。
其他回答
使用Python的PyYAML或YAML::Perl等库最容易进行复杂的解析。
如果您希望将所有YAML值解析为bash值,请尝试此脚本。这也可以处理注释。参见下面的示例用法:
# pparse.py
import yaml
import sys
def parse_yaml(yml, name=''):
if isinstance(yml, list):
for data in yml:
parse_yaml(data, name)
elif isinstance(yml, dict):
if (len(yml) == 1) and not isinstance(yml[list(yml.keys())[0]], list):
print(str(name+'_'+list(yml.keys())[0]+'='+str(yml[list(yml.keys())[0]]))[1:])
else:
for key in yml:
parse_yaml(yml[key], name+'_'+key)
if __name__=="__main__":
yml = yaml.safe_load(open(sys.argv[1]))
parse_yaml(yml)
test.yml
- folders:
- temp_folder: datasets/outputs/tmp
- keep_temp_folder: false
- MFA:
- MFA: false
- speaker_count: 1
- G2P:
- G2P: true
- G2P_model: models/MFA/G2P/english_g2p.zip
- input_folder: datasets/outputs/Youtube/ljspeech/wavs
- output_dictionary: datasets/outputs/Youtube/ljspeech/dictionary.dict
- dictionary: datasets/outputs/Youtube/ljspeech/dictionary.dict
- acoustic_model: models/MFA/acoustic/english.zip
- temp_folder: datasets/outputs/tmp
- jobs: 4
- align:
- config: configs/MFA/align.yaml
- dataset: datasets/outputs/Youtube/ljspeech/wavs
- output_folder: datasets/outputs/Youtube/ljspeech-aligned
- TTS:
- output_folder: datasets/outputs/Youtube
- preprocess:
- preprocess: true
- config: configs/TTS_preprocess.yaml # Default Config
- textgrid_folder: datasets/outputs/Youtube/ljspeech-aligned
- output_duration_folder: datasets/outputs/Youtube/durations
- sampling_rate: 44000 # Make sure sampling rate is same here as in preprocess config
需要YAML值的脚本:
yaml() {
eval $(python pparse.py "$1")
}
yaml "test.yml"
# What python printed to bash:
folders_temp_folder=datasets/outputs/tmp
folders_keep_temp_folder=False
MFA_MFA=False
MFA_speaker_count=1
MFA_G2P_G2P=True
MFA_G2P_G2P_model=models/MFA/G2P/english_g2p.zip
MFA_G2P_input_folder=datasets/outputs/Youtube/ljspeech/wavs
MFA_G2P_output_dictionary=datasets/outputs/Youtube/ljspeech/dictionary.dict
MFA_dictionary=datasets/outputs/Youtube/ljspeech/dictionary.dict
MFA_acoustic_model=models/MFA/acoustic/english.zip
MFA_temp_folder=datasets/outputs/tmp
MFA_jobs=4
MFA_align_config=configs/MFA/align.yaml
MFA_align_dataset=datasets/outputs/Youtube/ljspeech/wavs
MFA_align_output_folder=datasets/outputs/Youtube/ljspeech-aligned
TTS_output_folder=datasets/outputs/Youtube
TTS_preprocess_preprocess=True
TTS_preprocess_config=configs/TTS_preprocess.yaml
TTS_preprocess_textgrid_folder=datasets/outputs/Youtube/ljspeech-aligned
TTS_preprocess_output_duration_folder=datasets/outputs/Youtube/durations
TTS_preprocess_sampling_rate=44000
使用bash访问变量:
echo "$TTS_preprocess_sampling_rate";
>>> 44000
我已经用python编写了shyaml,用于从shell命令行查询YAML。
概述:
$ pip install shyaml ## installation
示例的YAML文件(具有复杂的功能):
$ cat <<EOF > test.yaml
name: "MyName !!"
subvalue:
how-much: 1.1
things:
- first
- second
- third
other-things: [a, b, c]
maintainer: "Valentin Lab"
description: |
Multiline description:
Line 1
Line 2
EOF
基本的查询:
$ cat test.yaml | shyaml get-value subvalue.maintainer
Valentin Lab
更复杂的循环查询复杂的值:
$ cat test.yaml | shyaml values-0 | \
while read -r -d $'\0' value; do
echo "RECEIVED: '$value'"
done
RECEIVED: '1.1'
RECEIVED: '- first
- second
- third'
RECEIVED: '2'
RECEIVED: 'Valentin Lab'
RECEIVED: 'Multiline description:
Line 1
Line 2'
以下几个要点:
all YAML types and syntax oddities are correctly handled, as multiline, quoted strings, inline sequences... \0 padded output is available for solid multiline entry manipulation. simple dotted notation to select sub-values (ie: subvalue.maintainer is a valid key). access by index is provided to sequences (ie: subvalue.things.-1 is the last element of the subvalue.things sequence.) access to all sequence/structs elements in one go for use in bash loops. you can output whole subpart of a YAML file as ... YAML, which blend well for further manipulations with shyaml.
更多的示例和文档可以在shyaml github页面或shyaml PyPI页面上找到。
我曾经使用python将yaml转换为json,并在jq中进行处理。
python -c "import yaml; import json; from pathlib import Path; print(json.dumps(yaml.safe_load(Path('file.yml').read_text())))" | jq '.'
另一种选择是将YAML转换为JSON,然后使用jq与JSON表示进行交互,从其中提取信息或编辑信息。
我写了一个简单的bash脚本,包含这个胶水-见Y2J项目在GitHub上
我知道这是非常具体的,但我认为我的回答可能对某些用户有帮助。 如果你的机器上安装了node和npm,你可以使用js-yaml。 首次安装:
npm i -g js-yaml
# or locally
npm i js-yaml
然后在bash脚本中
#!/bin/bash
js-yaml your-yaml-file.yml
如果你正在使用jq,你也可以这样做
#!/bin/bash
json="$(js-yaml your-yaml-file.yml)"
aproperty="$(jq '.apropery' <<< "$json")"
echo "$aproperty"
因为js-yaml将yaml文件转换为json字符串文字。然后,您可以在unix系统中的任何json解析器中使用该字符串。
推荐文章
- 如何从同一个YAML文件的其他地方引用YAML“设置”?
- 如何在Makefile中设置子进程的环境变量
- 如何让“wc -l”打印没有文件名的行数?
- 有效地测试Linux上的端口是否打开?
- 如何从另一个文件A中删除文件B中出现的行?
- Shell脚本删除超过n天的目录
- 如何检查shell脚本中是否存在命令?
- 如何使用查找命令从列表中查找所有具有扩展名的文件?
- 如何打破一个循环在Bash?
- 如何合并2 JSON对象从2个文件使用jq?
- 删除Bash脚本中的重复条目
- 如何将Bash命令的输出分配给变量?
- PHP中的异步shell执行器
- Git显示“警告:永久添加到已知主机列表”
- 检索Linux上单个进程的CPU使用情况和内存使用情况?