我希望提供一个结构化的配置文件,它对于非技术用户来说尽可能容易编辑(不幸的是它必须是一个文件),所以我想使用YAML。然而,我找不到任何方法从Unix shell脚本解析这个。
当前回答
以下是Stefan Farestam回答的扩展版本:
function parse_yaml {
local prefix=$2
local s='[[:space:]]*' w='[a-zA-Z0-9_]*' fs=$(echo @|tr @ '\034')
sed -ne "s|,$s\]$s\$|]|" \
-e ":1;s|^\($s\)\($w\)$s:$s\[$s\(.*\)$s,$s\(.*\)$s\]|\1\2: [\3]\n\1 - \4|;t1" \
-e "s|^\($s\)\($w\)$s:$s\[$s\(.*\)$s\]|\1\2:\n\1 - \3|;p" $1 | \
sed -ne "s|,$s}$s\$|}|" \
-e ":1;s|^\($s\)-$s{$s\(.*\)$s,$s\($w\)$s:$s\(.*\)$s}|\1- {\2}\n\1 \3: \4|;t1" \
-e "s|^\($s\)-$s{$s\(.*\)$s}|\1-\n\1 \2|;p" | \
sed -ne "s|^\($s\):|\1|" \
-e "s|^\($s\)-$s[\"']\(.*\)[\"']$s\$|\1$fs$fs\2|p" \
-e "s|^\($s\)-$s\(.*\)$s\$|\1$fs$fs\2|p" \
-e "s|^\($s\)\($w\)$s:$s[\"']\(.*\)[\"']$s\$|\1$fs\2$fs\3|p" \
-e "s|^\($s\)\($w\)$s:$s\(.*\)$s\$|\1$fs\2$fs\3|p" | \
awk -F$fs '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]; idx[i]=0}}
if(length($2)== 0){ vname[indent]= ++idx[indent] };
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) { vn=(vn)(vname[i])("_")}
printf("%s%s%s=\"%s\"\n", "'$prefix'",vn, vname[indent], $3);
}
}'
}
该版本支持字典和列表的-符号和短符号。以下输入:
global:
input:
- "main.c"
- "main.h"
flags: [ "-O3", "-fpic" ]
sample_input:
- { property1: value, property2: "value2" }
- { property1: "value3", property2: 'value 4' }
产生如下输出:
global_input_1="main.c"
global_input_2="main.h"
global_flags_1="-O3"
global_flags_2="-fpic"
global_sample_input_1_property1="value"
global_sample_input_1_property2="value2"
global_sample_input_2_property1="value3"
global_sample_input_2_property2="value 4"
as you can see the - items automatically get numbered in order to obtain different variable names for each item. In bash there are no multidimensional arrays, so this is one way to work around. Multiple levels are supported. To work around the problem with trailing white spaces mentioned by @briceburg one should enclose the values in single or double quotes. However, there are still some limitations: Expansion of the dictionaries and lists can produce wrong results when values contain commas. Also, more complex structures like values spanning multiple lines (like ssh-keys) are not (yet) supported.
A few words about the code: The first sed command expands the short form of dictionaries { key: value, ...} to regular and converts them to more simple yaml style. The second sed call does the same for the short notation of lists and converts [ entry, ... ] to an itemized list with the - notation. The third sed call is the original one that handled normal dictionaries, now with the addition to handle lists with - and indentations. The awk part introduces an index for each indentation level and increases it when the variable name is empty (i.e. when processing a list). The current value of the counters are used instead of the empty vname. When going up one level, the counters are zeroed.
编辑:我已经为此创建了一个github存储库。
其他回答
我已经用python编写了shyaml,用于从shell命令行查询YAML。
概述:
$ pip install shyaml ## installation
示例的YAML文件(具有复杂的功能):
$ cat <<EOF > test.yaml
name: "MyName !!"
subvalue:
how-much: 1.1
things:
- first
- second
- third
other-things: [a, b, c]
maintainer: "Valentin Lab"
description: |
Multiline description:
Line 1
Line 2
EOF
基本的查询:
$ cat test.yaml | shyaml get-value subvalue.maintainer
Valentin Lab
更复杂的循环查询复杂的值:
$ cat test.yaml | shyaml values-0 | \
while read -r -d $'\0' value; do
echo "RECEIVED: '$value'"
done
RECEIVED: '1.1'
RECEIVED: '- first
- second
- third'
RECEIVED: '2'
RECEIVED: 'Valentin Lab'
RECEIVED: 'Multiline description:
Line 1
Line 2'
以下几个要点:
all YAML types and syntax oddities are correctly handled, as multiline, quoted strings, inline sequences... \0 padded output is available for solid multiline entry manipulation. simple dotted notation to select sub-values (ie: subvalue.maintainer is a valid key). access by index is provided to sequences (ie: subvalue.things.-1 is the last element of the subvalue.things sequence.) access to all sequence/structs elements in one go for use in bash loops. you can output whole subpart of a YAML file as ... YAML, which blend well for further manipulations with shyaml.
更多的示例和文档可以在shyaml github页面或shyaml PyPI页面上找到。
你也可以考虑使用Grunt (JavaScript任务运行器)。可以很容易地与shell集成。它支持读取YAML (grunt.file.readYAML)和JSON (grunt.file.readJSON)文件。
这可以通过在Gruntfile.js(或Gruntfile.coffee)中创建一个任务来实现,例如:
module.exports = function (grunt) {
grunt.registerTask('foo', ['load_yml']);
grunt.registerTask('load_yml', function () {
var data = grunt.file.readYAML('foo.yml');
Object.keys(data).forEach(function (g) {
// ... switch (g) { case 'my_key':
});
});
};
然后在shell中简单地运行grunt foo(检查grunt—help是否有可用的任务)。
此外,你可以实现exec:foo任务(grunt-exec)与输入变量从你的任务(foo: {cmd: 'echo bar <%= foo %>'})为了打印输出在任何格式你想要的,然后管道到另一个命令。
还有一个类似于Grunt的工具,它叫做gulp,带有额外的插件gulp-yaml。
安装方法:npm Install——save-dev gulp-yaml
示例用法:
var yaml = require('gulp-yaml');
gulp.src('./src/*.yml')
.pipe(yaml())
.pipe(gulp.dest('./dist/'))
gulp.src('./src/*.yml')
.pipe(yaml({ space: 2 }))
.pipe(gulp.dest('./dist/'))
gulp.src('./src/*.yml')
.pipe(yaml({ safe: true }))
.pipe(gulp.dest('./dist/'))
要了解更多处理YAML格式的选项,请查看YAML网站上可用的项目、库和其他资源,这些资源可以帮助您解析该格式。
其他工具:
Jshon 解析、读取和创建JSON
另一种选择是将YAML转换为JSON,然后使用jq与JSON表示进行交互,从其中提取信息或编辑信息。
我写了一个简单的bash脚本,包含这个胶水-见Y2J项目在GitHub上
perl -ne 'chomp; printf qq/%s="%s"\n/, split(/\s*:\s*/,$_,2)' file.yml > file.sh
如果你有python 2和PyYAML,你可以使用我写的这个解析器parse_yaml.py。它做的一些更整洁的事情是让您选择一个前缀(以防您有多个具有类似变量的文件),并从yaml文件中选择一个值。
例如,如果你有这些yaml文件:
staging.yaml:
db:
type: sqllite
host: 127.0.0.1
user: dev
password: password123
prod.yaml:
db:
type: postgres
host: 10.0.50.100
user: postgres
password: password123
您可以加载两者而不会产生冲突。
$ eval $(python parse_yaml.py prod.yaml --prefix prod --cap)
$ eval $(python parse_yaml.py staging.yaml --prefix stg --cap)
$ echo $PROD_DB_HOST
10.0.50.100
$ echo $STG_DB_HOST
127.0.0.1
甚至可以选择你想要的值。
$ prod_user=$(python parse_yaml.py prod.yaml --get db_user)
$ prod_port=$(python parse_yaml.py prod.yaml --get db_port --default 5432)
$ echo prod_user
postgres
$ echo prod_port
5432
推荐文章
- 查看PS命令的全部输出
- 确保一次只运行一个shell脚本实例的快速方法
- 如何从同一个YAML文件的其他地方引用YAML“设置”?
- 如何在Makefile中设置子进程的环境变量
- 如何让“wc -l”打印没有文件名的行数?
- 有效地测试Linux上的端口是否打开?
- 如何从另一个文件A中删除文件B中出现的行?
- Shell脚本删除超过n天的目录
- 如何检查shell脚本中是否存在命令?
- 如何使用查找命令从列表中查找所有具有扩展名的文件?
- 如何打破一个循环在Bash?
- 如何合并2 JSON对象从2个文件使用jq?
- 删除Bash脚本中的重复条目
- 如何将Bash命令的输出分配给变量?
- PHP中的异步shell执行器