我希望提供一个结构化的配置文件,它对于非技术用户来说尽可能容易编辑(不幸的是它必须是一个文件),所以我想使用YAML。然而,我找不到任何方法从Unix shell脚本解析这个。
当前回答
可以将一个小脚本传递给一些解释器,比如Python。使用Ruby和它的YAML库的简单方法如下:
$ RUBY_SCRIPT="data = YAML::load(STDIN.read); puts data['a']; puts data['b']"
$ echo -e '---\na: 1234\nb: 4321' | ruby -ryaml -e "$RUBY_SCRIPT"
1234
4321
,其中data是来自yaml的值的散列(或数组)。
作为奖励,它可以很好地解析杰基尔的正面问题。
ruby -ryaml -e "puts YAML::load(open(ARGV.first).read)['tags']" example.md
其他回答
如果你有python 2和PyYAML,你可以使用我写的这个解析器parse_yaml.py。它做的一些更整洁的事情是让您选择一个前缀(以防您有多个具有类似变量的文件),并从yaml文件中选择一个值。
例如,如果你有这些yaml文件:
staging.yaml:
db:
type: sqllite
host: 127.0.0.1
user: dev
password: password123
prod.yaml:
db:
type: postgres
host: 10.0.50.100
user: postgres
password: password123
您可以加载两者而不会产生冲突。
$ eval $(python parse_yaml.py prod.yaml --prefix prod --cap)
$ eval $(python parse_yaml.py staging.yaml --prefix stg --cap)
$ echo $PROD_DB_HOST
10.0.50.100
$ echo $STG_DB_HOST
127.0.0.1
甚至可以选择你想要的值。
$ prod_user=$(python parse_yaml.py prod.yaml --get db_user)
$ prod_port=$(python parse_yaml.py prod.yaml --get db_port --default 5432)
$ echo prod_user
postgres
$ echo prod_port
5432
下面是一个bash-only解析器,利用sed和awk来解析简单的yaml文件:
function parse_yaml {
local prefix=$2
local s='[[:space:]]*' w='[a-zA-Z0-9_]*' fs=$(echo @|tr @ '\034')
sed -ne "s|^\($s\):|\1|" \
-e "s|^\($s\)\($w\)$s:$s[\"']\(.*\)[\"']$s\$|\1$fs\2$fs\3|p" \
-e "s|^\($s\)\($w\)$s:$s\(.*\)$s\$|\1$fs\2$fs\3|p" $1 |
awk -F$fs '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]}}
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
printf("%s%s%s=\"%s\"\n", "'$prefix'",vn, $2, $3);
}
}'
}
它可以理解以下文件:
## global definitions
global:
debug: yes
verbose: no
debugging:
detailed: no
header: "debugging started"
## output
output:
file: "yes"
在解析时使用:
parse_yaml sample.yml
将输出:
global_debug="yes"
global_verbose="no"
global_debugging_detailed="no"
global_debugging_header="debugging started"
output_file="yes"
它也理解由ruby生成的yaml文件,其中可能包含ruby符号,例如:
---
:global:
:debug: 'yes'
:verbose: 'no'
:debugging:
:detailed: 'no'
:header: debugging started
:output: 'yes'
并将输出与前一个示例相同的结果。
脚本中的典型用法是:
eval $(parse_yaml sample.yml)
Parse_yaml接受一个前缀参数,这样导入的所有设置都有一个公共前缀(这将减少名称空间冲突的风险)。
parse_yaml sample.yml "CONF_"
收益率:
CONF_global_debug="yes"
CONF_global_verbose="no"
CONF_global_debugging_detailed="no"
CONF_global_debugging_header="debugging started"
CONF_output_file="yes"
注意,之前文件中的设置可以被后面的设置引用:
## global definitions
global:
debug: yes
verbose: no
debugging:
detailed: no
header: "debugging started"
## output
output:
debug: $global_debug
另一个很好的用法是先解析默认文件,然后解析用户设置,这是可行的,因为后一个设置会覆盖第一个设置:
eval $(parse_yaml defaults.yml)
eval $(parse_yaml project.yml)
把我的答案从如何在bash中将json响应转换为yaml,因为这似乎是关于从命令行处理yaml文本解析的权威帖子。
我想添加一些关于yq YAML实现的细节。由于这个YAML解析器有两种实现,名称都是yq,如果不查看实现的DSL,就很难区分使用的是哪一种。有两个可用的实现
kislyuk/yq——更常被提及的版本,它是jq的包装器,用Python编写,使用PyYAML库进行YAML解析 mikefarah/yq -一个Go实现,使用Go -yaml v3解析器,有自己的动态DSL。
几乎所有主要发行版都可以通过标准安装包管理器进行安装
kislyuk/yq -安装说明 mikefarah/yq -安装说明
这两个版本都有一些优点和缺点,但有一些有效的点需要强调(从他们的回购指令中采用)
kislyuk - yq
Since the DSL is the adopted completely from jq, for users familiar with the latter, the parsing and manipulation becomes quite straightforward Supports mode to preserve YAML tags and styles, but loses comments during the conversion. Since jq doesn't preserve comments, during the round-trip conversion, the comments are lost. As part of the package, XML support is built in. An executable, xq, which transcodes XML to JSON using xmltodict and pipes it to jq, on which you can apply the same DSL to perform CRUD operations on the objects and round-trip the output back to XML. Supports in-place edit mode with -i flag (similar to sed -i)
迈克法拉/YQ
Prone to frequent changes in DSL, migration from 2.x - 3.x Rich support for anchors, styles and tags. But lookout for bugs once in a while A relatively simple Path expression syntax to navigate and match yaml nodes Supports YAML->JSON, JSON->YAML formatting and pretty printing YAML (with comments) Supports in-place edit mode with -i flag (similar to sed -i) Supports coloring the output YAML with -C flag (not applicable for JSON output) and indentation of the sub elements (default at 2 spaces) Supports Shell completion for most shells - Bash, zsh (because of powerful support from spf13/cobra used to generate CLI flags)
我对以下两个版本的YAML的看法(在其他答案中也有引用)
root_key1: this is value one
root_key2: "this is value two"
drink:
state: liquid
coffee:
best_served: hot
colour: brown
orange_juice:
best_served: cold
colour: orange
food:
state: solid
apple_pie:
best_served: warm
root_key_3: this is value three
对这两个实现执行的各种操作(一些常用操作)
修改根节点值—修改“root_key2”的值 修改数组内容,增加值-为coffee添加属性 修改数组内容,删除value - Delete属性从orange_juice 打印带有路径的键/值对—用于food下的所有项目
使用kislyuk / yq
Yq -y '。Root_key2 |= "this is a new value 你,你,喝。咖啡+={时间:"always"}' yaml Yq -y 'del(.drink.orange_juice.colour)' yaml yq - r的.food |路径(标量)美元p | (($ p |加入(“。”)),(getpath ($ p) | tojson)] | @tsv的yaml
这很简单。你所需要做的就是用-y标志将jq JSON输出转码回YAML。
用mikefarah - yq
Yq w yaml root_key2 "这是一个新值" Yq w yaml喝。咖啡。时间“总是” Yq d yaml饮料。橙汁。颜色 yq r yaml——printMode pv "food.**"
截至2020年12月21日,yq v4是测试版,支持许多强大的路径表达式,并支持类似于使用jq的DSL。阅读过渡说明-从V3升级
你也可以考虑使用Grunt (JavaScript任务运行器)。可以很容易地与shell集成。它支持读取YAML (grunt.file.readYAML)和JSON (grunt.file.readJSON)文件。
这可以通过在Gruntfile.js(或Gruntfile.coffee)中创建一个任务来实现,例如:
module.exports = function (grunt) {
grunt.registerTask('foo', ['load_yml']);
grunt.registerTask('load_yml', function () {
var data = grunt.file.readYAML('foo.yml');
Object.keys(data).forEach(function (g) {
// ... switch (g) { case 'my_key':
});
});
};
然后在shell中简单地运行grunt foo(检查grunt—help是否有可用的任务)。
此外,你可以实现exec:foo任务(grunt-exec)与输入变量从你的任务(foo: {cmd: 'echo bar <%= foo %>'})为了打印输出在任何格式你想要的,然后管道到另一个命令。
还有一个类似于Grunt的工具,它叫做gulp,带有额外的插件gulp-yaml。
安装方法:npm Install——save-dev gulp-yaml
示例用法:
var yaml = require('gulp-yaml');
gulp.src('./src/*.yml')
.pipe(yaml())
.pipe(gulp.dest('./dist/'))
gulp.src('./src/*.yml')
.pipe(yaml({ space: 2 }))
.pipe(gulp.dest('./dist/'))
gulp.src('./src/*.yml')
.pipe(yaml({ safe: true }))
.pipe(gulp.dest('./dist/'))
要了解更多处理YAML格式的选项,请查看YAML网站上可用的项目、库和其他资源,这些资源可以帮助您解析该格式。
其他工具:
Jshon 解析、读取和创建JSON
如果您知道您感兴趣的标记和您期望的yaml结构,那么在Bash中编写一个简单的yaml解析器并不难。
在下面的示例中,解析器将一个结构化YAML文件读入环境变量、数组和关联数组。
注意:这个解析器的复杂性与YAML文件的结构有关。对于YAML文件的每个结构化组件,都需要一个单独的子例程。高度结构化的YAML文件可能需要更复杂的方法,例如通用的递归下降解析器。
圣诞节。yaml文件:
# Xmas YAML example
---
# Values
pear-tree: partridge
turtle-doves: 2.718
french-hens: 3
# Array
calling-birds:
- huey
- dewey
- louie
- fred
# Structure
xmas-fifth-day:
calling-birds: four
french-hens: 3
golden-rings: 5
partridges:
count: 1
location: "a pear tree"
turtle-doves: two
解析器使用mapfile将文件作为数组读入内存,然后循环遍历每个标记并创建环境变量。
梨树、斑鸠和法国母鸡:最终成为简单的环境变量 呼叫鸟:变成一个数组 xmas-fifth-day:结构被表示为一个关联数组,但是如果您没有使用Bash 4.0或更高版本,您可以将这些数组编码为环境变量。 注释和空白将被忽略。
#!/bin/bash
# -------------------------------------------------------------------
# A simple parser for the xmas.yaml file
# -------------------------------------------------------------------
#
# xmas.yaml tags
# # - Ignored
# - Blank lines are ignored
# --- - Initialiser for days-of-xmas
# pear-tree: partridge - a string
# turtle-doves: 2.718 - a string, no float type in Bash
# french-hens: 3 - a number
# calling-birds: - an array of strings
# - huey - calling-birds[0]
# - dewey
# - louie
# - fred
# xmas-fifth-day: - an associative array
# calling-birds: four - a string
# french-hens: 3 - a number
# golden-rings: 5 - a number
# partridges: - changes the key to partridges.xxx
# count: 1 - a number
# location: "a pear tree" - a string
# turtle-doves: two - a string
#
# This requires the following routines
# ParseXMAS
# parses #, ---, blank line
# unexpected tag error
# calls days-of-xmas
#
# days-of-xmas
# parses pear-tree, turtle-doves, french-hens
# calls calling-birds
# calls xmas-fifth-day
#
# calling-birds
# elements of the array
#
# xmas-fifth-day
# parses calling-birds, french-hens, golden-rings, turtle-doves
# calls partridges
#
# partridges
# parses partridges.count, partridges.location
#
function ParseXMAS()
{
# days-of-xmas
# parses pear-tree, turtle-doves, french-hens
# calls calling-birds
# calls xmas-fifth-day
#
function days-of-xmas()
{
unset PearTree TurtleDoves FrenchHens
while [ $CURRENT_ROW -lt $ROWS ]
do
LINE=( ${CONFIG[${CURRENT_ROW}]} )
TAG=${LINE[0]}
unset LINE[0]
VALUE="${LINE[*]}"
echo " days-of-xmas[${CURRENT_ROW}] ${TAG}=${VALUE}"
if [ "$TAG" = "pear-tree:" ]
then
declare -g PearTree=$VALUE
elif [ "$TAG" = "turtle-doves:" ]
then
declare -g TurtleDoves=$VALUE
elif [ "$TAG" = "french-hens:" ]
then
declare -g FrenchHens=$VALUE
elif [ "$TAG" = "calling-birds:" ]
then
let CURRENT_ROW=$(($CURRENT_ROW + 1))
calling-birds
continue
elif [ "$TAG" = "xmas-fifth-day:" ]
then
let CURRENT_ROW=$(($CURRENT_ROW + 1))
xmas-fifth-day
continue
elif [ -z "$TAG" ] || [ "$TAG" = "#" ]
then
# Ignore comments and blank lines
true
else
# time to bug out
break
fi
let CURRENT_ROW=$(($CURRENT_ROW + 1))
done
}
# calling-birds
# elements of the array
function calling-birds()
{
unset CallingBirds
declare -ag CallingBirds
while [ $CURRENT_ROW -lt $ROWS ]
do
LINE=( ${CONFIG[${CURRENT_ROW}]} )
TAG=${LINE[0]}
unset LINE[0]
VALUE="${LINE[*]}"
echo " calling-birds[${CURRENT_ROW}] ${TAG}=${VALUE}"
if [ "$TAG" = "-" ]
then
CallingBirds[${#CallingBirds[*]}]=$VALUE
elif [ -z "$TAG" ] || [ "$TAG" = "#" ]
then
# Ignore comments and blank lines
true
else
# time to bug out
break
fi
let CURRENT_ROW=$(($CURRENT_ROW + 1))
done
}
# xmas-fifth-day
# parses calling-birds, french-hens, golden-rings, turtle-doves
# calls fifth-day-partridges
#
function xmas-fifth-day()
{
unset XmasFifthDay
declare -Ag XmasFifthDay
while [ $CURRENT_ROW -lt $ROWS ]
do
LINE=( ${CONFIG[${CURRENT_ROW}]} )
TAG=${LINE[0]}
unset LINE[0]
VALUE="${LINE[*]}"
echo " xmas-fifth-day[${CURRENT_ROW}] ${TAG}=${VALUE}"
if [ "$TAG" = "calling-birds:" ]
then
XmasFifthDay[CallingBirds]=$VALUE
elif [ "$TAG" = "french-hens:" ]
then
XmasFifthDay[FrenchHens]=$VALUE
elif [ "$TAG" = "golden-rings:" ]
then
XmasFifthDay[GOLDEN-RINGS]=$VALUE
elif [ "$TAG" = "turtle-doves:" ]
then
XmasFifthDay[TurtleDoves]=$VALUE
elif [ "$TAG" = "partridges:" ]
then
let CURRENT_ROW=$(($CURRENT_ROW + 1))
partridges
continue
elif [ -z "$TAG" ] || [ "$TAG" = "#" ]
then
# Ignore comments and blank lines
true
else
# time to bug out
break
fi
let CURRENT_ROW=$(($CURRENT_ROW + 1))
done
}
function partridges()
{
while [ $CURRENT_ROW -lt $ROWS ]
do
LINE=( ${CONFIG[${CURRENT_ROW}]} )
TAG=${LINE[0]}
unset LINE[0]
VALUE="${LINE[*]}"
echo " partridges[${CURRENT_ROW}] ${TAG}=${VALUE}"
if [ "$TAG" = "count:" ]
then
XmasFifthDay[PARTRIDGES.COUNT]=$VALUE
elif [ "$TAG" = "location:" ]
then
XmasFifthDay[PARTRIDGES.LOCATION]=$VALUE
elif [ -z "$TAG" ] || [ "$TAG" = "#" ]
then
# Ignore comments and blank lines
true
else
# time to bug out
break
fi
let CURRENT_ROW=$(($CURRENT_ROW + 1))
done
}
# ===================================================================
# Load the configuration file
mapfile CONFIG < xmas.yaml
let ROWS=${#CONFIG[@]}
let CURRENT_ROW=0
# +
# #
#
# ---
# -
while [ $CURRENT_ROW -lt $ROWS ]
do
LINE=( ${CONFIG[${CURRENT_ROW}]} )
TAG=${LINE[0]}
unset LINE[0]
VALUE="${LINE[*]}"
echo "[${CURRENT_ROW}] ${TAG}=${VALUE}"
if [ "$TAG" = "---" ]
then
let CURRENT_ROW=$(($CURRENT_ROW + 1))
days-of-xmas
continue
elif [ -z "$TAG" ] || [ "$TAG" = "#" ]
then
# Ignore comments and blank lines
true
else
echo "Unexpected tag at line $(($CURRENT_ROW + 1)): <${TAG}>={${VALUE}}"
break
fi
let CURRENT_ROW=$(($CURRENT_ROW + 1))
done
}
echo =========================================
ParseXMAS
echo =========================================
declare -p PearTree
declare -p TurtleDoves
declare -p FrenchHens
declare -p CallingBirds
declare -p XmasFifthDay
这将产生以下输出
=========================================
[0] #=Xmas YAML example
[1] ---=
days-of-xmas[2] #=Values
days-of-xmas[3] pear-tree:=partridge
days-of-xmas[4] turtle-doves:=2.718
days-of-xmas[5] french-hens:=3
days-of-xmas[6] =
days-of-xmas[7] #=Array
days-of-xmas[8] calling-birds:=
calling-birds[9] -=huey
calling-birds[10] -=dewey
calling-birds[11] -=louie
calling-birds[12] -=fred
calling-birds[13] =
calling-birds[14] #=Structure
calling-birds[15] xmas-fifth-day:=
days-of-xmas[15] xmas-fifth-day:=
xmas-fifth-day[16] calling-birds:=four
xmas-fifth-day[17] french-hens:=3
xmas-fifth-day[18] golden-rings:=5
xmas-fifth-day[19] partridges:=
partridges[20] count:=1
partridges[21] location:="a pear tree"
partridges[22] turtle-doves:=two
xmas-fifth-day[22] turtle-doves:=two
=========================================
declare -- PearTree="partridge"
declare -- TurtleDoves="2.718"
declare -- FrenchHens="3"
declare -a CallingBirds=([0]="huey" [1]="dewey" [2]="louie" [3]="fred")
declare -A XmasFifthDay=([CallingBirds]="four" [PARTRIDGES.LOCATION]="\"a pear tree\"" [FrenchHens]="3" [GOLDEN-RINGS]="5" [PARTRIDGES.COUNT]="1" [TurtleDoves]="two" )
推荐文章
- 如何从查找“类型d”中排除此/ current / dot文件夹
- 只使用md5sum获取哈希值(没有文件名)
- 使用sh shell比较字符串
- 在Bash中测试非零长度字符串:[-n "$var"]或["$var"]
- 如何创建Bash别名?
- 如何设置ssh超时时间?
- 将所有变量从一个shell脚本传递到另一个?
- 只列出UNIX中的目录
- 如何删除shell脚本中文件名的扩展名?
- 'find -exec'是Linux中的shell函数
- 临时更改bash中的当前工作目录以运行命令
- 重定向复制的标准输出到日志文件从bash脚本本身
- Shell脚本for循环语法
- Docker入口点运行bash脚本被“拒绝权限”
- Chmod递归