我希望提供一个结构化的配置文件,它对于非技术用户来说尽可能容易编辑(不幸的是它必须是一个文件),所以我想使用YAML。然而,我找不到任何方法从Unix shell脚本解析这个。


当前回答

很难说,因为这取决于您希望解析器从YAML文档中提取什么。对于简单的情况,你可以使用grep、cut、awk等。对于更复杂的解析,您需要使用成熟的解析库,如Python的PyYAML或YAML::Perl。

其他回答

我已经用python编写了shyaml,用于从shell命令行查询YAML。

概述:

$ pip install shyaml      ## installation

示例的YAML文件(具有复杂的功能):

$ cat <<EOF > test.yaml
name: "MyName !!"
subvalue:
    how-much: 1.1
    things:
        - first
        - second
        - third
    other-things: [a, b, c]
    maintainer: "Valentin Lab"
    description: |
        Multiline description:
        Line 1
        Line 2
EOF

基本的查询:

$ cat test.yaml | shyaml get-value subvalue.maintainer
Valentin Lab

更复杂的循环查询复杂的值:

$ cat test.yaml | shyaml values-0 | \
  while read -r -d $'\0' value; do
      echo "RECEIVED: '$value'"
  done
RECEIVED: '1.1'
RECEIVED: '- first
- second
- third'
RECEIVED: '2'
RECEIVED: 'Valentin Lab'
RECEIVED: 'Multiline description:
Line 1
Line 2'

以下几个要点:

all YAML types and syntax oddities are correctly handled, as multiline, quoted strings, inline sequences... \0 padded output is available for solid multiline entry manipulation. simple dotted notation to select sub-values (ie: subvalue.maintainer is a valid key). access by index is provided to sequences (ie: subvalue.things.-1 is the last element of the subvalue.things sequence.) access to all sequence/structs elements in one go for use in bash loops. you can output whole subpart of a YAML file as ... YAML, which blend well for further manipulations with shyaml.

更多的示例和文档可以在shyaml github页面或shyaml PyPI页面上找到。

我的用例可能与这篇原始文章所要求的完全相同,也可能不完全相同,但它肯定是相似的。

我需要拉一些YAML作为bash变量。YAML的深度永远不会超过一层。

YAML看起来是这样的:

KEY:                value
ANOTHER_KEY:        another_value
OH_MY_SO_MANY_KEYS: yet_another_value
LAST_KEY:           last_value

输出如下:

KEY="value"
ANOTHER_KEY="another_value"
OH_MY_SO_MANY_KEYS="yet_another_value"
LAST_KEY="last_value"

我用这一行实现了输出:

sed -e 's/:[^:\/\/]/="/g;s/$/"/g;s/ *=/=/g' file.yaml > file.sh

s/:[^:\/\/]/="/g查找:并将其替换为=",同时忽略://(对于url) S /$/"/g将"附加到每一行的末尾 S / *=/=/g删除=前面的所有空格

把我的答案从如何在bash中将json响应转换为yaml,因为这似乎是关于从命令行处理yaml文本解析的权威帖子。

我想添加一些关于yq YAML实现的细节。由于这个YAML解析器有两种实现,名称都是yq,如果不查看实现的DSL,就很难区分使用的是哪一种。有两个可用的实现

kislyuk/yq——更常被提及的版本,它是jq的包装器,用Python编写,使用PyYAML库进行YAML解析 mikefarah/yq -一个Go实现,使用Go -yaml v3解析器,有自己的动态DSL。

几乎所有主要发行版都可以通过标准安装包管理器进行安装

kislyuk/yq -安装说明 mikefarah/yq -安装说明

这两个版本都有一些优点和缺点,但有一些有效的点需要强调(从他们的回购指令中采用)

kislyuk - yq

Since the DSL is the adopted completely from jq, for users familiar with the latter, the parsing and manipulation becomes quite straightforward Supports mode to preserve YAML tags and styles, but loses comments during the conversion. Since jq doesn't preserve comments, during the round-trip conversion, the comments are lost. As part of the package, XML support is built in. An executable, xq, which transcodes XML to JSON using xmltodict and pipes it to jq, on which you can apply the same DSL to perform CRUD operations on the objects and round-trip the output back to XML. Supports in-place edit mode with -i flag (similar to sed -i)

迈克法拉/YQ

Prone to frequent changes in DSL, migration from 2.x - 3.x Rich support for anchors, styles and tags. But lookout for bugs once in a while A relatively simple Path expression syntax to navigate and match yaml nodes Supports YAML->JSON, JSON->YAML formatting and pretty printing YAML (with comments) Supports in-place edit mode with -i flag (similar to sed -i) Supports coloring the output YAML with -C flag (not applicable for JSON output) and indentation of the sub elements (default at 2 spaces) Supports Shell completion for most shells - Bash, zsh (because of powerful support from spf13/cobra used to generate CLI flags)


我对以下两个版本的YAML的看法(在其他答案中也有引用)

root_key1: this is value one
root_key2: "this is value two"

drink:
  state: liquid
  coffee:
    best_served: hot
    colour: brown
  orange_juice:
    best_served: cold
    colour: orange

food:
  state: solid
  apple_pie:
    best_served: warm

root_key_3: this is value three

对这两个实现执行的各种操作(一些常用操作)

修改根节点值—修改“root_key2”的值 修改数组内容,增加值-为coffee添加属性 修改数组内容,删除value - Delete属性从orange_juice 打印带有路径的键/值对—用于food下的所有项目

使用kislyuk / yq

Yq -y '。Root_key2 |= "this is a new value 你,你,喝。咖啡+={时间:"always"}' yaml Yq -y 'del(.drink.orange_juice.colour)' yaml yq - r的.food |路径(标量)美元p | (($ p |加入(“。”)),(getpath ($ p) | tojson)] | @tsv的yaml

这很简单。你所需要做的就是用-y标志将jq JSON输出转码回YAML。

用mikefarah - yq

Yq w yaml root_key2 "这是一个新值" Yq w yaml喝。咖啡。时间“总是” Yq d yaml饮料。橙汁。颜色 yq r yaml——printMode pv "food.**"


截至2020年12月21日,yq v4是测试版,支持许多强大的路径表达式,并支持类似于使用jq的DSL。阅读过渡说明-从V3升级

yq是一个轻量级、可移植的命令行YAML处理器

这个项目的目标是yaml文件的jq或sed。

(https://github.com/mikefarah/yq #自述)

作为示例(直接从文档中窃取),给出一个示例。Yaml文件:

---
bob:
  item1:
    cats: bananas
  item2:
    cats: apples

then

yq eval '.bob.*.cats' sample.yaml

将输出

- bananas
- apples

下面是一个bash-only解析器,利用sed和awk来解析简单的yaml文件:

function parse_yaml {
   local prefix=$2
   local s='[[:space:]]*' w='[a-zA-Z0-9_]*' fs=$(echo @|tr @ '\034')
   sed -ne "s|^\($s\):|\1|" \
        -e "s|^\($s\)\($w\)$s:$s[\"']\(.*\)[\"']$s\$|\1$fs\2$fs\3|p" \
        -e "s|^\($s\)\($w\)$s:$s\(.*\)$s\$|\1$fs\2$fs\3|p"  $1 |
   awk -F$fs '{
      indent = length($1)/2;
      vname[indent] = $2;
      for (i in vname) {if (i > indent) {delete vname[i]}}
      if (length($3) > 0) {
         vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
         printf("%s%s%s=\"%s\"\n", "'$prefix'",vn, $2, $3);
      }
   }'
}

它可以理解以下文件:

## global definitions
global:
  debug: yes
  verbose: no
  debugging:
    detailed: no
    header: "debugging started"

## output
output:
   file: "yes"

在解析时使用:

parse_yaml sample.yml

将输出:

global_debug="yes"
global_verbose="no"
global_debugging_detailed="no"
global_debugging_header="debugging started"
output_file="yes"

它也理解由ruby生成的yaml文件,其中可能包含ruby符号,例如:

---
:global:
  :debug: 'yes'
  :verbose: 'no'
  :debugging:
    :detailed: 'no'
    :header: debugging started
  :output: 'yes'

并将输出与前一个示例相同的结果。

脚本中的典型用法是:

eval $(parse_yaml sample.yml)

Parse_yaml接受一个前缀参数,这样导入的所有设置都有一个公共前缀(这将减少名称空间冲突的风险)。

parse_yaml sample.yml "CONF_"

收益率:

CONF_global_debug="yes"
CONF_global_verbose="no"
CONF_global_debugging_detailed="no"
CONF_global_debugging_header="debugging started"
CONF_output_file="yes"

注意,之前文件中的设置可以被后面的设置引用:

## global definitions
global:
  debug: yes
  verbose: no
  debugging:
    detailed: no
    header: "debugging started"

## output
output:
   debug: $global_debug

另一个很好的用法是先解析默认文件,然后解析用户设置,这是可行的,因为后一个设置会覆盖第一个设置:

eval $(parse_yaml defaults.yml)
eval $(parse_yaml project.yml)