我希望提供一个结构化的配置文件,它对于非技术用户来说尽可能容易编辑(不幸的是它必须是一个文件),所以我想使用YAML。然而,我找不到任何方法从Unix shell脚本解析这个。


当前回答

perl -ne 'chomp; printf qq/%s="%s"\n/, split(/\s*:\s*/,$_,2)' file.yml > file.sh

其他回答

很难说,因为这取决于您希望解析器从YAML文档中提取什么。对于简单的情况,你可以使用grep、cut、awk等。对于更复杂的解析,您需要使用成熟的解析库,如Python的PyYAML或YAML::Perl。

以下是Stefan Farestam回答的扩展版本:

function parse_yaml {
   local prefix=$2
   local s='[[:space:]]*' w='[a-zA-Z0-9_]*' fs=$(echo @|tr @ '\034')
   sed -ne "s|,$s\]$s\$|]|" \
        -e ":1;s|^\($s\)\($w\)$s:$s\[$s\(.*\)$s,$s\(.*\)$s\]|\1\2: [\3]\n\1  - \4|;t1" \
        -e "s|^\($s\)\($w\)$s:$s\[$s\(.*\)$s\]|\1\2:\n\1  - \3|;p" $1 | \
   sed -ne "s|,$s}$s\$|}|" \
        -e ":1;s|^\($s\)-$s{$s\(.*\)$s,$s\($w\)$s:$s\(.*\)$s}|\1- {\2}\n\1  \3: \4|;t1" \
        -e    "s|^\($s\)-$s{$s\(.*\)$s}|\1-\n\1  \2|;p" | \
   sed -ne "s|^\($s\):|\1|" \
        -e "s|^\($s\)-$s[\"']\(.*\)[\"']$s\$|\1$fs$fs\2|p" \
        -e "s|^\($s\)-$s\(.*\)$s\$|\1$fs$fs\2|p" \
        -e "s|^\($s\)\($w\)$s:$s[\"']\(.*\)[\"']$s\$|\1$fs\2$fs\3|p" \
        -e "s|^\($s\)\($w\)$s:$s\(.*\)$s\$|\1$fs\2$fs\3|p" | \
   awk -F$fs '{
      indent = length($1)/2;
      vname[indent] = $2;
      for (i in vname) {if (i > indent) {delete vname[i]; idx[i]=0}}
      if(length($2)== 0){  vname[indent]= ++idx[indent] };
      if (length($3) > 0) {
         vn=""; for (i=0; i<indent; i++) { vn=(vn)(vname[i])("_")}
         printf("%s%s%s=\"%s\"\n", "'$prefix'",vn, vname[indent], $3);
      }
   }'
}

该版本支持字典和列表的-符号和短符号。以下输入:

global:
  input:
    - "main.c"
    - "main.h"
  flags: [ "-O3", "-fpic" ]
  sample_input:
    -  { property1: value, property2: "value2" }
    -  { property1: "value3", property2: 'value 4' }

产生如下输出:

global_input_1="main.c"
global_input_2="main.h"
global_flags_1="-O3"
global_flags_2="-fpic"
global_sample_input_1_property1="value"
global_sample_input_1_property2="value2"
global_sample_input_2_property1="value3"
global_sample_input_2_property2="value 4"

as you can see the - items automatically get numbered in order to obtain different variable names for each item. In bash there are no multidimensional arrays, so this is one way to work around. Multiple levels are supported. To work around the problem with trailing white spaces mentioned by @briceburg one should enclose the values in single or double quotes. However, there are still some limitations: Expansion of the dictionaries and lists can produce wrong results when values contain commas. Also, more complex structures like values spanning multiple lines (like ssh-keys) are not (yet) supported.

A few words about the code: The first sed command expands the short form of dictionaries { key: value, ...} to regular and converts them to more simple yaml style. The second sed call does the same for the short notation of lists and converts [ entry, ... ] to an itemized list with the - notation. The third sed call is the original one that handled normal dictionaries, now with the addition to handle lists with - and indentations. The awk part introduces an index for each indentation level and increases it when the variable name is empty (i.e. when processing a list). The current value of the counters are used instead of the empty vname. When going up one level, the counters are zeroed.

编辑:我已经为此创建了一个github存储库。

把我的答案从如何在bash中将json响应转换为yaml,因为这似乎是关于从命令行处理yaml文本解析的权威帖子。

我想添加一些关于yq YAML实现的细节。由于这个YAML解析器有两种实现,名称都是yq,如果不查看实现的DSL,就很难区分使用的是哪一种。有两个可用的实现

kislyuk/yq——更常被提及的版本,它是jq的包装器,用Python编写,使用PyYAML库进行YAML解析 mikefarah/yq -一个Go实现,使用Go -yaml v3解析器,有自己的动态DSL。

几乎所有主要发行版都可以通过标准安装包管理器进行安装

kislyuk/yq -安装说明 mikefarah/yq -安装说明

这两个版本都有一些优点和缺点,但有一些有效的点需要强调(从他们的回购指令中采用)

kislyuk - yq

Since the DSL is the adopted completely from jq, for users familiar with the latter, the parsing and manipulation becomes quite straightforward Supports mode to preserve YAML tags and styles, but loses comments during the conversion. Since jq doesn't preserve comments, during the round-trip conversion, the comments are lost. As part of the package, XML support is built in. An executable, xq, which transcodes XML to JSON using xmltodict and pipes it to jq, on which you can apply the same DSL to perform CRUD operations on the objects and round-trip the output back to XML. Supports in-place edit mode with -i flag (similar to sed -i)

迈克法拉/YQ

Prone to frequent changes in DSL, migration from 2.x - 3.x Rich support for anchors, styles and tags. But lookout for bugs once in a while A relatively simple Path expression syntax to navigate and match yaml nodes Supports YAML->JSON, JSON->YAML formatting and pretty printing YAML (with comments) Supports in-place edit mode with -i flag (similar to sed -i) Supports coloring the output YAML with -C flag (not applicable for JSON output) and indentation of the sub elements (default at 2 spaces) Supports Shell completion for most shells - Bash, zsh (because of powerful support from spf13/cobra used to generate CLI flags)


我对以下两个版本的YAML的看法(在其他答案中也有引用)

root_key1: this is value one
root_key2: "this is value two"

drink:
  state: liquid
  coffee:
    best_served: hot
    colour: brown
  orange_juice:
    best_served: cold
    colour: orange

food:
  state: solid
  apple_pie:
    best_served: warm

root_key_3: this is value three

对这两个实现执行的各种操作(一些常用操作)

修改根节点值—修改“root_key2”的值 修改数组内容,增加值-为coffee添加属性 修改数组内容,删除value - Delete属性从orange_juice 打印带有路径的键/值对—用于food下的所有项目

使用kislyuk / yq

Yq -y '。Root_key2 |= "this is a new value 你,你,喝。咖啡+={时间:"always"}' yaml Yq -y 'del(.drink.orange_juice.colour)' yaml yq - r的.food |路径(标量)美元p | (($ p |加入(“。”)),(getpath ($ p) | tojson)] | @tsv的yaml

这很简单。你所需要做的就是用-y标志将jq JSON输出转码回YAML。

用mikefarah - yq

Yq w yaml root_key2 "这是一个新值" Yq w yaml喝。咖啡。时间“总是” Yq d yaml饮料。橙汁。颜色 yq r yaml——printMode pv "food.**"


截至2020年12月21日,yq v4是测试版,支持许多强大的路径表达式,并支持类似于使用jq的DSL。阅读过渡说明-从V3升级

考虑到Python3和PyYAML是非常容易满足的依赖关系,下面的代码可能会有所帮助:

yaml() {
    python3 -c "import yaml;print(yaml.safe_load(open('$1'))$2)"
}

VALUE=$(yaml ~/my_yaml_file.yaml "['a_key']")

另一种选择是将YAML转换为JSON,然后使用jq与JSON表示进行交互,从其中提取信息或编辑信息。

我写了一个简单的bash脚本,包含这个胶水-见Y2J项目在GitHub上