cat a.txt | xargs -I % echo %

在上面的例子中,xargs使用echo %作为命令参数。但在某些情况下,我需要多个命令来处理参数,而不是一个。例如:

cat a.txt | xargs -I % {command1; command2; ... }

但是xargs不接受这种形式。我知道的一种解决方案是,我可以定义一个函数来包装命令,但我想避免这样做,因为它很复杂。有没有更好的解决方案?


当前回答

cat a.txt | xargs -d $'\n' sh -c 'for arg do command1 "$arg"; command2 "$arg"; ...; done' _

...或者,不用无用地使用cat:

<a.txt xargs -d $'\n' sh -c 'for arg do command1 "$arg"; command2 "$arg"; ...; done' _

来解释一些细节:

The use of "$arg" instead of % (and the absence of -I in the xargs command line) is for security reasons: Passing data on sh's command-line argument list instead of substituting it into code prevents content that data might contain (such as $(rm -rf ~), to take a particularly malicious example) from being executed as code. Similarly, the use of -d $'\n' is a GNU extension which causes xargs to treat each line of the input file as a separate data item. Either this or -0 (which expects NULs instead of newlines) is necessary to prevent xargs from trying to apply shell-like (but not quite shell-compatible) parsing to the stream it reads. (If you don't have GNU xargs, you can use tr '\n' '\0' <a.txt | xargs -0 ... to get line-oriented reading without -d). The _ is a placeholder for $0, such that other data values added by xargs become $1 and onward, which happens to be the default set of values a for loop iterates over.

其他回答

我目前的BKM是

... | xargs -n1 -I % perl -e 'system("echo 1 %"); system("echo 2 %");'

不幸的是,这使用perl,它不太可能安装比bash;但是它处理的输入比接受的答案要多。(我欢迎不依赖perl的普遍版本。)

@KeithThompson的建议

 ... | xargs -I % sh -c 'command1; command2; ...'

很好-除非您的输入中有shell注释字符#,在这种情况下,第一个命令的部分和第二个命令的全部将被截断。

如果输入来自文件系统列表,例如ls或find,并且您的编辑器创建了名称为#的临时文件,则哈希值#可能非常常见。

问题示例:

$ bash 1366 $>  /bin/ls | cat
#Makefile#
#README#
Makefile
README

哎呀,问题来了:

$ bash 1367 $>  ls | xargs -n1 -I % sh -i -c 'echo 1 %; echo 2 %'
1
1
1
1 Makefile
2 Makefile
1 README
2 README

啊,这样好多了:

$ bash 1368 $>  ls | xargs -n1 -I % perl -e 'system("echo 1 %"); system("echo 2 %");'
1 #Makefile#
2 #Makefile#
1 #README#
2 #README#
1 Makefile
2 Makefile
1 README
2 README
$ bash 1369 $>  
cat a.txt | xargs -d $'\n' sh -c 'for arg do command1 "$arg"; command2 "$arg"; ...; done' _

...或者,不用无用地使用cat:

<a.txt xargs -d $'\n' sh -c 'for arg do command1 "$arg"; command2 "$arg"; ...; done' _

来解释一些细节:

The use of "$arg" instead of % (and the absence of -I in the xargs command line) is for security reasons: Passing data on sh's command-line argument list instead of substituting it into code prevents content that data might contain (such as $(rm -rf ~), to take a particularly malicious example) from being executed as code. Similarly, the use of -d $'\n' is a GNU extension which causes xargs to treat each line of the input file as a separate data item. Either this or -0 (which expects NULs instead of newlines) is necessary to prevent xargs from trying to apply shell-like (but not quite shell-compatible) parsing to the stream it reads. (If you don't have GNU xargs, you can use tr '\n' '\0' <a.txt | xargs -0 ... to get line-oriented reading without -d). The _ is a placeholder for $0, such that other data values added by xargs become $1 and onward, which happens to be the default set of values a for loop iterates over.

这是另一种没有xargs和cat的方法:

while read stuff; do
  command1 "$stuff"
  command2 "$stuff"
  ...
done < a.txt

我更喜欢允许试运行模式的样式(没有| sh):

cat a.txt | xargs -I % echo "command1; command2; ... " | sh

管道也适用:

cat a.txt | xargs -I % echo "echo % | cat " | sh

这似乎是最安全的版本。

tr '[\n]' '[\0]' < a.txt | xargs -r0 /bin/bash -c 'command1 "$@"; command2 "$@";' ''

(可以删除-0,并将tr替换为重定向(或者可以将文件替换为空分隔文件)。它主要是在那里,因为我主要使用xargs查找与-print0输出)(这可能也与xargs版本没有-0扩展名有关)

这是安全的,因为args在执行时将参数作为数组传递给shell。当使用["$@"][1]获得所有这些进程时,shell(至少是bash)会将它们作为未更改的数组传递给其他进程

如果你用…| xargs -r0 -I{} bash -c 'f="{}";命令"$f";' ",如果字符串包含双引号,则赋值将失败。这对于每个使用-i或-i的变体都是正确的。(由于它被替换为字符串,您总是可以通过在输入数据中插入意外字符(如引号、反引号或美元符号)来注入命令)

如果命令一次只能接受一个参数:

tr '[\n]' '[\0]' < a.txt | xargs -r0 -n1 /bin/bash -c 'command1 "$@"; command2 "$@";' ''

或者用更少的进程:

tr '[\n]' '[\0]' < a.txt | xargs -r0 /bin/bash -c 'for f in "$@"; do command1 "$f"; command2 "$f"; done;' ''

如果您有GNU xargs或其他带有-P扩展名的文件,并且希望并行运行32个进程,每个进程每个命令的参数不超过10个:

tr '[\n]' '[\0]' < a.txt | xargs -r0 -n10 -P32 /bin/bash -c 'command1 "$@"; command2 "$@";' ''

这对于输入中的任何特殊字符都应该是健壮的。(如果输入是空分隔的。)如果某些行包含换行符,tr版本将得到一些无效的输入,但对于换行符分隔的文件,这是不可避免的。

bash -c的第一个空白参数是由于:(来自bash手册页)(感谢@clacke)

-c   If the -c option is present, then  commands  are  read  from  the  first  non-option  argument  com‐
     mand_string.   If there are arguments after the command_string, the first argument is assigned to $0
     and any remaining arguments are assigned to the positional parameters.  The assignment  to  $0  sets
     the name of the shell, which is used in warning and error messages.