我目前通过一堆不好记的AWK、sed、Bash和一小部分Perl来做我的文本文件操作。

我已经看到提到的一些地方,python很适合做这种事情。我如何使用Python来代替shell脚本,AWK, sed和朋友?


当前回答

在ShellPy库中可以使用python而不是bash。

下面是一个从Github下载Python用户头像的例子:

import json
import os
import tempfile

# get the api answer with curl
answer = `curl https://api.github.com/users/python
# syntactic sugar for checking returncode of executed process for zero
if answer:
    answer_json = json.loads(answer.stdout)
    avatar_url = answer_json['avatar_url']

    destination = os.path.join(tempfile.gettempdir(), 'python.png')

    # execute curl once again, this time to get the image
    result = `curl {avatar_url} > {destination}
    if result:
        # if there were no problems show the file
        p`ls -l {destination}
    else:
        print('Failed to download avatar')

    print('Avatar downloaded')
else:
    print('Failed to access github api')

如您所见,所有在grave重音(')符号内的表达式都在shell中执行。在Python代码中,您可以捕获此执行的结果并对其执行操作。例如:

log = `git log --pretty=oneline --grep='Create'

这一行首先在shell中执行git log——pretty=oneline——grep='Create',然后将结果赋值给log变量。结果具有以下属性:

从已执行进程的Stdout中Stdout整个文本

Stderr从已执行进程的Stderr得到的整个文本

Returncode执行的返回码

这是该库的总体概述,更详细的描述和示例可以在这里找到。

其他回答

最好的选择是专门针对您的问题的工具。如果它正在处理文本文件,那么Sed、Awk和Perl是最佳竞争者。Python是一种通用动态语言。与任何通用语言一样,它支持文件操作,但这不是它的核心目的。如果我特别需要一种动态语言,我会考虑Python或Ruby。

简而言之,非常好地学习Sed和Awk,以及所有其他随*nix风格而来的好东西(所有Bash内置,grep, tr等等)。如果您感兴趣的是文本文件处理,那么您已经使用了正确的方法。

In the beginning there was sh, sed, and awk (and find, and grep, and...). It was good. But awk can be an odd little beast and hard to remember if you don't use it often. Then the great camel created Perl. Perl was a system administrator's dream. It was like shell scripting on steroids. Text processing, including regular expressions were just part of the language. Then it got ugly... People tried to make big applications with Perl. Now, don't get me wrong, Perl can be an application, but it can (can!) look like a mess if you're not really careful. Then there is all this flat data business. It's enough to drive a programmer nuts.

进入Python、Ruby等。这些都是非常好的通用语言。它们支持文本处理,并且做得很好(尽管可能与语言的基本核心没有那么紧密地交织在一起)。但它们也可以很好地扩展,并且在一天结束时仍然有漂亮的代码。他们还发展了相当庞大的社区,有大量的图书馆,几乎可以提供任何东西。

Now, much of the negativeness towards Perl is a matter of opinion, and certainly some people can write very clean Perl, but with this many people complaining about it being too easy to create obfuscated code, you know some grain of truth is there. The question really becomes then, are you ever going to use this language for more than simple bash script replacements. If not, learn some more Perl.. it is absolutely fantastic for that. If, on the other hand, you want a language that will grow with you as you want to do more, may I suggest Python or Ruby.

不管怎样,祝你好运!

在ShellPy库中可以使用python而不是bash。

下面是一个从Github下载Python用户头像的例子:

import json
import os
import tempfile

# get the api answer with curl
answer = `curl https://api.github.com/users/python
# syntactic sugar for checking returncode of executed process for zero
if answer:
    answer_json = json.loads(answer.stdout)
    avatar_url = answer_json['avatar_url']

    destination = os.path.join(tempfile.gettempdir(), 'python.png')

    # execute curl once again, this time to get the image
    result = `curl {avatar_url} > {destination}
    if result:
        # if there were no problems show the file
        p`ls -l {destination}
    else:
        print('Failed to download avatar')

    print('Avatar downloaded')
else:
    print('Failed to access github api')

如您所见,所有在grave重音(')符号内的表达式都在shell中执行。在Python代码中,您可以捕获此执行的结果并对其执行操作。例如:

log = `git log --pretty=oneline --grep='Create'

这一行首先在shell中执行git log——pretty=oneline——grep='Create',然后将结果赋值给log变量。结果具有以下属性:

从已执行进程的Stdout中Stdout整个文本

Stderr从已执行进程的Stderr得到的整个文本

Returncode执行的返回码

这是该库的总体概述,更详细的描述和示例可以在这里找到。

我刚刚发现了如何结合bash和ipython的最佳部分。到目前为止,这似乎比使用subprocess等更舒服。你可以很容易地复制现有bash脚本的大部分内容,例如以python的方式添加错误处理:) 这是我的结果:

#!/usr/bin/env ipython3

# *** How to have the most comfort scripting experience of your life ***
# ######################################################################
#
# … by using ipython for scripting combined with subcommands from bash!
#
# 1. echo "#!/usr/bin/env ipython3" > scriptname.ipy    # creates new ipy-file
#
# 2. chmod +x scriptname.ipy                            # make in executable
#
# 3. starting with line 2, write normal python or do some of
#    the ! magic of ipython, so that you can use unix commands
#    within python and even assign their output to a variable via
#    var = !cmd1 | cmd2 | cmd3                          # enjoy ;)
#
# 4. run via ./scriptname.ipy - if it fails with recognizing % and !
#    but parses raw python fine, please check again for the .ipy suffix

# ugly example, please go and find more in the wild
files = !ls *.* | grep "y"
for file in files:
  !echo $file | grep "p"
# sorry for this nonsense example ;)

请参阅IPython文档,了解系统shell命令并将其用作系统shell。

是的,当然。

看看这些库,它们可以帮助你不再编写shell脚本(Plumbum的座右铭)。

铅 军士 上海

另外,如果你想用基于Python的东西替换awk, sed和grep,那么我推荐pyp -

“The Pyed Piper”,或pyp,是一个linux命令行文本操作 工具类似于awk或sed,但使用标准的python字符串和 列表方法以及自定义函数进化为快速生成 在紧张的生产环境中产生的结果。