很久以前，我写了一个web-spider，我用多线程使并发请求同时发生。那是在我年轻时使用Python的时候，那时我还不知道GIL以及它给多线程代码带来的麻烦(IE，大多数时候这些东西最终都被序列化了!)……
我想重做这段代码，使其更健壮，性能更好。我基本上有两种方法可以做到这一点:我可以使用2.6+中新的多处理模块，或者我可以使用某种基于反应堆/事件的模型。我宁愿选择后者，因为它更简单，更不容易出错。
所以这个问题涉及到什么样的框架最适合我的需求。以下是我目前所知道的一些选择:
Twisted: The granddaddy of Python reactor frameworks: seems complex and a bit bloated however. Steep learning curve for a small task. Eventlet: From the guys at lindenlab. Greenlet based framework that's geared towards these kinds of tasks. I had a look at the code though and it's not too pretty: non-pep8 compliant, scattered with prints (why do people do this in a framework!?), API seems a little inconsistent. PyEv: Immature, doesn't seem to be anyone using it right now though it is based on libevent so it's got a solid backend. asyncore: From the stdlib: über low-level, seems like a lot of legwork involved just to get something off the ground. tornado: Though this is a server oriented product designed to server dynamic websites it does feature an async HTTP client and a simple ioloop. Looks like it could get the job done but not what it was intended for. [edit: doesn't run on Windows unfortunately, which counts it out for me - its a requirement for me to support this lame platform]
我还遗漏了什么吗?当然，一定有一个库在那里，适合一个简化异步网络库的甜蜜点!
[edit: big thanks to intgr for his pointer to this page. If you scroll to the bottom you will see there is a really nice list of projects that aim to tackle this task in one way or another. It seems actually that things have indeed moved on since the inception of Twisted: people now seem to favour a co-routine based solution rather than a traditional reactor / callback oriented one. The benefits of this approach are clearer more direct code: I've certainly found in the past, especially when working with boost.asio in C++ that callback based code can lead to designs that can be hard-to-follow and are relatively obscure to the untrained eye. Using co-routines allows you to write code that looks a little more synchronous at least. I guess now my task is to work out which one of these many libraries I like the look of and give it a go! Glad I asked now...]
[编辑:可能对任何关注或无意中发现这个问题或在任何意义上关心这个主题的人都感兴趣:我发现了一篇关于这项工作可用工具的当前状态的非常棒的文章]
