我想知道Python中是否有用于异步方法调用的库。如果你能做点什么就太好了
@async
def longComputation():
<code>
token = longComputation()
token.registerCallback(callback_function)
# alternative, polling
while not token.finished():
doSomethingElse()
if token.finished():
result = token.result()
或者异步调用非异步例程
def longComputation()
<code>
token = asynccall(longComputation())
如果在语言核心中有一个更精细的策略就太好了。考虑过这个问题吗?
当前回答
您可以实现一个装饰器来使您的函数异步，尽管这有点棘手。多处理模块充满了小怪癖和看似任意的限制——尽管如此，更有理由将它封装在友好的界面后面。
from inspect import getmodule
from multiprocessing import Pool
def async(decorated):
r'''Wraps a top-level function around an asynchronous dispatcher.
when the decorated function is called, a task is submitted to a
process pool, and a future object is returned, providing access to an
eventual return value.
The future object has a blocking get() method to access the task
result: it will return immediately if the job is already done, or block
until it completes.
This decorator won't work on methods, due to limitations in Python's
pickling machinery (in principle methods could be made pickleable, but
good luck on that).
'''
# Keeps the original function visible from the module global namespace,
# under a name consistent to its __name__ attribute. This is necessary for
# the multiprocessing pickling machinery to work properly.
module = getmodule(decorated)
decorated.__name__ += '_original'
setattr(module, decorated.__name__, decorated)
def send(*args, **opts):
return async.pool.apply_async(decorated, args, opts)
return send
下面的代码说明了装饰器的用法:
@async
def printsum(uid, values):
summed = 0
for value in values:
summed += value
print("Worker %i: sum value is %i" % (uid, summed))
return (uid, summed)
if __name__ == '__main__':
from random import sample
# The process pool must be created inside __main__.
async.pool = Pool(4)
p = range(0, 1000)
results = []
for i in range(4):
result = printsum(i, sample(p, 100))
results.append(result)
for result in results:
print("Worker %i: sum value is %i" % result.get())
在实际的情况下，我将详细介绍装饰器，提供一些方法来关闭它以进行调试(同时保持未来的接口在适当的位置)，或者可能是处理异常的工具;但我认为这充分说明了原理。
其他回答
你可以使用eventlet。它允许您编写看似同步的代码，但却可以在网络上异步操作。
下面是一个超级小爬虫的例子:
urls = ["http://www.google.com/intl/en_ALL/images/logo.gif",
"https://wiki.secondlife.com/w/images/secondlife.jpg",
"http://us.i1.yimg.com/us.yimg.com/i/ww/beta/y3.gif"]
import eventlet
from eventlet.green import urllib2
def fetch(url):
return urllib2.urlopen(url).read()
pool = eventlet.GreenPool()
for body in pool.imap(fetch, urls):
print "got body", len(body)
喜欢的东西:
import threading
thr = threading.Thread(target=foo, args=(), kwargs={})
thr.start() # Will run "foo"
....
thr.is_alive() # Will return whether foo is running currently
....
thr.join() # Will wait till "foo" is done
有关详细信息，请参阅https://docs.python.org/library/threading.html上的文档。
Python 3.7及以后版本中更新的asyncio运行方法是使用asyncio.run()，而不是创建循环并调用loop.run_until_complete()并关闭它:
import asyncio
import datetime
async def display_date(delay):
loop = asyncio.get_running_loop()
end_time = loop.time() + delay
while True:
print("Blocking...", datetime.datetime.now())
await asyncio.sleep(1)
if loop.time() > end_time:
print("Done.")
break
asyncio.run(display_date(5))
它不在语言核心中，但Twisted是一个非常成熟的库，可以做你想要的事情。它引入Deferred对象，您可以将回调或错误处理程序(“errbacks”)附加到该对象。Deferred基本上是一个“承诺”，即一个函数最终会有一个结果。
您可以实现一个装饰器来使您的函数异步，尽管这有点棘手。多处理模块充满了小怪癖和看似任意的限制——尽管如此，更有理由将它封装在友好的界面后面。
from inspect import getmodule
from multiprocessing import Pool
def async(decorated):
r'''Wraps a top-level function around an asynchronous dispatcher.
when the decorated function is called, a task is submitted to a
process pool, and a future object is returned, providing access to an
eventual return value.
The future object has a blocking get() method to access the task
result: it will return immediately if the job is already done, or block
until it completes.
This decorator won't work on methods, due to limitations in Python's
pickling machinery (in principle methods could be made pickleable, but
good luck on that).
'''
# Keeps the original function visible from the module global namespace,
# under a name consistent to its __name__ attribute. This is necessary for
# the multiprocessing pickling machinery to work properly.
module = getmodule(decorated)
decorated.__name__ += '_original'
setattr(module, decorated.__name__, decorated)
def send(*args, **opts):
return async.pool.apply_async(decorated, args, opts)
return send
下面的代码说明了装饰器的用法:
@async
def printsum(uid, values):
summed = 0
for value in values:
summed += value
print("Worker %i: sum value is %i" % (uid, summed))
return (uid, summed)
if __name__ == '__main__':
from random import sample
# The process pool must be created inside __main__.
async.pool = Pool(4)
p = range(0, 1000)
results = []
for i in range(4):
result = printsum(i, sample(p, 100))
results.append(result)
for result in results:
print("Worker %i: sum value is %i" % result.get())
在实际的情况下，我将详细介绍装饰器，提供一些方法来关闭它以进行调试(同时保持未来的接口在适当的位置)，或者可能是处理异常的工具;但我认为这充分说明了原理。
