注意:本文假设Python 3。x语法。__

生成器只是一个函数，它返回一个对象，接下来可以对其调用，这样对于每次调用它都会返回一些值，直到引发StopIteration异常，表明所有值都已生成。这样的对象称为迭代器。

普通函数使用return返回单个值，就像在Java中一样。然而，在Python中有一种替代方法，称为yield。在函数的任何地方使用yield使其成为生成器。请注意以下代码:

>>> def myGen(n):
...     yield n
...     yield n + 1
... 
>>> g = myGen(6)
>>> next(g)
6
>>> next(g)
7
>>> next(g)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

正如你所看到的，myGen(n)是一个产生n和n + 1的函数。每次对next的调用都会产生一个值，直到产生所有值为止。For循环在后台调用next，这样:

>>> for n in myGen(6):
...     print(n)
... 
6
7

同样，还有生成器表达式，它提供了一种方法来简洁地描述某些常见类型的生成器:

>>> g = (n for n in range(3, 5))
>>> next(g)
3
>>> next(g)
4
>>> next(g)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

注意，生成器表达式很像列表推导式:

>>> lc = [n for n in range(3, 5)]
>>> lc
[3, 4]

Observe that a generator object is generated once, but its code is not run all at once. Only calls to next actually execute (part of) the code. Execution of the code in a generator stops once a yield statement has been reached, upon which it returns a value. The next call to next then causes execution to continue in the state in which the generator was left after the last yield. This is a fundamental difference with regular functions: those always start execution at the "top" and discard their state upon returning a value.

关于这个问题还有很多要说的。例如，可以将数据发送回生成器(参考)。但是，我建议您在了解生成器的基本概念之前不要研究这个问题。

现在你可能会问:为什么要使用生成器?有几个很好的理由:

Certain concepts can be described much more succinctly using generators. Instead of creating a function which returns a list of values, one can write a generator which generates the values on the fly. This means that no list needs to be constructed, meaning that the resulting code is more memory efficient. In this way one can even describe data streams which would simply be too large to fit in memory. Generators allow for a natural way to describe infinite streams. Consider for example the Fibonacci numbers: >>> def fib(): ... a, b = 0, 1 ... while True: ... yield a ... a, b = b, a + b ... >>> import itertools >>> list(itertools.islice(fib(), 10)) [0, 1, 1, 2, 3, 5, 8, 13, 21, 34] This code uses itertools.islice to take a finite number of elements from an infinite stream. You are advised to have a good look at the functions in the itertools module, as they are essential tools for writing advanced generators with great ease.

†关于Python <=2.6:在上面的例子中，next是一个函数，它调用给定对象的__next__方法。在Python <=2.6中使用了稍微不同的技术，即o.next()而不是next(o)。Python 2.7有next()调用.next，所以你不需要在2.7中使用下面的方法:

>>> g = (n for n in range(3, 5))
>>> g.next()
3

生成器实际上是一个函数，它在完成之前返回(数据)，但它在该点暂停，您可以在该点恢复该函数。

>>> def myGenerator():
...     yield 'These'
...     yield 'words'
...     yield 'come'
...     yield 'one'
...     yield 'at'
...     yield 'a'
...     yield 'time'

>>> myGeneratorInstance = myGenerator()
>>> next(myGeneratorInstance)
These
>>> next(myGeneratorInstance)
words

等等。生成器的(或一个)好处是，因为它们一次处理一块数据，所以您可以处理大量数据;对于列表，过多的内存需求可能成为一个问题。生成器，就像列表一样，是可迭代的，所以它们可以以相同的方式使用:

>>> for word in myGeneratorInstance:
...     print word
These
words
come
one
at 
a 
time

例如，请注意生成器提供了另一种处理无穷大的方法

>>> from time import gmtime, strftime
>>> def myGen():
...     while True:
...         yield strftime("%a, %d %b %Y %H:%M:%S +0000", gmtime())    
>>> myGeneratorInstance = myGen()
>>> next(myGeneratorInstance)
Thu, 28 Jun 2001 14:17:15 +0000
>>> next(myGeneratorInstance)
Thu, 28 Jun 2001 14:18:02 +0000   

生成器封装了一个无限循环，但这不是问题，因为每次您都只能得到每个答案。

使用列表推导式的经验表明，它们在Python中具有广泛的实用性。然而，许多用例不需要在内存中创建一个完整的列表。相反，它们每次只需要迭代一个元素。

例如，下面的求和代码将在内存中构建一个完整的方块列表，遍历这些值，当引用不再需要时，删除列表:

Sum ([x*x for x in range(10)])

通过使用生成器表达式来节省内存:

求和(x*x for x in range(10))

容器对象的构造函数也有类似的好处:

s = Set(word  for line in page  for word in line.split())
d = dict( (k, func(k)) for k in keylist)

生成器表达式对于sum()， min()和max()这样的函数特别有用，它们将可迭代输入减少为单个值:

max(len(line)  for line in file  if line.strip())

more

性能差异:

macOS Big Sur 11.1
MacBook Pro (13-inch, M1, 2020)
Chip Apple M1
Memory 8gb

案例1

import random
import psutil # pip install psutil
import os
from datetime import datetime


def memory_usage_psutil():
    # return the memory usage in MB
    process = psutil.Process(os.getpid())
    mem = process.memory_info().rss / float(2 ** 20)
    return '{:.2f} MB'.format(mem)


names = ['John', 'Milovan', 'Adam', 'Steve', 'Rick', 'Thomas']
majors = ['Math', 'Engineering', 'CompSci', 'Arts', 'Business']

print('Memory (Before): {}'.format(memory_usage_psutil()))


def people_list(num_people):
    result = []
    for i in range(num_people):
        person = {
            'id': i,
            'name': random.choice(names),
            'major': random.choice(majors)
        }
        result.append(person)
    return result


t1 = datetime.now()
people = people_list(1000000)
t2 = datetime.now()


print('Memory (After) : {}'.format(memory_usage_psutil()))
print('Took {} Seconds'.format(t2 - t1))

输出:

Memory (Before): 50.38 MB
Memory (After) : 1140.41 MB
Took 0:00:01.056423 Seconds

函数，返回一个包含100万个结果的列表。 在底部，我打印出内存使用情况和总时间。 基本内存使用大约是50.38兆字节，在我创建了100万条记录的列表之后，你可以看到它增加了近1140.41兆字节，花了1.1秒。

案例2

import random
import psutil # pip install psutil
import os
from datetime import datetime

def memory_usage_psutil():
    # return the memory usage in MB
    process = psutil.Process(os.getpid())
    mem = process.memory_info().rss / float(2 ** 20)
    return '{:.2f} MB'.format(mem)


names = ['John', 'Milovan', 'Adam', 'Steve', 'Rick', 'Thomas']
majors = ['Math', 'Engineering', 'CompSci', 'Arts', 'Business']

print('Memory (Before): {}'.format(memory_usage_psutil()))

def people_generator(num_people):
    for i in range(num_people):
        person = {
            'id': i,
            'name': random.choice(names),
            'major': random.choice(majors)
        }
        yield person


t1 = datetime.now()
people = people_generator(1000000)
t2 = datetime.now()

print('Memory (After) : {}'.format(memory_usage_psutil()))
print('Took {} Seconds'.format(t2 - t1))

输出:

Memory (Before): 50.52 MB
Memory (After) : 50.73 MB
Took 0:00:00.000008 Seconds

After I ran this that the memory is almost exactly the same and that's because the generator hasn't actually done anything yet it's not holding those million values in memory it's waiting for me to grab the next one. Basically it didn't take any time because as soon as it gets to the first yield statement it stops. I think that it is generator a little bit more readable and it also gives you big performance boosts not only with execution time but with memory. As well and you can still use all of the comprehensions and this generator expression here so you don't lose anything in that area. So those are a few reasons why you would use generators and also some of the advantages that come along with that.

它有助于明确区分函数foo和生成器foo(n):

def foo(n):
    yield n
    yield n+1

Foo是一个函数。 Foo(6)是一个生成器对象。

使用生成器对象的典型方式是在循环中:

for n in foo(6):
    print(n)

循环打印

# 6
# 7

可以将生成器视为可恢复函数。

Yield的行为类似于return，产生的值被生成器“返回”。然而，与return不同的是，下一次生成器被请求一个值时，生成器的函数foo将从它停止的地方恢复——在最后一个yield语句之后——并继续运行，直到遇到另一个yield语句。

在幕后，当您调用bar=foo(6)时，生成器对象bar为您定义了一个下一个属性。

你可以自己调用它来获取foo产生的值:

next(bar)    # Works in Python 2.6 or Python 3.x
bar.next()   # Works in Python 2.5+, but is deprecated. Use next() if possible.

当foo结束时(并且没有更多的输出值)，调用next(bar)将抛出StopInteration错误。

我相信迭代器和生成器的第一次出现是在Icon编程语言中，大约20年前。

你可能会喜欢Icon的概述，它可以让你在不关注语法的情况下理解它们(因为Icon是一种你可能不知道的语言，Griswold是在向来自其他语言的人解释他的语言的好处)。

在阅读了几段之后，生成器和迭代器的效用可能会变得更加明显。