为了演示如何组合itertools recipe,我将使用consume recipe尽可能直接地将成对的recipe扩展回窗口recipe:
def consume(iterator, n):
"Advance the iterator n-steps ahead. If n is none, consume entirely."
# Use functions that consume iterators at C speed.
if n is None:
# feed the entire iterator into a zero-length deque
collections.deque(iterator, maxlen=0)
else:
# advance to the empty slice starting at position n
next(islice(iterator, n, n), None)
def window(iterable, n=2):
"s -> (s0, ...,s(n-1)), (s1, ...,sn), (s2, ..., s(n+1)), ..."
iters = tee(iterable, n)
# Could use enumerate(islice(iters, 1, None), 1) to avoid consume(it, 0), but that's
# slower for larger window sizes, while saving only small fixed "noop" cost
for i, it in enumerate(iters):
consume(it, i)
return zip(*iters)
窗口的配方与成对的相同,它只是将第二个tee-ed迭代器上的单个元素“consume”替换为逐步增加n - 1个迭代器上的consume。使用consume而不是在islice中包装每个迭代器稍微快一些(对于足够大的可迭代对象),因为您只在消费阶段支付islice包装开销,而不是在提取每个窗口值的过程中(因此它以n为界,而不是iterable中项目的数量)。
在性能方面,与其他一些解决方案相比,这是相当不错的(并且比我测试的任何其他解决方案都要好)。在Python 3.5.0, Linux x86-64上测试,使用ipython %timeit magic。
Kindall是deque解决方案,通过使用islice而不是自制生成器表达式来调整性能/正确性,并测试产生的长度,因此当可迭代对象比窗口短时不会产生结果,以及通过位置传递deque的maxlen而不是通过关键字(对于较小的输入产生惊人的差异):
>>> %timeit -r5 deque(windowkindall(range(10), 3), 0)
100000 loops, best of 5: 1.87 μs per loop
>>> %timeit -r5 deque(windowkindall(range(1000), 3), 0)
10000 loops, best of 5: 72.6 μs per loop
>>> %timeit -r5 deque(windowkindall(range(1000), 30), 0)
1000 loops, best of 5: 71.6 μs per loop
与之前改编的kindall解决方案相同,但每个yield win都更改为yield tuple(win),因此存储来自生成器的结果,而不需要所有存储的结果真正成为最新结果的视图(在这种情况下,所有其他合理的解决方案都是安全的),并将tuple=tuple添加到函数定义中,以将tuple的使用从LEGB中的B移动到L:
>>> %timeit -r5 deque(windowkindalltupled(range(10), 3), 0)
100000 loops, best of 5: 3.05 μs per loop
>>> %timeit -r5 deque(windowkindalltupled(range(1000), 3), 0)
10000 loops, best of 5: 207 μs per loop
>>> %timeit -r5 deque(windowkindalltupled(range(1000), 30), 0)
1000 loops, best of 5: 348 μs per loop
如上所示的基于消费的解决方案:
>>> %timeit -r5 deque(windowconsume(range(10), 3), 0)
100000 loops, best of 5: 3.92 μs per loop
>>> %timeit -r5 deque(windowconsume(range(1000), 3), 0)
10000 loops, best of 5: 42.8 μs per loop
>>> %timeit -r5 deque(windowconsume(range(1000), 30), 0)
1000 loops, best of 5: 232 μs per loop
与consume相同,但是内联了consume的else case以避免函数调用,n是None测试以减少运行时间,特别是对于设置开销是工作中有意义的一部分的小输入:
>>> %timeit -r5 deque(windowinlineconsume(range(10), 3), 0)
100000 loops, best of 5: 3.57 μs per loop
>>> %timeit -r5 deque(windowinlineconsume(range(1000), 3), 0)
10000 loops, best of 5: 40.9 μs per loop
>>> %timeit -r5 deque(windowinlineconsume(range(1000), 30), 0)
1000 loops, best of 5: 211 μs per loop
(旁注:成对的一种变体,重复使用tee和默认参数2来创建嵌套的tee对象,因此任何给定的迭代器只前进一次,而不是独立地消耗越来越多的次数,类似于MrDrFenner的答案类似于非内联消耗,并且在所有测试中比内联消耗更慢,所以为了简洁起见,我省略了这些结果)。
As you can see, if you don't care about the possibility of the caller needing to store results, my optimized version of kindall's solution wins most of the time, except in the "large iterable, small window size case" (where inlined consume wins); it degrades quickly as the iterable size increases, while not degrading at all as the window size increases (every other solution degrades more slowly for iterable size increases, but also degrades for window size increases). It can even be adapted for the "need tuples" case by wrapping in map(tuple, ...), which runs ever so slightly slower than putting the tupling in the function, but it's trivial (takes 1-5% longer) and lets you keep the flexibility of running faster when you can tolerate repeatedly returning the same value.
如果需要安全的存储返回值,除了最小的输入大小外,内联消耗在所有输入大小上都占优势(非内联消耗略慢,但扩展相似)。基于deque & tuple的解决方案只对最小的输入获胜,因为更小的设置成本,收益也很小;当可迭代对象变长时,它会严重退化。
为了记录,我使用的kindall的解决方案的改编版本产生元组是:
def windowkindalltupled(iterable, n=2, tuple=tuple):
it = iter(iterable)
win = deque(islice(it, n), n)
if len(win) < n:
return
append = win.append
yield tuple(win)
for e in it:
append(e)
yield tuple(win)
在函数定义行中删除缓存tuple,并在每个yield中使用tuple,以获得更快但不太安全的版本。