已经有很多答案了，但不幸的是，它们中的大多数只是一个几乎不可优化的问题上的微型经济……

在我参与的几个项目中，行数是软件的核心功能，以最快的速度处理大量文件是至关重要的。

行数的主要瓶颈是I/O访问，因为您需要读取每一行以检测行返回字符，因此没有其他方法。第二个潜在的瓶颈是内存管理:一次加载的内存越多，处理的速度就越快，但与第一个瓶颈相比，这个瓶颈可以忽略不计。

因此，除了禁用gc收集和其他微管理技巧等微小优化外，还有3种主要方法可以减少行计数函数的处理时间:

Hardware solution: the major and most obvious way is non-programmatic: buy a very fast SSD/flash hard drive. By far, this is how you can get the biggest speed boosts. Data preparation solution: if you generate or can modify how the files you process are generated, or if it's acceptable that you can pre-process them, first convert the line return to unix style (\n) as this will save 1 character compared to Windows or MacOS styles (not a big save but it's an easy gain), and secondly and most importantly, you can potentially write lines of fixed length. If you need variable length, you can always pad smaller lines. This way, you can calculate instantly the number of lines from the total filesize, which is much faster to access. Often, the best solution to a problem is to pre-process it so that it better fits your end purpose. Parallelization + hardware solution: if you can buy multiple hard disks (and if possible SSD flash disks), then you can even go beyond the speed of one disk by leveraging parallelization, by storing your files in a balanced way (easiest is to balance by total size) among disks, and then read in parallel from all those disks. Then, you can expect to get a multiplier boost in proportion with the number of disks you have. If buying multiple disks is not an option for you, then parallelization likely won't help (except if your disk has multiple reading headers like some professional-grade disks, but even then the disk's internal cache memory and PCB circuitry will likely be a bottleneck and prevent you from fully using all heads in parallel, plus you have to devise a specific code for this hard drive you'll use because you need to know the exact cluster mapping so that you store your files on clusters under different heads, and so that you can read them with different heads after). Indeed, it's commonly known that sequential reading is almost always faster than random reading, and parallelization on a single disk will have a performance more similar to random reading than sequential reading (you can test your hard drive speed in both aspects using CrystalDiskMark for example).

如果这些都不是选择，那么你只能依靠微观管理技巧来提高行数函数的速度，但不要指望有什么真正重要的东西。相反，您可以预期，与您将看到的速度改进回报相比，您花费在调整上的时间将是不均衡的。

另一种可能性:

import subprocess

def num_lines_in_file(fpath):
    return int(subprocess.check_output('wc -l %s' % fpath, shell=True).strip().split()[0])

如果文件能放进内存，那么

with open(fname) as f:
    count = len(f.read().split(b'\n')) - 1

大文件的另一种选择是使用xreadlines():

count = 0
for line in open(thefilepath).xreadlines(  ): count += 1

对于Python 3，请参阅:在Python 3中什么替代xreadlines() ?

创建一个可执行脚本文件count.py:

#!/usr/bin/python

import sys
count = 0
for line in sys.stdin:
    count+=1

然后将文件的内容导入python脚本:cat huge.txt | ./count.py。管道也适用于Powershell，因此您将最终计算行数。

对我来说，在Linux上它比简单的解决方案快30%:

count=1
with open('huge.txt') as f:
    count+=1

对我来说，这个变体是最快的:

#!/usr/bin/env python

def main():
    f = open('filename')                  
    lines = 0
    buf_size = 1024 * 1024
    read_f = f.read # loop optimization

    buf = read_f(buf_size)
    while buf:
        lines += buf.count('\n')
        buf = read_f(buf_size)

    print lines

if __name__ == '__main__':
    main()

原因:缓冲比逐行和逐字符串读取快。计数也非常快