在Stack Overflow的问题中有一个关于这个问题的好话题，“收益返回”比“老派”返回慢吗?

它说:

ReadAllLines loads all of the lines into memory and returns a string[]. All well and good if the file is small. If the file is larger than will fit in memory, you'll run out of memory. ReadLines, on the other hand, uses yield return to return one line at a time. With it, you can read any size file. It doesn't load the whole file into memory. Say you wanted to find the first line that contains the word "foo", and then exit. Using ReadAllLines, you'd have to read the entire file into memory, even if "foo" occurs on the first line. With ReadLines, you only read one line. Which one would be faster?

在Stack Overflow的问题中有一个关于这个问题的好话题，“收益返回”比“老派”返回慢吗?

它说:

ReadAllLines loads all of the lines into memory and returns a string[]. All well and good if the file is small. If the file is larger than will fit in memory, you'll run out of memory. ReadLines, on the other hand, uses yield return to return one line at a time. With it, you can read any size file. It doesn't load the whole file into memory. Say you wanted to find the first line that contains the word "foo", and then exit. Using ReadAllLines, you'd have to read the entire file into memory, even if "foo" occurs on the first line. With ReadLines, you only read one line. Which one would be faster?

使用以下代码:

foreach (string line in File.ReadAllLines(fileName))

这是阅读表现的巨大差异。

这是以内存消耗为代价的，但完全值得!

当您需要有效地读取和处理一个巨大的文本文件时，ReadLines()和ReadAllLines()可能会抛出内存耗尽异常，这是我的情况。另一方面，单独阅读每一行会花费很长时间。解决方案是按块读取文件，如下所示。

类:

    //can return empty lines sometimes
    class LinePortionTextReader
    {
        private const int BUFFER_SIZE = 100000000; //100M characters
        StreamReader sr = null;
        string remainder = "";

        public LinePortionTextReader(string filePath)
        {
            if (File.Exists(filePath))
            {
                sr = new StreamReader(filePath);
                remainder = "";
            }
        }

        ~LinePortionTextReader()
        {
            if(null != sr) { sr.Close(); }
        }

        public string[] ReadBlock()
        {
            if(null==sr) { return new string[] { }; }
            char[] buffer = new char[BUFFER_SIZE];
            int charactersRead = sr.Read(buffer, 0, BUFFER_SIZE);
            if (charactersRead < 1) { return new string[] { }; }
            bool lastPart = (charactersRead < BUFFER_SIZE);
            if (lastPart)
            {
                char[] buffer2 = buffer.Take<char>(charactersRead).ToArray();
                buffer = buffer2;
            }
            string s = new string(buffer);
            string[] sresult = s.Split(new string[] { "\r\n" }, StringSplitOptions.None);
            sresult[0] = remainder + sresult[0];
            if (!lastPart)
            {
                remainder = sresult[sresult.Length - 1];
                sresult[sresult.Length - 1] = "";
            }
            return sresult;
        }

        public bool EOS
        {
            get
            {
                return (null == sr) ? true: sr.EndOfStream;
            }
        }
    }

使用示例:

    class Program
    {
        static void Main(string[] args)
        {
            if (args.Length < 3)
            {
                Console.WriteLine("multifind.exe <where to search> <what to look for, one value per line> <where to put the result>");
                return;
            }

            if (!File.Exists(args[0]))
            {
                Console.WriteLine("source file not found");
                return;
            }
            if (!File.Exists(args[1]))
            {
                Console.WriteLine("reference file not found");
                return;
            }

            TextWriter tw = new StreamWriter(args[2], false);

            string[] refLines = File.ReadAllLines(args[1]);

            LinePortionTextReader lptr = new LinePortionTextReader(args[0]);
            int blockCounter = 0;
            while (!lptr.EOS)
            {
                string[] srcLines = lptr.ReadBlock();
                for (int i = 0; i < srcLines.Length; i += 1)
                {
                    string theLine = srcLines[i];
                    if (!string.IsNullOrEmpty(theLine)) //can return empty lines sometimes
                    {
                        for (int j = 0; j < refLines.Length; j += 1)
                        {
                            if (theLine.Contains(refLines[j]))
                            {
                                tw.WriteLine(theLine);
                                break;
                            }
                        }
                    }
                }

                blockCounter += 1;
                Console.WriteLine(String.Format("100 Mb blocks processed: {0}", blockCounter));
            }
            tw.Close();
        }
    }

我相信拆分字符串和数组处理可以显著改善， 然而，这里的目标是最小化磁盘读取的数量。

虽然file . readalllines()是读取文件的最简单方法之一，但它也是最慢的方法之一。

如果你只是想读取文件中的行而不做太多事情，根据这些基准测试，读取文件的最快方法是古老的方法:

using (StreamReader sr = File.OpenText(fileName))
{
        string s = String.Empty;
        while ((s = sr.ReadLine()) != null)
        {
               //do minimal amount of work here
        }
}

然而，如果你必须对每一行做很多事情，那么本文得出的结论是，最好的方法是以下(如果你知道你要读取多少行，那么预先分配一个字符串[]会更快):

AllLines = new string[MAX]; //only allocate memory here

using (StreamReader sr = File.OpenText(fileName))
{
        int x = 0;
        while (!sr.EndOfStream)
        {
               AllLines[x] = sr.ReadLine();
               x += 1;
        }
} //Finished. Close the file

//Now parallel process each line in the file
Parallel.For(0, AllLines.Length, x =>
{
    DoYourStuff(AllLines[x]); //do your work here
});

如果你正在使用。net 4，只需使用File即可。ReadLines为你做了所有这些。我怀疑它和你的差不多，除了它也可以使用FileOptions。SequentialScan和一个更大的缓冲区(128看起来很小)。