我需要使用Java逐行读取大约5-6 GB的大型文本文件。

我怎样才能快速做到呢?


当前回答

FileReader不会让你指定编码,如果你需要指定它,使用inputstreamreader代替:

try {
    BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(filePath), "Cp1252"));         

    String line;
    while ((line = br.readLine()) != null) {
        // process the line.
    }
    br.close();

} catch (IOException e) {
    e.printStackTrace();
}

如果从Windows导入该文件,它可能具有ANSI编码(Cp1252),因此必须指定编码。

其他回答

您需要在BufferedReader类中使用readLine()方法。 从该类创建一个新对象,并对他操作此方法,并将其保存为字符串。

BufferReader Javadoc

看看这个博客:

Java逐行读取文件- Java教程

可以指定缓冲区大小或 可以使用默认大小。的 违约对大多数人来说已经足够大了 目的。

// Open the file
FileInputStream fstream = new FileInputStream("textfile.txt");
BufferedReader br = new BufferedReader(new InputStreamReader(fstream));

String strLine;

//Read File Line By Line
while ((strLine = br.readLine()) != null)   {
  // Print the content on the console
  System.out.println (strLine);
}

//Close the input stream
fstream.close();

我的阅读习惯通常很简单:

void readResource(InputStream source) throws IOException {
    BufferedReader stream = null;
    try {
        stream = new BufferedReader(new InputStreamReader(source));
        while (true) {
            String line = stream.readLine();
            if(line == null) {
                break;
            }
            //process line
            System.out.println(line)
        }
    } finally {
        closeQuiet(stream);
    }
}

static void closeQuiet(Closeable closeable) {
    if (closeable != null) {
        try {
            closeable.close();
        } catch (IOException ignore) {
        }
    }
}

你也可以使用Apache Commons IO:

File file = new File("/home/user/file.txt");
try {
    List<String> lines = FileUtils.readLines(file);
} catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
}

在Java 7中:

String folderPath = "C:/folderOfMyFile";
Path path = Paths.get(folderPath, "myFileName.csv"); //or any text file eg.: txt, bat, etc
Charset charset = Charset.forName("UTF-8");

try (BufferedReader reader = Files.newBufferedReader(path , charset)) {
  while ((line = reader.readLine()) != null ) {
    //separate all csv fields into string array
    String[] lineVariables = line.split(","); 
  }
} catch (IOException e) {
    System.err.println(e);
}