我想找到最快的方法来检查一个文件是否存在于标准c++ 11, 14, 17,或C。我有成千上万的文件,在对它们做一些事情之前,我需要检查它们是否都存在。在下面的函数中,我可以写什么来代替/* SOMETHING */ ?

inline bool exist(const std::string& name)
{
    /* SOMETHING */
}

当前回答

备注:在c++ 14中,一旦文件系统TS完成并被采用,解决方案将使用:

std::experimental::filesystem::exists("helloworld.txt");

从c++ 17开始,只有:

std::filesystem::exists("helloworld.txt");

其他回答

all_of (begin(R), end(R), [](auto&p){ exists(p); })

其中R是你的路径序列,exists()来自未来std或当前boost。如果你自己卷,简单点,

bool exists (string const& p) { return ifstream{p}; }

分支解决方案并不是绝对可怕的,它不会吞噬文件描述符,

bool exists (const char* p) {
    #if defined(_WIN32) || defined(_WIN64)
    return p && 0 != PathFileExists (p);
    #else
    struct stat sb;
    return p && 0 == stat (p, &sb);
    #endif
}

It depends on where the files reside. For instance, if they are all supposed to be in the same directory, you can read all the directory entries into a hash table and then check all the names against the hash table. This might be faster on some systems than checking each file individually. The fastest way to check each file individually depends on your system ... if you're writing ANSI C, the fastest way is fopen because it's the only way (a file might exist but not be openable, but you probably really want openable if you need to "do something on it"). C++, POSIX, Windows all offer additional options.

While I'm at it, let me point out some problems with your question. You say that you want the fastest way, and that you have thousands of files, but then you ask for the code for a function to test a single file (and that function is only valid in C++, not C). This contradicts your requirements by making an assumption about the solution ... a case of the XY problem. You also say "in standard c++11(or)c++(or)c" ... which are all different, and this also is inconsistent with your requirement for speed ... the fastest solution would involve tailoring the code to the target system. The inconsistency in the question is highlighted by the fact that you accepted an answer that gives solutions that are system-dependent and are not standard C or C++.

所有其他答案都侧重于单独检查每个文件,但如果所有文件都在一个目录(文件夹)中,那么读取目录并检查是否存在所需的每个文件名可能会更有效。

即使文件分布在几个目录中,这也可能更有效,这取决于目录与文件的确切比例。一旦您开始接近每个目标文件都在自己的目录中,或者在相同的目录中有许多其他文件,而您不想检查,那么我预计它最终会变得比单独检查每个文件更低效。

A good heuristic: working on a bunch of data you already have is much faster than asking the operating system for any amount of data. System call overhead is huge relative to individual machine instructions. So it is almost always going to be faster to ask the OS "give me the entire list of files in this directory" and then to dig through that list, and slower to ask the OS "give me information on this file", "okay now give me information on this other file", "now give me information on ...", and so on.

每个优秀的C库都以一种有效的方式实现了“遍历目录下的所有文件”api,就像缓冲I/O一样——在内部,它一次从操作系统中读取一个大的目录条目列表,即使这些api看起来像要求操作系统单独读取每个条目。


如果我有这个要求,我会

尽一切可能鼓励设计和使用,这样所有的文件都在一个文件夹里,没有其他文件在那个文件夹里, 将我需要呈现的文件名列表放入内存中的数据结构中,该数据结构具有O(1)或至少O(log(n))次查找和删除次数(就像哈希映射或二叉树), 列出该目录中的文件,并从内存中的“列表”(哈希映射或二叉树)中“检查”(删除)每个文件。

Except depending on the exact use case, maybe instead of deleting entries from a hash map or tree, I would keep track of a "do I have this file?" boolean for each entry, and figure out a data structure that would make it O(1) to ask "do I have every file?". Maybe a binary tree but the struct for each non-leaf node also has a boolean that is a logical-and of the booleans of its leaf nodes. That scales well - after setting a boolean in a leaf node, you just walk up the tree and set each node's "have this?" boolean with the && of its child node's boolean (and you don't need to recurse down those other child nodes, since if you're doing this process consistently every time you go to set one of the leaves to true, they will be set to true if and only if all of their children are.)


不幸的是,在c++ 17之前没有标准的方法来做这件事。

c++ 17得到std::filesystem::directory_iterator。

当然,有一个相应的boost::filesystem::directory_iterator,我认为它可以在旧版本的c++中工作。

最接近标准C方法的是opendir和readdir from direct .h。这是一个标准的C接口,它只是在POSIX中标准化,而不是在C标准本身中。它可以在Mac OS、Linux、所有bsd、其他UNIX/类UNIX系统和任何其他POSIX/SUS系统上开箱即用。对于Windows,有一个direct .h实现,你只需要下载并放入你的include路径。

然而,由于您正在寻找最快的方法,您可能想要超越便携式/标准的东西。

在Linux上,您可以通过使用原始系统调用getdents64手动指定缓冲区大小来优化性能。

在Windows上,经过一些挖掘,看起来为了获得最大的性能,你想要使用FindFirstFileEx与FindExInfoBasic和FIND_FIRST_EX_LARGE_FETCH,当你可以的时候,这是许多开源库,如上面的direct .h Windows似乎没有做到的。但是对于需要处理比前两个Windows版本更老的东西的代码,您不妨只使用直接的FindFirstFile,而不使用额外的标记。

上面的任何一种方法都不包括计划9,您将需要dirread或dirreadall(如果您可以安全地假设您有足够的内存用于整个目录内容,则选择后者)。如果您想要更好地控制缓冲区大小以提高性能,可以使用普通读取或读取并解码目录条目数据——它们是有文档记录的与机器无关的格式,而且我认为提供了辅助函数。

我不知道其他的操作系统。


稍后我可能会用一些测试编辑这个答案。也欢迎其他人编辑测试结果。

检测文件是否存在于windows中。

bool DoesExistFile(std::wstring filePath)
{
    bool result = true;

    HANDLE fileHandle = CreateFile(
        filePath.c_str(),
        GENERIC_READ,
        0,
        NULL,
        OPEN_EXISTING,
        0,
        NULL);

    if ((fileHandle != NULL) && (fileHandle != INVALID_HANDLE_VALUE))
        CloseHandle(fileHandle);
    else
    {
        DWORD error = GetLastError();

        if ((error == ERROR_FILE_NOT_FOUND) || (error == ERROR_PATH_NOT_FOUND))
            result = false;
    }

    return result;
}

使用MFC可以实现以下功能

CFileStatus FileStatus;
BOOL bFileExists = CFile::GetStatus(FileName,FileStatus);

哪里的文件名是一个字符串表示文件,您正在检查的存在