我需要在c++中加载和使用CSV文件数据。在这一点上,它实际上只是一个以逗号分隔的解析器(即不用担心转义新行和逗号)。主要需要的是逐行解析器,它将在每次调用方法时为下一行返回一个向量。
我发现这篇文章看起来很有前途: http://www.boost.org/doc/libs/1_35_0/libs/spirit/example/fundamental/list_parser.cpp
我从未使用过Boost's Spirit,但我愿意尝试一下。但前提是我忽略了一个更直接的解决方案。
我需要在c++中加载和使用CSV文件数据。在这一点上,它实际上只是一个以逗号分隔的解析器(即不用担心转义新行和逗号)。主要需要的是逐行解析器,它将在每次调用方法时为下一行返回一个向量。
我发现这篇文章看起来很有前途: http://www.boost.org/doc/libs/1_35_0/libs/spirit/example/fundamental/list_parser.cpp
我从未使用过Boost's Spirit,但我愿意尝试一下。但前提是我忽略了一个更直接的解决方案。
当前回答
这是一个旧线程,但它仍然在搜索结果的顶部,所以我添加我的解决方案使用std::stringstream和一个简单的字符串替换方法由Yves Baumes我在这里找到。
下面的例子将逐行读取文件,忽略以//开头的注释行,并将其他行解析为字符串、int和double的组合。Stringstream进行解析,但希望字段由空格分隔,因此我使用stringreplace首先将逗号转换为空格。它可以处理制表符,但不处理带引号的字符串。
坏的或丢失的输入被简单地忽略,这可能是好事,也可能不是好事,这取决于您的情况。
#include <string>
#include <sstream>
#include <fstream>
void StringReplace(std::string& str, const std::string& oldStr, const std::string& newStr)
// code by Yves Baumes
// http://stackoverflow.com/questions/1494399/how-do-i-search-find-and-replace-in-a-standard-string
{
size_t pos = 0;
while((pos = str.find(oldStr, pos)) != std::string::npos)
{
str.replace(pos, oldStr.length(), newStr);
pos += newStr.length();
}
}
void LoadCSV(std::string &filename) {
std::ifstream stream(filename);
std::string in_line;
std::string Field;
std::string Chan;
int ChanType;
double Scale;
int Import;
while (std::getline(stream, in_line)) {
StringReplace(in_line, ",", " ");
std::stringstream line(in_line);
line >> Field >> Chan >> ChanType >> Scale >> Import;
if (Field.substr(0,2)!="//") {
// do your stuff
// this is CBuilder code for demonstration, sorry
ShowMessage((String)Field.c_str() + "\n" + Chan.c_str() + "\n" + IntToStr(ChanType) + "\n" +FloatToStr(Scale) + "\n" +IntToStr(Import));
}
}
}
其他回答
你可以在escaped_list_separator中使用Boost Tokenizer。
Escaped_list_separator解析csv的超集。Boost::记号赋予器
这只使用Boost标记器头文件,不需要链接到Boost库。
下面是一个例子,(详情请参阅c++中使用Boost Tokenizer解析CSV文件或Boost:: Tokenizer):
#include <iostream> // cout, endl
#include <fstream> // fstream
#include <vector>
#include <string>
#include <algorithm> // copy
#include <iterator> // ostream_operator
#include <boost/tokenizer.hpp>
int main()
{
using namespace std;
using namespace boost;
string data("data.csv");
ifstream in(data.c_str());
if (!in.is_open()) return 1;
typedef tokenizer< escaped_list_separator<char> > Tokenizer;
vector< string > vec;
string line;
while (getline(in,line))
{
Tokenizer tok(line);
vec.assign(tok.begin(),tok.end());
// vector now contains strings from one row, output to cout here
copy(vec.begin(), vec.end(), ostream_iterator<string>(cout, "|"));
cout << "\n----------------------" << endl;
}
}
这是一个旧线程,但它仍然在搜索结果的顶部,所以我添加我的解决方案使用std::stringstream和一个简单的字符串替换方法由Yves Baumes我在这里找到。
下面的例子将逐行读取文件,忽略以//开头的注释行,并将其他行解析为字符串、int和double的组合。Stringstream进行解析,但希望字段由空格分隔,因此我使用stringreplace首先将逗号转换为空格。它可以处理制表符,但不处理带引号的字符串。
坏的或丢失的输入被简单地忽略,这可能是好事,也可能不是好事,这取决于您的情况。
#include <string>
#include <sstream>
#include <fstream>
void StringReplace(std::string& str, const std::string& oldStr, const std::string& newStr)
// code by Yves Baumes
// http://stackoverflow.com/questions/1494399/how-do-i-search-find-and-replace-in-a-standard-string
{
size_t pos = 0;
while((pos = str.find(oldStr, pos)) != std::string::npos)
{
str.replace(pos, oldStr.length(), newStr);
pos += newStr.length();
}
}
void LoadCSV(std::string &filename) {
std::ifstream stream(filename);
std::string in_line;
std::string Field;
std::string Chan;
int ChanType;
double Scale;
int Import;
while (std::getline(stream, in_line)) {
StringReplace(in_line, ",", " ");
std::stringstream line(in_line);
line >> Field >> Chan >> ChanType >> Scale >> Import;
if (Field.substr(0,2)!="//") {
// do your stuff
// this is CBuilder code for demonstration, sorry
ShowMessage((String)Field.c_str() + "\n" + Chan.c_str() + "\n" + IntToStr(ChanType) + "\n" +FloatToStr(Scale) + "\n" +IntToStr(Import));
}
}
}
@sastanin的解决方案的一个小版本,以便它可以处理引号中的换行。
std::vector<std::vector<std::string>> readCSV(std::istream &in) {
std::vector<std::vector<std::string>> table;
while (!in.eof()) {
CSVState state = CSVState::UnquotedField;
std::vector<std::string> fields {""};
size_t i = 0; // index of the current field
for (char c : row) {
switch (state) {
case CSVState::UnquotedField:
switch (c) {
case ',': // end of field
fields.push_back(""); i++;
break;
case '"': state = CSVState::QuotedField;
break;
default: fields[i].push_back(c);
break; }
break;
case CSVState::QuotedField:
switch (c) {
case '"': state = CSVState::QuotedQuote;
break;
default: fields[i].push_back(c);
break; }
break;
case CSVState::QuotedQuote:
switch (c) {
case ',': // , after closing quote
fields.push_back(""); i++;
state = CSVState::UnquotedField;
break;
case '"': // "" -> "
fields[i].push_back('"');
state = CSVState::QuotedField;
break;
case '\n': // newline
table.push_back(fields);
state = CSVState::UnquotedField;
fields = vector<string>{""};
i = 0;
default: // end of quote
state = CSVState::UnquotedField;
break; }
break;
}
}
}
return table;
}
下面是Unicode CSV解析器的另一个实现(使用wchar_t)。我写了一部分,乔纳森·莱弗勒写了剩下的部分。
注意:此解析器旨在尽可能地复制Excel的行为,特别是在导入损坏或格式错误的CSV文件时。
这是最初的问题-用多行字段和转义双引号解析CSV文件
这是作为SSCCE(简短,自包含,正确示例)的代码。
#include <stdbool.h>
#include <wchar.h>
#include <wctype.h>
extern const wchar_t *nextCsvField(const wchar_t *p, wchar_t sep, bool *newline);
// Returns a pointer to the start of the next field,
// or zero if this is the last field in the CSV
// p is the start position of the field
// sep is the separator used, i.e. comma or semicolon
// newline says whether the field ends with a newline or with a comma
const wchar_t *nextCsvField(const wchar_t *p, wchar_t sep, bool *newline)
{
// Parse quoted sequences
if ('"' == p[0]) {
p++;
while (1) {
// Find next double-quote
p = wcschr(p, L'"');
// If we don't find it or it's the last symbol
// then this is the last field
if (!p || !p[1])
return 0;
// Check for "", it is an escaped double-quote
if (p[1] != '"')
break;
// Skip the escaped double-quote
p += 2;
}
}
// Find next newline or comma.
wchar_t newline_or_sep[4] = L"\n\r ";
newline_or_sep[2] = sep;
p = wcspbrk(p, newline_or_sep);
// If no newline or separator, this is the last field.
if (!p)
return 0;
// Check if we had newline.
*newline = (p[0] == '\r' || p[0] == '\n');
// Handle "\r\n", otherwise just increment
if (p[0] == '\r' && p[1] == '\n')
p += 2;
else
p++;
return p;
}
static wchar_t *csvFieldData(const wchar_t *fld_s, const wchar_t *fld_e, wchar_t *buffer, size_t buflen)
{
wchar_t *dst = buffer;
wchar_t *end = buffer + buflen - 1;
const wchar_t *src = fld_s;
if (*src == L'"')
{
const wchar_t *p = src + 1;
while (p < fld_e && dst < end)
{
if (p[0] == L'"' && p+1 < fld_s && p[1] == L'"')
{
*dst++ = p[0];
p += 2;
}
else if (p[0] == L'"')
{
p++;
break;
}
else
*dst++ = *p++;
}
src = p;
}
while (src < fld_e && dst < end)
*dst++ = *src++;
if (dst >= end)
return 0;
*dst = L'\0';
return(buffer);
}
static void dissect(const wchar_t *line)
{
const wchar_t *start = line;
const wchar_t *next;
bool eol;
wprintf(L"Input %3zd: [%.*ls]\n", wcslen(line), wcslen(line)-1, line);
while ((next = nextCsvField(start, L',', &eol)) != 0)
{
wchar_t buffer[1024];
wprintf(L"Raw Field: [%.*ls] (eol = %d)\n", (next - start - eol), start, eol);
if (csvFieldData(start, next-1, buffer, sizeof(buffer)/sizeof(buffer[0])) != 0)
wprintf(L"Field %3zd: [%ls]\n", wcslen(buffer), buffer);
start = next;
}
}
static const wchar_t multiline[] =
L"First field of first row,\"This field is multiline\n"
"\n"
"but that's OK because it's enclosed in double quotes, and this\n"
"is an escaped \"\" double quote\" but this one \"\" is not\n"
" \"This is second field of second row, but it is not multiline\n"
" because it doesn't start \n"
" with an immediate double quote\"\n"
;
int main(void)
{
wchar_t line[1024];
while (fgetws(line, sizeof(line)/sizeof(line[0]), stdin))
dissect(line);
dissect(multiline);
return 0;
}
如果可以的话,这是我简单快速的贡献。 没有提高。
接受分隔符和分隔符中的分隔符,只要成对或远离分隔符即可。
#include <iostream>
#include <vector>
#include <fstream>
std::vector<std::string> SplitCSV(const std::string &data, char separator, char delimiter)
{
std::vector<std::string> Values;
std::string Val = "";
bool VDel = false; // Is within delimiter?
size_t CDel = 0; // Delimiters counter within delimiters.
const char *C = data.c_str();
size_t P = 0;
do
{
if ((Val.length() == 0) && (C[P] == delimiter))
{
VDel = !VDel;
CDel = 0;
P++;
continue;
}
if (VDel)
{
if (C[P] == delimiter)
{
if (((CDel % 2) == 0) && ( (C[P+1] == separator) || (C[P+1] == 0) || (C[P+1] == '\n') || (C[P+1] == '\r') ))
{
VDel = false;
CDel = 0;
P++;
continue;
}
else
CDel++;
}
}
else
{
if (C[P] == separator)
{
Values.push_back(Val);
Val = "";
P++;
continue;
}
if ((C[P] == 0) || (C[P] == '\n') || (C[P] == '\r'))
break;
}
Val += C[P];
P++;
} while(P < data.length());
Values.push_back(Val);
return Values;
}
bool ReadCsv(const std::string &fname, std::vector<std::vector<std::string>> &data,
char separator = ',', char delimiter = '\"')
{
bool Ret = false;
std::ifstream FCsv(fname);
if (FCsv)
{
FCsv.seekg(0, FCsv.end);
size_t Siz = FCsv.tellg();
if (Siz > 0)
{
FCsv.seekg(0);
data.clear();
std::string Line;
while (getline(FCsv, Line, '\n'))
data.push_back(SplitCSV(Line, separator, delimiter));
Ret = true;
}
FCsv.close();
}
return Ret;
}
int main(int argc, char *argv[])
{
std::vector<std::vector<std::string>> Data;
ReadCsv("fsample.csv", Data);
return 0;
}