Java有一个方便的分割方法:
String str = "The quick brown fox";
String[] results = str.split(" ");
在c++中有简单的方法来做到这一点吗?
Java有一个方便的分割方法:
String str = "The quick brown fox";
String[] results = str.split(" ");
在c++中有简单的方法来做到这一点吗?
当前回答
如果你正在使用c++ ranges——完整的range -v3库,而不是c++ 20所接受的有限功能——你可以这样做:
auto results = str | ranges::views::tokenize(" ",1);
... 这是惰性求值。你也可以在这个范围内设置一个向量:
auto results = str | ranges::views::tokenize(" ",1) | ranges::to<std::vector>();
如果str有n个字符组成m个单词,这将占用O(m)个空间和O(n)个时间。
参见标准库自己的标记化示例。
其他回答
MFC/ATL有一个非常好的标记器。从MSDN:
CAtlString str( "%First Second#Third" );
CAtlString resToken;
int curPos= 0;
resToken= str.Tokenize("% #",curPos);
while (resToken != "")
{
printf("Resulting token: %s\n", resToken);
resToken= str.Tokenize("% #",curPos);
};
Output
Resulting Token: First
Resulting Token: Second
Resulting Token: Third
If the maximum length of the input string to be tokenized is known, one can exploit this and implement a very fast version. I am sketching the basic idea below, which was inspired by both strtok() and the "suffix array"-data structure described Jon Bentley's "Programming Perls" 2nd edition, chapter 15. The C++ class in this case only gives some organization and convenience of use. The implementation shown can be easily extended for removing leading and trailing whitespace characters in the tokens.
基本上,可以将分隔符替换为以字符串结束的'\0'字符,并设置指向修改后字符串中的标记的指针。在极端情况下,当字符串仅由分隔符组成时,将得到字符串长度加1个空标记。复制要修改的字符串是可行的。
头文件:
class TextLineSplitter
{
public:
TextLineSplitter( const size_t max_line_len );
~TextLineSplitter();
void SplitLine( const char *line,
const char sep_char = ',',
);
inline size_t NumTokens( void ) const
{
return mNumTokens;
}
const char * GetToken( const size_t token_idx ) const
{
assert( token_idx < mNumTokens );
return mTokens[ token_idx ];
}
private:
const size_t mStorageSize;
char *mBuff;
char **mTokens;
size_t mNumTokens;
inline void ResetContent( void )
{
memset( mBuff, 0, mStorageSize );
// mark all items as empty:
memset( mTokens, 0, mStorageSize * sizeof( char* ) );
// reset counter for found items:
mNumTokens = 0L;
}
};
Implementattion文件:
TextLineSplitter::TextLineSplitter( const size_t max_line_len ):
mStorageSize ( max_line_len + 1L )
{
// allocate memory
mBuff = new char [ mStorageSize ];
mTokens = new char* [ mStorageSize ];
ResetContent();
}
TextLineSplitter::~TextLineSplitter()
{
delete [] mBuff;
delete [] mTokens;
}
void TextLineSplitter::SplitLine( const char *line,
const char sep_char /* = ',' */,
)
{
assert( sep_char != '\0' );
ResetContent();
strncpy( mBuff, line, mMaxLineLen );
size_t idx = 0L; // running index for characters
do
{
assert( idx < mStorageSize );
const char chr = line[ idx ]; // retrieve current character
if( mTokens[ mNumTokens ] == NULL )
{
mTokens[ mNumTokens ] = &mBuff[ idx ];
} // if
if( chr == sep_char || chr == '\0' )
{ // item or line finished
// overwrite separator with a 0-terminating character:
mBuff[ idx ] = '\0';
// count-up items:
mNumTokens ++;
} // if
} while( line[ idx++ ] );
}
使用的场景是:
// create an instance capable of splitting strings up to 1000 chars long:
TextLineSplitter spl( 1000 );
spl.SplitLine( "Item1,,Item2,Item3" );
for( size_t i = 0; i < spl.NumTokens(); i++ )
{
printf( "%s\n", spl.GetToken( i ) );
}
输出:
Item1
Item2
Item3
简单的c++代码(标准c++ 98),接受多个分隔符(在std::string中指定),只使用向量、字符串和迭代器。
#include <iostream>
#include <vector>
#include <string>
#include <stdexcept>
std::vector<std::string>
split(const std::string& str, const std::string& delim){
std::vector<std::string> result;
if (str.empty())
throw std::runtime_error("Can not tokenize an empty string!");
std::string::const_iterator begin, str_it;
begin = str_it = str.begin();
do {
while (delim.find(*str_it) == std::string::npos && str_it != str.end())
str_it++; // find the position of the first delimiter in str
std::string token = std::string(begin, str_it); // grab the token
if (!token.empty()) // empty token only when str starts with a delimiter
result.push_back(token); // push the token into a vector<string>
while (delim.find(*str_it) != std::string::npos && str_it != str.end())
str_it++; // ignore the additional consecutive delimiters
begin = str_it; // process the remaining tokens
} while (str_it != str.end());
return result;
}
int main() {
std::string test_string = ".this is.a.../.simple;;test;;;END";
std::string delim = "; ./"; // string containing the delimiters
std::vector<std::string> tokens = split(test_string, delim);
for (std::vector<std::string>::const_iterator it = tokens.begin();
it != tokens.end(); it++)
std::cout << *it << std::endl;
}
这是一个简单的stl解决方案(~5行!)使用std::find和std::find_first_not_of来处理重复的分隔符(例如空格或句号),以及开头和结尾的分隔符:
#include <string>
#include <vector>
void tokenize(std::string str, std::vector<string> &token_v){
size_t start = str.find_first_not_of(DELIMITER), end=start;
while (start != std::string::npos){
// Find next occurence of delimiter
end = str.find(DELIMITER, start);
// Push back the token found into vector
token_v.push_back(str.substr(start, end-start));
// Skip all occurences of the delimiter to find new start
start = str.find_first_not_of(DELIMITER, end);
}
}
现场试试吧!
c++标准库算法普遍基于迭代器,而不是具体的容器。不幸的是,这使得在c++标准库中很难提供类似java的split函数,尽管没有人认为这很方便。但是它的返回类型是什么呢?std::向量< std:: basic_string <…> >吗?也许吧,但这样我们就被迫执行(可能是冗余的和昂贵的)分配。
相反,c++提供了大量基于任意复杂的分隔符分割字符串的方法,但它们都没有像其他语言中那样封装得很好。各种各样的方法填满了整个博客文章。
在最简单的情况下,你可以使用std::string::find进行迭代,直到你击中std::string::npos,然后使用std::string::substr提取内容。
一个更流畅的(和惯用的,但基本的)版本在空格上拆分将使用std::istringstream:
auto iss = std::istringstream{"The quick brown fox"};
auto str = std::string{};
while (iss >> str) {
process(str);
}
使用std::istream_iterators,还可以使用vector的迭代器范围构造函数将string流的内容复制到vector中。
多个库(如Boost.Tokenizer)提供特定的标记器。
更高级的分裂需要正则表达式。c++特别为此提供了std::regex_token_iterator:
auto const str = "The quick brown fox"s;
auto const re = std::regex{R"(\s+)"};
auto const vec = std::vector<std::string>(
std::sregex_token_iterator{begin(str), end(str), re, -1},
std::sregex_token_iterator{}
);