我从各种RSS订阅中阅读大量文本，并将它们插入到我的数据库中。
当然，在提要中使用了几种不同的字符编码，例如UTF-8和ISO 8859-1。
不幸的是，文本的编码有时会有问题。例子:
“Fußball”中的“ß”在我的数据库中应该是这样的:“Ÿ”。如果是“Ÿ”，则显示正确。
有时，“Fußball”中的“ß”在我的数据库中看起来像这样:“ß”。当然，这样就会显示错误。
在其他情况下，“ß”被保存为“ß”-因此没有任何变化。然后它也会被错误地显示。
我怎么做才能避免情况2和3?
我如何使所有的编码相同，最好是UTF-8?什么时候我必须使用utf8_encode()，什么时候我必须使用utf8_decode()(很清楚的效果是什么，但什么时候我必须使用函数?)，什么时候我必须对输入什么都不做?
如何让所有编码都相同呢?也许使用函数mb_detect_encoding()?我能写一个函数吗?所以我的问题是:
如何找出文本使用的编码?
我如何将其转换为UTF-8 -无论旧的编码是什么?
这样的函数可行吗?
function correct_encoding($text) {
$current_encoding = mb_detect_encoding($text, 'auto');
$text = iconv($current_encoding, 'UTF-8', $text);
return $text;
}
我已经测试过了，但是不行。有什么问题吗?
从头文件中获取编码并将其转换为UTF-8。
$post_url = 'http://website.domain';
/// Get headers ///////////////////////////////////////////////
function get_headers_curl($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 15);
$r = curl_exec($ch);
return $r;
}
$the_header = get_headers_curl($post_url);
/// Check for redirect ////////////////////////////////////////
if (preg_match("/Location:/i", $the_header)) {
$arr = explode('Location:', $the_header);
$location = $arr[1];
$location = explode(chr(10), $location);
$location = $location[0];
$the_header = get_headers_curl(trim($location));
}
/// Get charset ///////////////////////////////////////////////
if (preg_match("/charset=/i", $the_header)) {
$arr = explode('charset=', $the_header);
$charset = $arr[1];
$charset = explode(chr(10), $charset);
$charset = $charset[0];
}
///////////////////////////////////////////////////////////////////
// echo $charset;
if($charset && $charset != 'UTF-8') {
$html = iconv($charset, "UTF-8", $html);
}
检测编码是困难的。
mb_detect_encoding works by guessing, based on a number of candidates that you pass it. In some encodings, certain byte-sequences are invalid, an therefore it can distinguish between various candidates. Unfortunately, there are a lot of encodings, where the same bytes are valid (but different). In these cases, there is no way to determine the encoding; You can implement your own logic to make guesses in these cases. For example, data coming from a Japanese site might be more likely to have a Japanese encoding.
As long as you only deal with Western European languages, the three major encodings to consider are utf-8, iso-8859-1 and cp-1252. Since these are defaults for many platforms, they are also the most likely to be reported wrongly about. Eg. if people use different encodings, they are likely to be frank about it, since else their software would break very often. Therefore, a good strategy is to trust the provider, unless the encoding is reported as one of those three. You should still doublecheck that it is indeed valid, using mb_check_encoding (note that valid is not the same as being - the same input may be valid for many encodings). If it is one of those, you can then use mb_detect_encoding to distinguish between them. Luckily that is fairly deterministic; You just need to use the proper detect-sequence, which is UTF-8,ISO-8859-1,WINDOWS-1252.
一旦检测到编码，就需要将其转换为内部表示(UTF-8是唯一明智的选择)。函数utf8_encode将ISO-8859-1转换为UTF-8，因此它只能用于特定的输入类型。对于其他编码，使用mb_convert_encoding。
我在http://deer.org.ua/2009/10/06/1/:上找到了一个解决方案
class Encoding
{
/**
* http://deer.org.ua/2009/10/06/1/
* @param $string
* @return null
*/
public static function detect_encoding($string)
{
static $list = ['utf-8', 'windows-1251'];
foreach ($list as $item) {
try {
$sample = iconv($item, $item, $string);
} catch (\Exception $e) {
continue;
}
if (md5($sample) == md5($string)) {
return $item;
}
}
return null;
}
}
$content = file_get_contents($file['tmp_name']);
$encoding = Encoding::detect_encoding($content);
if ($encoding != 'utf-8') {
$result = iconv($encoding, 'utf-8', $content);
} else {
$result = $content;
}
我认为@是一个糟糕的决定，并对deer.org.ua的解决方案做了一些更改。
