测试：十六进制字符串到字节数组

我注意到，大多数测试都是在将Bytes数组转换为十六进制字符串的函数上执行的。因此，在这篇文章中，我将关注另一方面：将十六进制字符串转换为字节数组的函数。若您只对结果感兴趣，可以跳到“摘要”部分。测试代码文件在文章末尾提供。

标签

我想根据接受的答案（Tomalak）将函数命名为StringToByteArrayV1，或将其快捷到V1。其余函数将以相同的方式命名：V2、V3、V4、…、。。。，等

参与功能索引

Tomalak的StringToByteArrayV1（公认答案）Mykroft的StringToByteArrayV2（使用SoapHexBinary）drphrozen的StringToByteArrayV3（查找表）CoperNick的StringToByteArrayV4（字节操作）Chris F编写的StringToByteArrayV5_1（字节操作）Chris F的StringToByteArrayV5_2（V5_1+根据Amir Rezaei的评论修改）Chris F的StringToByteArrayV5_3（V5_2+根据Ben Voigt的评论对其进行了修改）（您可以在发布后的测试代码中看到它的最终形状）Ben Mosher编写的StringToByteArrayV6（字节操作）Maratius的StringToByteArrayV7（字节操作-安全版本）Maratius的StringToByteArrayV8（字节操作-不安全版本）StringToByteArrayV9（按Geograph）AlejandroAlis编写的StringToByteArrayV10Fredrik Hu编写的StringToByteArrayV11Maarten Bodewes编写的StringToByteArrayV12ClausAndersen编写的StringToByteArrayV13Stas Makutin编写的StringToByteArrayV14JJJ的StringToByteArrayV15JamieSee的StringToByteArrayV16spacepille的StringToByteArrayV17Gregory Morse编写的StringToByteArrayV18Rick编写的StringToByteArrayV19SandRock的StringToByteArrayV20Paul编写的StringToByteArrayV21

正确性测试

我通过传递1字节的所有256个可能值来测试正确性，然后检查输出是否正确。结果：

V18中以“00”开头的字符串有问题（请参阅Roger Stewart对此的评论）。除了通过所有测试。如果十六进制字符串字母是大写的：所有函数都成功传递如果十六进制字符串字母是小写的，则以下函数失败：V5_1、V5_2、v7、V8、V15、V19

注：V5_3解决了这个问题（V5_1和V5_2）

性能测试

我已经使用Stopwatch类进行了性能测试。

长字符串的性能

input length: 10,000,000 bytes
runs: 100
average elapsed time per run:
V1 = 136.4ms
V2 = 104.5ms
V3 = 22.0ms
V4 = 9.9ms
V5_1 = 10.2ms
V5_2 = 9.0ms
V5_3 = 9.3ms
V6 = 18.3ms
V7 = 9.8ms
V8 = 8.8ms
V9 = 10.2ms
V10 = 19.0ms
V11 = 12.2ms
V12 = 27.4ms
V13 = 21.8ms
V14 = 12.0ms
V15 = 14.9ms
V16 = 15.3ms
V17 = 9.5ms
V18 got excluded from this test, because it was very slow when using very long string
V19 = 222.8ms
V20 = 66.0ms
V21 = 15.4ms

V1 average ticks per run: 1363529.4
V2 is more fast than V1 by: 1.3 times (ticks ratio)
V3 is more fast than V1 by: 6.2 times (ticks ratio)
V4 is more fast than V1 by: 13.8 times (ticks ratio)
V5_1 is more fast than V1 by: 13.3 times (ticks ratio)
V5_2 is more fast than V1 by: 15.2 times (ticks ratio)
V5_3 is more fast than V1 by: 14.8 times (ticks ratio)
V6 is more fast than V1 by: 7.4 times (ticks ratio)
V7 is more fast than V1 by: 13.9 times (ticks ratio)
V8 is more fast than V1 by: 15.4 times (ticks ratio)
V9 is more fast than V1 by: 13.4 times (ticks ratio)
V10 is more fast than V1 by: 7.2 times (ticks ratio)
V11 is more fast than V1 by: 11.1 times (ticks ratio)
V12 is more fast than V1 by: 5.0 times (ticks ratio)
V13 is more fast than V1 by: 6.3 times (ticks ratio)
V14 is more fast than V1 by: 11.4 times (ticks ratio)
V15 is more fast than V1 by: 9.2 times (ticks ratio)
V16 is more fast than V1 by: 8.9 times (ticks ratio)
V17 is more fast than V1 by: 14.4 times (ticks ratio)
V19 is more SLOW than V1 by: 1.6 times (ticks ratio)
V20 is more fast than V1 by: 2.1 times (ticks ratio)
V21 is more fast than V1 by: 8.9 times (ticks ratio)

V18的长串性能

V18 took long time at the previous test, 
so let's decrease length for it:  
input length: 1,000,000 bytes
runs: 100
average elapsed time per run: V1 = 14.1ms , V18 = 146.7ms
V1 average ticks per run: 140630.3
V18 is more SLOW than V1 by: 10.4 times (ticks ratio)

短字符串的性能

input length: 100 byte
runs: 1,000,000
V1 average ticks per run: 14.6
V2 is more fast than V1 by: 1.4 times (ticks ratio)
V3 is more fast than V1 by: 5.9 times (ticks ratio)
V4 is more fast than V1 by: 15.7 times (ticks ratio)
V5_1 is more fast than V1 by: 15.1 times (ticks ratio)
V5_2 is more fast than V1 by: 18.4 times (ticks ratio)
V5_3 is more fast than V1 by: 16.3 times (ticks ratio)
V6 is more fast than V1 by: 5.3 times (ticks ratio)
V7 is more fast than V1 by: 15.7 times (ticks ratio)
V8 is more fast than V1 by: 18.0 times (ticks ratio)
V9 is more fast than V1 by: 15.5 times (ticks ratio)
V10 is more fast than V1 by: 7.8 times (ticks ratio)
V11 is more fast than V1 by: 12.4 times (ticks ratio)
V12 is more fast than V1 by: 5.3 times (ticks ratio)
V13 is more fast than V1 by: 5.2 times (ticks ratio)
V14 is more fast than V1 by: 13.4 times (ticks ratio)
V15 is more fast than V1 by: 9.9 times (ticks ratio)
V16 is more fast than V1 by: 9.2 times (ticks ratio)
V17 is more fast than V1 by: 16.2 times (ticks ratio)
V18 is more fast than V1 by: 1.1 times (ticks ratio)
V19 is more SLOW than V1 by: 1.6 times (ticks ratio)
V20 is more fast than V1 by: 1.9 times (ticks ratio)
V21 is more fast than V1 by: 11.4 times (ticks ratio)

测试代码

在使用以下代码之前，最好先阅读本文下面的免责声明部分https://github.com/Ghosticollis/performance-tests/blob/main/MTestPerformance.cs

总结

由于性能良好，我建议使用以下函数之一，并支持大写和小写：

CoperNick的StringToByteArrayV4StringToByteArrayV9（按Geograph）spacepille的StringToByteArrayV17StringToByteArrayV5_3基本上由Chris F开发（它基于V5_1，但我根据Amir Rezaei和Ben Voigt的评论对其进行了增强）。

以下是V5_3的最终形状：

static byte[] HexStringToByteArrayV5_3(string hexString) {
    int hexStringLength = hexString.Length;
    byte[] b = new byte[hexStringLength / 2];
    for (int i = 0; i < hexStringLength; i += 2) {
        int topChar = hexString[i];
        topChar = (topChar > 0x40 ? (topChar & ~0x20) - 0x37 : topChar - 0x30) << 4;
        int bottomChar = hexString[i + 1];
        bottomChar = bottomChar > 0x40 ? (bottomChar & ~0x20) - 0x37 : bottomChar - 0x30;
        b[i / 2] = (byte)(topChar + bottomChar);
    }
    return b;
}

免责声明

警告：我没有适当的测试知识。这些原始测试的主要目的是快速概述所有发布的函数的优点。如果您需要准确的结果，请使用适当的测试工具。

最后，我想说，我是新来的，在斯塔科弗洛活跃，如果我的职位空缺，我很抱歉。如果您能发表评论，我们将不胜感激。

两个mashup，将两个半字节操作合并为一个。

可能非常有效的版本：

public static string ByteArrayToString2(byte[] ba)
{
    char[] c = new char[ba.Length * 2];
    for( int i = 0; i < ba.Length * 2; ++i)
    {
        byte b = (byte)((ba[i>>1] >> 4*((i&1)^1)) & 0xF);
        c[i] = (char)(55 + b + (((b-10)>>31)&-7));
    }
    return new string( c );
}

Decadent linq与比特黑客版本：

public static string ByteArrayToString(byte[] ba)
{
    return string.Concat( ba.SelectMany( b => new int[] { b >> 4, b & 0xF }).Select( b => (char)(55 + b + (((b-10)>>31)&-7))) );
}

并反转：

public static byte[] HexStringToByteArray( string s )
{
    byte[] ab = new byte[s.Length>>1];
    for( int i = 0; i < s.Length; i++ )
    {
        int b = s[i];
        b = (b - '0') + ((('9' - b)>>31)&-7);
        ab[i>>1] |= (byte)(b << 4*((i&1)^1));
    }
    return ab;
}

我将参加这个比特拨弄比赛，因为我有一个同样使用比特拨弄来解码十六进制的答案。请注意，使用字符数组可能会更快，因为调用StringBuilder方法也需要时间。

public static String ToHex (byte[] data)
{
    int dataLength = data.Length;
    // pre-create the stringbuilder using the length of the data * 2, precisely enough
    StringBuilder sb = new StringBuilder (dataLength * 2);
    for (int i = 0; i < dataLength; i++) {
        int b = data [i];

        // check using calculation over bits to see if first tuple is a letter
        // isLetter is zero if it is a digit, 1 if it is a letter
        int isLetter = (b >> 7) & ((b >> 6) | (b >> 5)) & 1;

        // calculate the code using a multiplication to make up the difference between
        // a digit character and an alphanumerical character
        int code = '0' + ((b >> 4) & 0xF) + isLetter * ('A' - '9' - 1);
        // now append the result, after casting the code point to a character
        sb.Append ((Char)code);

        // do the same with the lower (less significant) tuple
        isLetter = (b >> 3) & ((b >> 2) | (b >> 1)) & 1;
        code = '0' + (b & 0xF) + isLetter * ('A' - '9' - 1);
        sb.Append ((Char)code);
    }
    return sb.ToString ();
}

public static byte[] FromHex (String hex)
{

    // pre-create the array
    int resultLength = hex.Length / 2;
    byte[] result = new byte[resultLength];
    // set validity = 0 (0 = valid, anything else is not valid)
    int validity = 0;
    int c, isLetter, value, validDigitStruct, validDigit, validLetterStruct, validLetter;
    for (int i = 0, hexOffset = 0; i < resultLength; i++, hexOffset += 2) {
        c = hex [hexOffset];

        // check using calculation over bits to see if first char is a letter
        // isLetter is zero if it is a digit, 1 if it is a letter (upper & lowercase)
        isLetter = (c >> 6) & 1;

        // calculate the tuple value using a multiplication to make up the difference between
        // a digit character and an alphanumerical character
        // minus 1 for the fact that the letters are not zero based
        value = ((c & 0xF) + isLetter * (-1 + 10)) << 4;

        // check validity of all the other bits
        validity |= c >> 7; // changed to >>, maybe not OK, use UInt?

        validDigitStruct = (c & 0x30) ^ 0x30;
        validDigit = ((c & 0x8) >> 3) * (c & 0x6);
        validity |= (isLetter ^ 1) * (validDigitStruct | validDigit);

        validLetterStruct = c & 0x18;
        validLetter = (((c - 1) & 0x4) >> 2) * ((c - 1) & 0x2);
        validity |= isLetter * (validLetterStruct | validLetter);

        // do the same with the lower (less significant) tuple
        c = hex [hexOffset + 1];
        isLetter = (c >> 6) & 1;
        value ^= (c & 0xF) + isLetter * (-1 + 10);
        result [i] = (byte)value;

        // check validity of all the other bits
        validity |= c >> 7; // changed to >>, maybe not OK, use UInt?

        validDigitStruct = (c & 0x30) ^ 0x30;
        validDigit = ((c & 0x8) >> 3) * (c & 0x6);
        validity |= (isLetter ^ 1) * (validDigitStruct | validDigit);

        validLetterStruct = c & 0x18;
        validLetter = (((c - 1) & 0x4) >> 2) * ((c - 1) & 0x2);
        validity |= isLetter * (validLetterStruct | validLetter);
    }

    if (validity != 0) {
        throw new ArgumentException ("Hexadecimal encoding incorrect for input " + hex);
    }

    return result;
}

从Java代码转换而来。

如果性能很重要，这里有一个优化的解决方案：

    static readonly char[] _hexDigits = "0123456789abcdef".ToCharArray();
    public static string ToHexString(this byte[] bytes)
    {
        char[] digits = new char[bytes.Length * 2];
        for (int i = 0; i < bytes.Length; i++)
        {
            int d1, d2;
            d1 = Math.DivRem(bytes[i], 16, out d2);
            digits[2 * i] = _hexDigits[d1];
            digits[2 * i + 1] = _hexDigits[d2];
        }
        return new string(digits);
    }

它比BitConverter.ToString快2.5倍，比BitConverter.ToString+删除“-”字符快7倍。

下面还通过允许本机小写选项扩展了这里的优秀答案，并且还处理null或空输入，并使其成为扩展方法。

    /// <summary>
    /// Converts the byte array to a hex string very fast. Excellent job
    /// with code lightly adapted from 'community wiki' here: https://stackoverflow.com/a/14333437/264031
    /// (the function was originally named: ByteToHexBitFiddle). Now allows a native lowerCase option
    /// to be input and allows null or empty inputs (null returns null, empty returns empty).
    /// </summary>
    public static string ToHexString(this byte[] bytes, bool lowerCase = false)
    {
        if (bytes == null)
            return null;
        else if (bytes.Length == 0)
            return "";

        char[] c = new char[bytes.Length * 2];

        int b;
        int xAddToAlpha = lowerCase ? 87 : 55;
        int xAddToDigit = lowerCase ? -39 : -7;

        for (int i = 0; i < bytes.Length; i++) {

            b = bytes[i] >> 4;
            c[i * 2] = (char)(xAddToAlpha + b + (((b - 10) >> 31) & xAddToDigit));

            b = bytes[i] & 0xF;
            c[i * 2 + 1] = (char)(xAddToAlpha + b + (((b - 10) >> 31) & xAddToDigit));
        }

        string val = new string(c);
        return val;
    }

    public static string ToHexString(this IEnumerable<byte> bytes, bool lowerCase = false)
    {
        if (bytes == null)
            return null;
        byte[] arr = bytes.ToArray();
        return arr.ToHexString(lowerCase);
    }

我想出了一个不同的代码，它可以容忍额外的字符（空格、破折号…）。它的灵感主要来自这里一些可以接受的快速答案。它允许解析以下“文件”

00-aa-84-fb
12 32 FF CD
12 00
12_32_FF_CD
1200d5e68a

/// <summary>Reads a hex string into bytes</summary>
public static IEnumerable<byte> HexadecimalStringToBytes(string hex) {
    if (hex == null)
        throw new ArgumentNullException(nameof(hex));

    char c, c1 = default(char);
    bool hasc1 = false;
    unchecked   {
        for (int i = 0; i < hex.Length; i++) {
            c = hex[i];
            bool isValid = 'A' <= c && c <= 'f' || 'a' <= c && c <= 'f' || '0' <= c && c <= '9';
            if (!hasc1) {
                if (isValid) {
                    hasc1 = true;
                }
            } else {
                hasc1 = false;
                if (isValid) {
                    yield return (byte)((GetHexVal(c1) << 4) + GetHexVal(c));
                }
            }

            c1 = c;
        } 
    }
}

/// <summary>Reads a hex string into a byte array</summary>
public static byte[] HexadecimalStringToByteArray(string hex)
{
    if (hex == null)
        throw new ArgumentNullException(nameof(hex));

    var bytes = new List<byte>(hex.Length / 2);
    foreach (var item in HexadecimalStringToBytes(hex)) {
        bytes.Add(item);
    }

    return bytes.ToArray();
}

private static byte GetHexVal(char val)
{
    return (byte)(val - (val < 0x3A ? 0x30 : val < 0x5B ? 0x37 : 0x57));
    //                   ^^^^^^^^^^^^^^^^^   ^^^^^^^^^^^^^^^^^   ^^^^
    //                       digits 0-9       upper char A-Z     a-z
}

复制时请参考完整代码。包括单元测试。

有些人可能会说它对额外的字符太宽容了。因此，不要依赖此代码来执行验证（或更改）。