到目前为止,我已经避免了测试多线程代码的噩梦,因为它似乎是一个太大的雷区。我想知道人们是如何测试依赖于线程的代码以获得成功执行的,或者人们是如何测试那些仅在两个线程以给定方式交互时才会出现的问题的?

对于今天的程序员来说,这似乎是一个非常关键的问题,恕我直言,将我们的知识集中在这个问题上是很有用的。


当前回答

它并不完美,但我用c#写了这个帮助程序:

using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;

namespace Proto.Promises.Tests.Threading
{
    public class ThreadHelper
    {
        public static readonly int multiThreadCount = Environment.ProcessorCount * 100;
        private static readonly int[] offsets = new int[] { 0, 10, 100, 1000 };

        private readonly Stack<Task> _executingTasks = new Stack<Task>(multiThreadCount);
        private readonly Barrier _barrier = new Barrier(1);
        private int _currentParticipants = 0;
        private readonly TimeSpan _timeout;

        public ThreadHelper() : this(TimeSpan.FromSeconds(10)) { } // 10 second timeout should be enough for most cases.

        public ThreadHelper(TimeSpan timeout)
        {
            _timeout = timeout;
        }

        /// <summary>
        /// Execute the action multiple times in parallel threads.
        /// </summary>
        public void ExecuteMultiActionParallel(Action action)
        {
            for (int i = 0; i < multiThreadCount; ++i)
            {
                AddParallelAction(action);
            }
            ExecutePendingParallelActions();
        }

        /// <summary>
        /// Execute the action once in a separate thread.
        /// </summary>
        public void ExecuteSingleAction(Action action)
        {
            AddParallelAction(action);
            ExecutePendingParallelActions();
        }

        /// <summary>
        /// Add an action to be run in parallel.
        /// </summary>
        public void AddParallelAction(Action action)
        {
            var taskSource = new TaskCompletionSource<bool>();
            lock (_executingTasks)
            {
                ++_currentParticipants;
                _barrier.AddParticipant();
                _executingTasks.Push(taskSource.Task);
            }
            new Thread(() =>
            {
                try
                {
                    _barrier.SignalAndWait(); // Try to make actions run in lock-step to increase likelihood of breaking race conditions.
                    action.Invoke();
                    taskSource.SetResult(true);
                }
                catch (Exception e)
                {
                    taskSource.SetException(e);
                }
            }).Start();
        }

        /// <summary>
        /// Runs the pending actions in parallel, attempting to run them in lock-step.
        /// </summary>
        public void ExecutePendingParallelActions()
        {
            Task[] tasks;
            lock (_executingTasks)
            {
                _barrier.SignalAndWait();
                _barrier.RemoveParticipants(_currentParticipants);
                _currentParticipants = 0;
                tasks = _executingTasks.ToArray();
                _executingTasks.Clear();
            }
            try
            {
                if (!Task.WaitAll(tasks, _timeout))
                {
                    throw new TimeoutException($"Action(s) timed out after {_timeout}, there may be a deadlock.");
                }
            }
            catch (AggregateException e)
            {
                // Only throw one exception instead of aggregate to try to avoid overloading the test error output.
                throw e.Flatten().InnerException;
            }
        }

        /// <summary>
        /// Run each action in parallel multiple times with differing offsets for each run.
        /// <para/>The number of runs is 4^actions.Length, so be careful if you don't want the test to run too long.
        /// </summary>
        /// <param name="expandToProcessorCount">If true, copies each action on additional threads up to the processor count. This can help test more without increasing the time it takes to complete.
        /// <para/>Example: 2 actions with 6 processors, runs each action 3 times in parallel.</param>
        /// <param name="setup">The action to run before each parallel run.</param>
        /// <param name="teardown">The action to run after each parallel run.</param>
        /// <param name="actions">The actions to run in parallel.</param>
        public void ExecuteParallelActionsWithOffsets(bool expandToProcessorCount, Action setup, Action teardown, params Action[] actions)
        {
            setup += () => { };
            teardown += () => { };
            int actionCount = actions.Length;
            int expandCount = expandToProcessorCount ? Math.Max(Environment.ProcessorCount / actionCount, 1) : 1;
            foreach (var combo in GenerateCombinations(offsets, actionCount))
            {
                setup.Invoke();
                for (int k = 0; k < expandCount; ++k)
                {
                    for (int i = 0; i < actionCount; ++i)
                    {
                        int offset = combo[i];
                        Action action = actions[i];
                        AddParallelAction(() =>
                        {
                            for (int j = offset; j > 0; --j) { } // Just spin in a loop for the offset.
                            action.Invoke();
                        });
                    }
                }
                ExecutePendingParallelActions();
                teardown.Invoke();
            }
        }

        // Input: [1, 2, 3], 3
        // Ouput: [
        //          [1, 1, 1],
        //          [2, 1, 1],
        //          [3, 1, 1],
        //          [1, 2, 1],
        //          [2, 2, 1],
        //          [3, 2, 1],
        //          [1, 3, 1],
        //          [2, 3, 1],
        //          [3, 3, 1],
        //          [1, 1, 2],
        //          [2, 1, 2],
        //          [3, 1, 2],
        //          [1, 2, 2],
        //          [2, 2, 2],
        //          [3, 2, 2],
        //          [1, 3, 2],
        //          [2, 3, 2],
        //          [3, 3, 2],
        //          [1, 1, 3],
        //          [2, 1, 3],
        //          [3, 1, 3],
        //          [1, 2, 3],
        //          [2, 2, 3],
        //          [3, 2, 3],
        //          [1, 3, 3],
        //          [2, 3, 3],
        //          [3, 3, 3]
        //        ]
        private static IEnumerable<int[]> GenerateCombinations(int[] options, int count)
        {
            int[] indexTracker = new int[count];
            int[] combo = new int[count];
            for (int i = 0; i < count; ++i)
            {
                combo[i] = options[0];
            }
            // Same algorithm as picking a combination lock.
            int rollovers = 0;
            while (rollovers < count)
            {
                yield return combo; // No need to duplicate the array since we're just reading it.
                for (int i = 0; i < count; ++i)
                {
                    int index = ++indexTracker[i];
                    if (index == options.Length)
                    {
                        indexTracker[i] = 0;
                        combo[i] = options[0];
                        if (i == rollovers)
                        {
                            ++rollovers;
                        }
                    }
                    else
                    {
                        combo[i] = options[index];
                        break;
                    }
                }
            }
        }
    }
}

使用示例:

[Test]
public void DeferredMayBeBeResolvedAndPromiseAwaitedConcurrently_void0()
{
    Promise.Deferred deferred = default(Promise.Deferred);
    Promise promise = default(Promise);

    int invokedCount = 0;

    var threadHelper = new ThreadHelper();
    threadHelper.ExecuteParallelActionsWithOffsets(false,
        // Setup
        () =>
        {
            invokedCount = 0;
            deferred = Promise.NewDeferred();
            promise = deferred.Promise;
        },
        // Teardown
        () => Assert.AreEqual(1, invokedCount),
        // Parallel Actions
        () => deferred.Resolve(),
        () => promise.Then(() => { Interlocked.Increment(ref invokedCount); }).Forget()
    );
}

其他回答

听着,要做到这一点并不容易。我正在做一个本来就是多线程的项目。事件来自操作系统,我必须并发地处理它们。

处理测试复杂的多线程应用程序代码的最简单方法是:如果它太复杂而无法测试,那么您做错了。如果您有一个单独的实例,其中有多个线程作用于它,并且您无法测试这些线程相互踩在一起的情况,那么您的设计需要重做。它既简单又复杂。

有许多方法可以为多线程编程,以避免线程同时通过实例运行。最简单的方法是使所有对象都是不可变的。当然,这通常是不可能的。因此,您必须在设计中确定线程与同一实例交互的地方,并减少这些地方的数量。通过这样做,您可以隔离多线程实际发生的几个类,从而降低测试系统的总体复杂性。

但是您必须意识到,即使这样做,您仍然不能测试两个线程相互践踏的每一种情况。要做到这一点,您必须在同一个测试中并发地运行两个线程,然后准确地控制它们在任何给定时刻执行的行。你能做的就是模拟这种情况。但这可能需要您专门为测试编写代码,这充其量是迈向真正解决方案的半步。

测试代码是否存在线程问题的最好方法可能是对代码进行静态分析。如果您的线程代码没有遵循有限的线程安全模式集,那么您可能会遇到问题。我相信VS中的代码分析确实包含了一些线程的知识,但可能不多。

看,就目前的情况来看(可能还会持续很长一段时间),测试多线程应用程序的最佳方法是尽可能降低线程代码的复杂性。最小化线程交互的区域,尽可能地进行测试,并使用代码分析来识别危险区域。

For J2E code, I've used SilkPerformer, LoadRunner and JMeter for concurrency testing of threads. They all do the same thing. Basically, they give you a relatively simple interface for administrating their version of the proxy server, required, in order to analyze the TCP/IP data stream, and simulate multiple users making simultaneous requests to your app server. The proxy server can give you the ability to do things like analyze the requests made, by presenting the whole page and URL sent to the server, as well as the response from the server, after processing the request.

您可以在不安全的http模式下找到一些错误,在这种模式下,您至少可以分析正在发送的表单数据,并为每个用户系统地更改表单数据。但真正的测试是在https(安全套接字层)中运行。然后,您还必须有系统地修改会话和cookie数据,这可能有点复杂。

在测试并发性时,我发现的最好的错误是,当我发现开发人员在登录时依赖Java垃圾收集来关闭登录时建立的到LDAP服务器的连接请求。这导致用户暴露在其他用户的会话中,当试图分析服务器瘫痪时发生了什么,几乎每隔几秒钟就能完成一次事务时,结果非常令人困惑。

In the end, you or someone will probably have to buckle down and analyze the code for blunders like the one I just mentioned. And an open discussion across departments, like the one that occurred, when we unfolded the problem described above, are most useful. But these tools are the best solution to testing multi-threaded code. JMeter is open source. SilkPerformer and LoadRunner are proprietary. If you really want to know whether your app is thread safe, that's how the big boys do it. I've done this for very large companies professionally, so I'm not guessing. I'm speaking from personal experience.

提醒一句:理解这些工具确实需要一些时间。这不是简单地安装软件并启动GUI的问题,除非您已经接触过多线程编程。我试图确定需要理解的3个关键领域(表单、会话和cookie数据),希望至少从理解这些主题开始,可以帮助您集中精力快速获得结果,而不必通读整个文档。

我喜欢编写两个或多个测试方法在并行线程上执行,并且每个方法都调用被测对象。我一直在使用Sleep()调用来协调来自不同线程的调用顺序,但这并不真正可靠。它也慢得多,因为你必须睡足够长的时间,时间通常是有效的。

我从编写FindBugs的同一组中找到了多线程TC Java库。它允许您在不使用Sleep()的情况下指定事件的顺序,而且它是可靠的。我还没试过。

这种方法的最大限制是它只允许您测试您怀疑会引起麻烦的场景。正如其他人所说,您确实需要将多线程代码隔离到少量简单类中,以便有希望彻底测试它们。

一旦您仔细测试了您预计会导致问题的场景,那么在类中抛出一堆并发请求的不科学测试是寻找意外问题的好方法。

更新:我已经玩了一些多线程TC Java库,它工作得很好。我还将它的一些特性移植到一个。net版本,我称之为TickingTest。

Pete Goodliffe有一个关于线程代码单元测试的系列。

是很困难的。我采用了更简单的方法,尽量将线程代码从实际测试中抽象出来。皮特确实提到了我分手的方式是错误的但我要么是正确的,要么就是我很幸运。

我曾经有过测试线程代码的不幸任务,这绝对是我写过的最难的测试。

在编写测试时,我使用委托和事件的组合。基本上,它都是关于使用PropertyNotifyChanged事件和WaitCallback或某种轮询的ConditionalWaiter。

我不确定这是否是最好的方法,但它对我来说是有效的。