如果您强制要求单元测试的代码覆盖率的最低百分比,甚至可能作为提交到存储库的要求,它会是什么?

请解释你是如何得出你的答案的(因为如果你所做的只是选择一个数字,那么我自己也可以完成;)


当前回答

在我看来,答案是“这取决于你有多少时间”。我试着达到100%,但如果我没有在我拥有的时间内完成它,我也不会大惊小怪。

当我编写单元测试时,我戴着与开发产品代码时不同的帽子。我考虑测试的代码声称要做什么,以及可能破坏它的情况是什么。

我通常遵循以下标准或规则:

单元测试应该是关于我的代码的预期行为的一种文档形式。给定特定输入的预期输出以及它可能抛出的客户端可能想要捕获的异常(我的代码的用户应该知道什么?) 单元测试应该帮助我发现我可能还没有想到的假设条件。(如何使我的代码稳定和健壮?)

如果这两条规则不能产生100%的覆盖率,那就顺其自然吧。但是一旦我有时间,我就会分析未覆盖的块和行,并确定是否仍然存在没有单元测试的测试用例,或者是否需要重构代码以消除不必要的代码。

其他回答

我更喜欢做BDD,它使用自动化验收测试、可能还有其他集成测试和单元测试的组合。对我来说,问题是自动化测试套件作为一个整体的目标覆盖率应该是多少。

That aside, the answer depends on your methodology, language and testing and coverage tools. When doing TDD in Ruby or Python it's not hard to maintain 100% coverage, and it's well worth doing so. It's much easier to manage 100% coverage than 90-something percent coverage. That is, it's much easier to fill coverage gaps as they appear (and when doing TDD well coverage gaps are rare and usually worth your time) than it is to manage a list of coverage gaps that you haven't gotten around to and miss coverage regressions due to your constant background of uncovered code.

答案也取决于项目的历史。我发现上述方法只适用于从一开始就以这种方式管理的项目。我已经极大地改进了大型遗留项目的覆盖率,这样做是值得的,但是我从来没有发现回过头去填补每个覆盖率空白是可行的,因为旧的未经测试的代码不能很好地理解,不能正确和快速地完成这些工作。

简单回答:60-80%

长一点的回答: 我认为这完全取决于你项目的性质。我通常通过单元测试每一个实际部分来开始一个项目。在项目的第一次“发布”时,你应该有一个相当不错的基于你所做的编程类型的基础百分比。在这一点上,您可以开始“强制”最小的代码覆盖率。

一般来说,从我读过的几篇工程卓越最佳实践论文来看,单元测试中80%的新代码是产生最佳回报的点。如果超过这个CC%,所付出的努力就会产生更少的缺陷。这是许多大公司所采用的最佳实践。

不幸的是,这些结果大多是公司内部的,所以我没有公开的文献可供参考。

代码覆盖率只是另一个度量。就其本身而言,它可能非常具有误导性(参见www.thoughtworks.com/insights/blog/are-test-coverage-metrics-overrated)。因此,您的目标不应该是实现100%的代码覆盖率,而是要确保您测试了应用程序的所有相关场景。

对于一个设计良好的系统,单元测试从一开始就驱动开发,我认为85%是一个相当低的数字。设计为可测试的小类应该不难更好地覆盖。

我们很容易用这样的话来回避这个问题:

覆盖的行不等于测试的逻辑,不应该对百分比进行过多的解读。

没错,但是关于代码覆盖有一些重要的地方需要注意。根据我的经验,如果使用得当,这个指标实际上非常有用。话虽如此,我并没有见过所有的系统,我敢肯定有很多系统很难看到代码覆盖率分析增加任何真正的价值。代码可能看起来很不一样,可用测试框架的范围也可能不同。

此外,我的推理主要涉及相当短的测试反馈循环。对于我正在开发的产品,最短的反馈循环非常灵活,涵盖了从类测试到进程间信号的所有内容。测试一个可交付的子产品通常需要5分钟,对于这样短的反馈循环,确实可以使用测试结果(特别是我们在这里看到的代码覆盖率指标)来拒绝或接受存储库中的提交。

当使用代码覆盖率度量时,您不应该只有一个必须实现的固定(任意)百分比。在我看来,这样做并不能给您带来代码覆盖率分析的真正好处。相反,定义以下指标:

低水位标记(LWM),在测试系统中所见过的最低裸露线数 高水位标记(HWM),在测试系统中所见过的最高代码覆盖率

只有在不超过LWM和不低于HWM的情况下,才能添加新代码。换句话说,不允许减少代码覆盖率,并且应该覆盖新代码。注意我如何说应该和不必须(下面解释)。

但这难道不意味着,你将不可能清理那些久经考验、不再有用的旧垃圾吗?是的,这就是为什么你在这些事情上必须务实。有些情况下必须打破规则,但根据我的经验,对于典型的日常集成来说,这些指标非常有用。他们给出了以下两个暗示。

Testable code is promoted. When adding new code you really have to make an effort to make the code testable, because you will have to try and cover all of it with your test cases. Testable code is usually a good thing. Test coverage for legacy code is increasing over time. When adding new code and not being able to cover it with a test case, one can try to cover some legacy code instead to get around the LWM rule. This sometimes necessary cheating at least gives the positive side effect that the coverage of legacy code will increase over time, making the seemingly strict enforcement of these rules quite pragmatic in practice.

同样,如果反馈循环太长,在集成过程中设置这样的东西可能是完全不切实际的。

我还想提到代码覆盖度量的另外两个一般好处。

Code coverage analysis is part of the dynamic code analysis (as opposed to the static one, i.e. Lint). Problems found during the dynamic code analysis (by tools such as the purify family, http://www-03.ibm.com/software/products/en/rational-purify-family) are things like uninitialized memory reads (UMR), memory leaks, etc. These problems can only be found if the code is covered by an executed test case. The code that is the hardest to cover in a test case is usually the abnormal cases in the system, but if you want the system to fail gracefully (i.e. error trace instead of crash) you might want to put some effort into covering the abnormal cases in the dynamic code analysis as well. With just a little bit of bad luck, a UMR can lead to a segfault or worse. People take pride in keeping 100% for new code, and people discuss testing problems with a similar passion as other implementation problems. How can this function be written in a more testable manner? How would you go about trying to cover this abnormal case, etc.

为了完整起见,一个是否定的。

In a large project with many involved developers, everyone is not going to be a test-genius for sure. Some people tend to use the code coverage metric as proof that the code is tested and this is very far from the truth, as mentioned in many of the other answers to this question. It is ONE metric that can give you some nice benefits if used properly, but if it is misused it can in fact lead to bad testing. Aside from the very valuable side effects mentioned above a covered line only shows that the system under test can reach that line for some input data and that it can execute without hanging or crashing.