如果您强制要求单元测试的代码覆盖率的最低百分比,甚至可能作为提交到存储库的要求,它会是什么?
请解释你是如何得出你的答案的(因为如果你所做的只是选择一个数字,那么我自己也可以完成;)
如果您强制要求单元测试的代码覆盖率的最低百分比,甚至可能作为提交到存储库的要求,它会是什么?
请解释你是如何得出你的答案的(因为如果你所做的只是选择一个数字,那么我自己也可以完成;)
当前回答
当我认为我的代码没有经过足够的单元测试,并且我不确定接下来要测试什么时,我使用覆盖率来帮助我决定接下来要测试什么。
如果我在一个单元测试中增加覆盖率——我知道这个单元测试有价值。
这适用于未覆盖的代码,50%覆盖或97%覆盖。
其他回答
我认为不可能有这样的B/W规则。 应该审查代码,特别注意关键细节。 然而,如果它没有经过测试,它就有一个bug!
对于一个设计良好的系统,单元测试从一开始就驱动开发,我认为85%是一个相当低的数字。设计为可测试的小类应该不难更好地覆盖。
我们很容易用这样的话来回避这个问题:
覆盖的行不等于测试的逻辑,不应该对百分比进行过多的解读。
没错,但是关于代码覆盖有一些重要的地方需要注意。根据我的经验,如果使用得当,这个指标实际上非常有用。话虽如此,我并没有见过所有的系统,我敢肯定有很多系统很难看到代码覆盖率分析增加任何真正的价值。代码可能看起来很不一样,可用测试框架的范围也可能不同。
此外,我的推理主要涉及相当短的测试反馈循环。对于我正在开发的产品,最短的反馈循环非常灵活,涵盖了从类测试到进程间信号的所有内容。测试一个可交付的子产品通常需要5分钟,对于这样短的反馈循环,确实可以使用测试结果(特别是我们在这里看到的代码覆盖率指标)来拒绝或接受存储库中的提交。
当使用代码覆盖率度量时,您不应该只有一个必须实现的固定(任意)百分比。在我看来,这样做并不能给您带来代码覆盖率分析的真正好处。相反,定义以下指标:
低水位标记(LWM),在测试系统中所见过的最低裸露线数 高水位标记(HWM),在测试系统中所见过的最高代码覆盖率
只有在不超过LWM和不低于HWM的情况下,才能添加新代码。换句话说,不允许减少代码覆盖率,并且应该覆盖新代码。注意我如何说应该和不必须(下面解释)。
但这难道不意味着,你将不可能清理那些久经考验、不再有用的旧垃圾吗?是的,这就是为什么你在这些事情上必须务实。有些情况下必须打破规则,但根据我的经验,对于典型的日常集成来说,这些指标非常有用。他们给出了以下两个暗示。
Testable code is promoted. When adding new code you really have to make an effort to make the code testable, because you will have to try and cover all of it with your test cases. Testable code is usually a good thing. Test coverage for legacy code is increasing over time. When adding new code and not being able to cover it with a test case, one can try to cover some legacy code instead to get around the LWM rule. This sometimes necessary cheating at least gives the positive side effect that the coverage of legacy code will increase over time, making the seemingly strict enforcement of these rules quite pragmatic in practice.
同样,如果反馈循环太长,在集成过程中设置这样的东西可能是完全不切实际的。
我还想提到代码覆盖度量的另外两个一般好处。
Code coverage analysis is part of the dynamic code analysis (as opposed to the static one, i.e. Lint). Problems found during the dynamic code analysis (by tools such as the purify family, http://www-03.ibm.com/software/products/en/rational-purify-family) are things like uninitialized memory reads (UMR), memory leaks, etc. These problems can only be found if the code is covered by an executed test case. The code that is the hardest to cover in a test case is usually the abnormal cases in the system, but if you want the system to fail gracefully (i.e. error trace instead of crash) you might want to put some effort into covering the abnormal cases in the dynamic code analysis as well. With just a little bit of bad luck, a UMR can lead to a segfault or worse. People take pride in keeping 100% for new code, and people discuss testing problems with a similar passion as other implementation problems. How can this function be written in a more testable manner? How would you go about trying to cover this abnormal case, etc.
为了完整起见,一个是否定的。
In a large project with many involved developers, everyone is not going to be a test-genius for sure. Some people tend to use the code coverage metric as proof that the code is tested and this is very far from the truth, as mentioned in many of the other answers to this question. It is ONE metric that can give you some nice benefits if used properly, but if it is misused it can in fact lead to bad testing. Aside from the very valuable side effects mentioned above a covered line only shows that the system under test can reach that line for some input data and that it can execute without hanging or crashing.
Alberto Savoia的这篇散文恰好回答了这个问题(以一种非常有趣的方式!):
http://www.artima.com/forums/flat.jsp?forum=106&thread=204677
Testivus On Test Coverage Early one morning, a programmer asked the great master: “I am ready to write some unit tests. What code coverage should I aim for?” The great master replied: “Don’t worry about coverage, just write some good tests.” The programmer smiled, bowed, and left. ... Later that day, a second programmer asked the same question. The great master pointed at a pot of boiling water and said: “How many grains of rice should I put in that pot?” The programmer, looking puzzled, replied: “How can I possibly tell you? It depends on how many people you need to feed, how hungry they are, what other food you are serving, how much rice you have available, and so on.” “Exactly,” said the great master. The second programmer smiled, bowed, and left. ... Toward the end of the day, a third programmer came and asked the same question about code coverage. “Eighty percent and no less!” Replied the master in a stern voice, pounding his fist on the table. The third programmer smiled, bowed, and left. ... After this last reply, a young apprentice approached the great master: “Great master, today I overheard you answer the same question about code coverage with three different answers. Why?” The great master stood up from his chair: “Come get some fresh tea with me and let’s talk about it.” After they filled their cups with smoking hot green tea, the great master began to answer: “The first programmer is new and just getting started with testing. Right now he has a lot of code and no tests. He has a long way to go; focusing on code coverage at this time would be depressing and quite useless. He’s better off just getting used to writing and running some tests. He can worry about coverage later.” “The second programmer, on the other hand, is quite experience both at programming and testing. When I replied by asking her how many grains of rice I should put in a pot, I helped her realize that the amount of testing necessary depends on a number of factors, and she knows those factors better than I do – it’s her code after all. There is no single, simple, answer, and she’s smart enough to handle the truth and work with that.” “I see,” said the young apprentice, “but if there is no single simple answer, then why did you answer the third programmer ‘Eighty percent and no less’?” The great master laughed so hard and loud that his belly, evidence that he drank more than just green tea, flopped up and down. “The third programmer wants only simple answers – even when there are no simple answers … and then does not follow them anyway.” The young apprentice and the grizzled great master finished drinking their tea in contemplative silence.
如果这是一个完美的世界,100%的代码将被单元测试覆盖。然而,因为这不是一个完美的世界,这是一个你有时间做什么的问题。因此,我建议少关注特定的百分比,而更多地关注关键领域。如果你的代码写得很好(或者至少是一个合理的副本),应该有几个关键点将api暴露给其他代码。
将您的测试工作集中在这些api上。确保api是1)良好的文档化的,2)已经编写了与文档相匹配的测试用例。如果预期的结果与文档不匹配,那么您的代码、文档或测试用例中都存在bug。所有这些都是值得调查的。
好运!
如果你已经做了相当长一段时间的单元测试,我认为没有理由不接近95%以上。然而,至少,我总是使用80%的测试,即使是刚开始测试的时候。
这个数字应该只包括在项目中编写的代码(不包括框架、插件等),甚至可能排除完全由调用外部代码编写的代码组成的某些类。这种电话应该被嘲笑。