| Subcribe via RSS

Perl Module Monday: Devel::Cover

July 4th, 2011 Posted in CPAN, Perl

(I’m trying to get back in the habit of writing again, really.)

In my previous post, I mentioned Devel::Cover. This module has been quite a tool to add to my toolbox. Before I started using it, I thought my test suites were pretty good, overall. Now I have a much clearer picture of what I am and am not actually testing.

First, a bit on what coverage analysis is, and what it isn’t.

At the simplest level, coverage analysis is counting how often you reach each line of code in your software. If you don’t hit a particular line at all, then that line hasn’t been tested. Of course, there’s more to it than that— there’s branch coverage, condition/decision coverage, etc. But at the most basic, what you are looking at when you do code coverage analysis is whether or not you are exercising all of the code you have written.

Coverage is not the same as profiling. The number of times a given line or subroutine is reached is not a useful performance indicator. A subroutine that gets run 1000 times can still take less time and resources than another one that gets run only 10. So don’t look to Devel::Cover for that, look at any of the profiling tools out there, instead.

But back to the point: coverage and Devel::Cover. This module is still alpha-level software, but it’s a very functional, useful alpha. It produces data on coverage of subroutines, statements, branches and conditions. (It also covers POD, but I have that disabled when I run coverage tests, because I have a separate suite for running Test::Pod::Coverage, and Devel::Cover doesn’t give you the facilities for tuning the pod coverage parameters.) It produces quite readable output in HTML format, that is easy cross-reference and follow as you learn just how inadequate your tests truly are. Well, that’s the way it worked for me, at least.

In the 10 days or so since I started using Devel::Cover, I’ve written nearly 275 new tests for RPC::XML— an increase of over 44% from the tests that are in the 0.74 release. And I’m not done… I’ve only boosted the coverage significantly on 3 or 4 of the modules in the package. Bugs? Oh, have I found bugs! I’ve focused mainly on the parsers (I have parsers based on both XML::Parser and XML::LibXML) thus far, and the current code in my Github repo is a lot more hardened than what I’ve had out there before.

My biggest complaint, aside from the expected alpha-level-software glitches, is that it doesn’t play well with AutoLoader/AutoSplit. But that’s a topic for a different blog post.

Devel::Cover… if you write modules and release them on CPAN, you should definitely give this a look.

2 Responses to “Perl Module Monday: Devel::Cover”

  1. mirodNo Gravatar Says:

    I certainly would NOT call Devel::Cover alpha-level-software: it has been around for a long time, over 10 years actually. I have been using it for more 7 or 8 years. More tellingly, it was used in the phalanx project, you may remember it, around 2002, to display test coverage of the 100 most important modules (including CGI, which IIRC uses AutoLoader, so there may be a way to get coverage for your code). So it has been widely used for a long time and on a wide variety of test cases.

    The fact that it doesn’t play nice with Auto(Loader|Split) is either a bug (which I see you submitted) or a limit of the technology used. It doesn’t make the module alpha.


  2. rjrayNo Gravatar Says:

    I only call it alpha because the module’s documentation refers to itself as alpha, in the second paragraph under “DESCRIPTION”. I agree completely that it is a very strong, very functional tool. This is why I recommend it so strongly!

    (CGI does use auto-loading, but it does not use splitting. Rather, it dynamically creates subroutines on demand using symbol table manipulation and closures.)


Leave a Reply