| Subcribe via RSS

Perl Module Monday: Devel::Cover

July 4th, 2011 | 2 Comments | Posted in CPAN, Perl

(I’m trying to get back in the habit of writing again, really.)

In my previous post, I mentioned Devel::Cover. This module has been quite a tool to add to my toolbox. Before I started using it, I thought my test suites were pretty good, overall. Now I have a much clearer picture of what I am and am not actually testing.

First, a bit on what coverage analysis is, and what it isn’t.

At the simplest level, coverage analysis is counting how often you reach each line of code in your software. If you don’t hit a particular line at all, then that line hasn’t been tested. Of course, there’s more to it than that— there’s branch coverage, condition/decision coverage, etc. But at the most basic, what you are looking at when you do code coverage analysis is whether or not you are exercising all of the code you have written.

Coverage is not the same as profiling. The number of times a given line or subroutine is reached is not a useful performance indicator. A subroutine that gets run 1000 times can still take less time and resources than another one that gets run only 10. So don’t look to Devel::Cover for that, look at any of the profiling tools out there, instead.

But back to the point: coverage and Devel::Cover. This module is still alpha-level software, but it’s a very functional, useful alpha. It produces data on coverage of subroutines, statements, branches and conditions. (It also covers POD, but I have that disabled when I run coverage tests, because I have a separate suite for running Test::Pod::Coverage, and Devel::Cover doesn’t give you the facilities for tuning the pod coverage parameters.) It produces quite readable output in HTML format, that is easy cross-reference and follow as you learn just how inadequate your tests truly are. Well, that’s the way it worked for me, at least.

In the 10 days or so since I started using Devel::Cover, I’ve written nearly 275 new tests for RPC::XML— an increase of over 44% from the tests that are in the 0.74 release. And I’m not done… I’ve only boosted the coverage significantly on 3 or 4 of the modules in the package. Bugs? Oh, have I found bugs! I’ve focused mainly on the parsers (I have parsers based on both XML::Parser and XML::LibXML) thus far, and the current code in my Github repo is a lot more hardened than what I’ve had out there before.

My biggest complaint, aside from the expected alpha-level-software glitches, is that it doesn’t play well with AutoLoader/AutoSplit. But that’s a topic for a different blog post.

Devel::Cover… if you write modules and release them on CPAN, you should definitely give this a look.

Tags: , ,

I (Apparently) Suck at Writing Tests

July 3rd, 2011 | No Comments | Posted in CPAN, Perl

(This probably looks familiar. Somehow, my WordPress install got rolled-back a few days. I don’t know how or why. I only know that this post disappeared, as did changes to a plug-in and changes to my “On Javascript and Flash” page. So apologies for seeing this post twice. I’m restoring it from Google’s cache. Clearly I need to spend what remains of the long holiday weekend setting up automated regular backups of my WP installation, something I should have already done were I not so lazy.)

As I go in to what is (for me) a 4-day weekend*, my plans are largely based around writing tests. Lots of tests, most likely. Because, it seems, I’m not very good at it.

That’s a pretty harsh statement for me to make, especially considering that I work in a QA organization. (Though, to be fair, I don’t actually write tests here— I work on the framework that my organization uses to develop their automated test suites.) And (to be fair) I’m probably being overly hard on myself. But I recently started playing around with Devel::Cover, and I was shocked to find out how much of my code is not covered by my tests.

The gist I show above is the sequence of commands that I use to generate coverage data. I adapted this from someone else’s blog post about Devel::Cover. It’s simple, but the part that had kept me from using this before now was not knowing how to apply it to the test suite as a whole, which is what setting HARNESS_PERL_SWITCHES does. (Turns out this is also covered in the man page for Devel::Cover, but for some reason I hadn’t seen that…)

The results have not been pretty.

My most-used module, Image::Size? Overall coverage is 57%, with statement coverage at 67.7%. My RPC::XML module is currently at 78.1% overall (84.1% statement) in my development copy, but that’s after many hours of work over the last week or so. It started out in the low 60s. And I haven’t even applied this to all of my modules.

It seems that my weakest area is in negative testing— I’m just not covering all of my error cases. In one module from RPC::XML, the statement coverage was barely over 50% because the module itself (RPC::XML::Parser) is meant as a common base-class with some code in new() to allow for backwards-compatibility with older versions of RPC::XML. I define the other methods that child classes should override, but I never tested the error-catching code that ensures that the methods are overridden. An hour and a new test-suite later, and this is one of only two modules that has 100% statement coverage (but not 100% overall… it’s only 66.7% covered on conditionals).

Of course, I’m being overly hard on myself. And I’m not really that serious about considering myself so poor at test development. What this (ongoing) exercise has shown me, is how much more I have to learn about it. I do try to practice test-driven development where I can, but I’m not sure that has yielded good coverage in all cases. I tend to practice defensive coding, trying to cover all the potential error cases where I can. But that is no guarantee that I have written tests for all of the potential error cases. Indeed, so far it’s pretty clear that I haven’t.

So if you haven’t looked in to Devel::Cover yet, give it a go. It’s been a great help to me!

* My employer is celebrating our ranking on Fortune’s Great Places to Work list by giving us an extra day off for the holiday weekend. There’s a reason why I like this place!

Tags: , ,

Perl Module Release: RPC-XML 0.74

January 23rd, 2011 | No Comments | Posted in CPAN, Perl, Software, XML
Version: 0.74

Released: Sunday January 23, 2011, 12:50:00 PM -0800


  • t/90_rt54183_sigpipe.t

RT #56800: Make this suite skip all tests on Windows platforms.

  • lib/Apache/RPC/Server.pm

Clean up some run-time “use of undefined value” messages.

  • lib/RPC/XML/Parser/XMLLibXML.pm
  • lib/RPC/XML/Parser/XMLParser.pm
  • t/90_rt58323_push_parser.t (added)

RT #58323: Started as making the parser interfaces correctly report errors when passed null-length strings or “0″ values. Turned out that the error return interface from XMLLibXML.pm was not consistent with the rest of the system, so fixed that as well.

  • lib/RPC/XML/Server.pm
  • t/40_server.t

RT #58240: Applied a patch from Martijn van de Streek that adds access to the HTTP::Request object to called method code.

  • lib/RPC/XML.pm
  • lib/RPC/XML/Parser/XMLLibXML.pm
  • lib/RPC/XML/Parser/XMLParser.pm
  • t/90_rt58065_allow_nil.t (added)

RT #58065: Allow the parsing of <nil /> tags when they are encountered, even if $RPC::XML::ALLOW_NIL is not set. Only limit the generation of these tags.

  • lib/RPC/XML/Server.pm
  • t/41_server_hang.t

This test sporadically fails, so enhance the error message for more info. Also alter the test slightly, hoping it fixes the random failures.

  • etc/make_method

Applied perlcritic to the make_method tool.

  • lib/XML/RPC.pm
  • t/10_data.t
  • t/20_xml_parser.t
  • t/21_xml_libxml.t

RT #62916: Previous adjustments to the dateTime.iso8601 stringification caused it to no longer fit the XML-RPC spec. Fixed.

  • lib/RPC/XML.pm
  • lib/RPC/XML/Client.pm
  • lib/RPC/XML/Parser/XMLParser.pm
  • lib/RPC/XML/ParserFactory.pm
  • lib/RPC/XML/Server.pm

Used warnings::unused to find unused variables not found by Perl::Critic.

  • t/10_data.t

Realized I had no boundary-tests for ints in smart_encode(). This revealed some problems with i8 values on my 32-bit system. Don’t want to introduce dependency on BigInt right now, so marked those tests “TODO”.

Tags: , , ,

Perl Module Release: Image-Size 3.230

August 29th, 2010 | No Comments | Posted in CPAN, Perl, Software
Version: 3.230

Released: Sunday August 29, 2010, 04:00:00 PM -0700


  • imgsize
  • lib/Image/Size.pm

perlcritic clean-ups from new rules.

  • lib/Image/Size.pm
  • t/Test_emf_small.emf (added)
  • t/all.t

RT #59995: Added support for Windows Enhanced Metafile Format (EMF).

  • t/00_load.t (deleted)
  • t/01_pod.t (deleted)
  • t/02_pod_coverage.t (deleted)
  • t/03_meta.t (deleted)
  • t/04_minimumversion.t (deleted)
  • t/05_critic.t (deleted)
  • xt/00_load.t (added)
  • xt/01_pod.t (added)
  • xt/02_pod_coverage.t (added)
  • xt/03_meta.t (added)
  • xt/04_minimumversion.t (added)
  • xt/05_critic.t (added)

Move the author/distro-sanity tests to an “xt” directory.

Tags: , , ,

Perl Module Release: Env-Export 0.21

April 25th, 2010 | No Comments | Posted in CPAN, Perl, Software
Version: 0.21

Released: Sunday April 25, 2010, 10:10:00 PM -0700


  • lib/Env/Export.pm

Code clean-up inspired by perlcritic.

Tags: , , ,

Perl Module Release: Image-Size 3.221

April 25th, 2010 | No Comments | Posted in CPAN, Perl, Software
Version: 3.221
Released: Sunday April 25, 2010, 02:10:00 PM -0700


  • imgsize
  • lib/Image/Size.pm

Large-scale code and documentation clean-up based on perlcritic
and visual review.

Tags: , , ,

Perl Module Release: RPC-XML 0.73

March 16th, 2010 | No Comments | Posted in CPAN, Perl, Software, XML

Version: 0.73

Released: Tuesday March 16, 2010, 10:45:00 PM -0700


  • t/28_parser_bugs_50013.t (deleted)
  • t/90_rt50013_parser_bugs.t (added)

Rename of t/28_parser_bugs_50013.t to fit more universal scheme for
test suites that directly address specific RT bugs.

  • lib/RPC/XML/Server.pm
  • t/90_rt54183_sigpipe.t (added)

RT #54183: Provide handling of SIGPIPE when sending the response to the client,
in case they’ve terminated the connection.


Forgot to add the new test suite to MANIFEST.

  • lib/RPC/XML/Server.pm

Forgot to update the module version number.

  • lib/RPC/XML.pm

Fix typo in reftype() call.

  • lib/RPC/XML.pm
  • t/90_rt54494_blessed_refs.t (added)

RT #54494: Fix handling of blessed references in smart_encode().

  • lib/Apache/RPC/Server.pm
  • lib/Apache/RPC/Status.pm
  • lib/RPC/XML.pm
  • lib/RPC/XML/Client.pm
  • lib/RPC/XML/Function.pm
  • lib/RPC/XML/Method.pm
  • lib/RPC/XML/Parser.pm
  • lib/RPC/XML/Parser/XMLLibXML.pm
  • lib/RPC/XML/Parser/XMLParser.pm
  • lib/RPC/XML/ParserFactory.pm
  • lib/RPC/XML/Procedure.pm
  • lib/RPC/XML/Server.pm

Large-scale code clean-up driven by Perl::Critic. All critic flags
down to severity 1 now removed.


Forgot to add t/90_rt54494_blessed_refs.t when it was created.

Tags: , , ,

Perl Module Monday: Data::Section

December 28th, 2009 | No Comments | Posted in CPAN, Perl

For this week, I’m turning the first of several modules I was introduced to by reading RJBS’ Advent Calendar. This module scratches a particular itch that I’ve had on several occasions, including fairly recently.

Data::Section, in simple terms, turns your __DATA__ section into a hash-table of strings keyed by indexes you choose. It gives you a clean, programmatic way of having multiple “mini-data” sections within the main one, in a fashion similar to Inline::Files (but without the niggling caveat that it could overwrite your source files). The author notes that it was originally envisioned as a way for modules to store their own (multiple) templates, which is a task I’ve had to deal with before. But it would be useful anywhere you want to store large chunks of text data, without resorting to here-documents. It also allows for inheritance of data from a module’s super-classes that also use Data::Section, allowing a class to re-use existing data and only “declare” the extra sections it needs, or those sections that need to be different. You can learn more at the advent entry for this module.

I already have a small CGI app that will be getting the Data::Section treatment, and I hope it can be useful to you as well.

Tags: , ,

No PMM Post This Week

December 21st, 2009 | No Comments | Posted in Perl

There will be no Perl Module Monday this week, as I’m on vacation for the holidays and haven’t had the time to research and select a module this week…

Tags: ,

Perl Module Monday: Linux::Inotify2

December 14th, 2009 | 1 Comment | Posted in CPAN, Perl

Here’s another short one this week…

I first learned about Linux::Inotify2 from a posting on acme’s blog. It’s a neat interface to the Linux Inotify system, which itself is basically a way of being informed whenever things happen in the file-system.

I was able to make use of this almost immediately– I took the sample script from acme’s post and made a few adjustments, then used it to look at the libraries that Image::Size loads during start-up. Turns out that there were a couple of them that didn’t need to be loaded until/unless needed, and once I saw this I was able to move them around and speed up the compile-time phase of the module.

This module has some great potential for monitoring, measuring, etc. I’m toying with a graphical version of acme’s script that I can have running during test suite runs, to get a better idea of other places I can improve things.

Check out the module, and check out acme’s original post on the subject, too!

Tags: , ,