| Subcribe via RSS

How To Tell That CPAN Testers Is Doing Its Job

August 16th, 2011 Posted in CPAN, Perl

…or, “How To Generate Well Over 100 Failure Reports With Just One Release.”

Ever wonder whether CPAN Testers was really gathering data for you? Ever wonder whether you’re getting the daily summary emails you signed up for? Here’s how you can tell, in just a few (moderately) easy steps:

  1. Start with a fairly-robust CPAN distribution. Let’s call it “RPC-XML“, just for argument’s sake.
  2. Have it include a test suite like this one, that test-loads all the classes.
  3. Remove one of the classes, let’s say, “RPC/XML/Method.pm“.
  4. This is the important part, don’t change the test suite.
  5. Dutifully run your tests, note 100% pass rate due to the fact that you have a perfectly usable RPC/XML/Method.pm from an existing installation somewhere under /usr/lib/perl or /usr/share/perl.
  6. Release, and wait. The reports will come rolling in within just a few short days.
  7. Now you know that the emails really do get sent reliably!

So, technically this is an issue with the use_ok() predicate in Test::More. And I’ve opened an issue on this, in GitHub. But this may very well be a case of, “Doctor, it hurts when I do this.” And if Schwern decides that it isn’t a bug, I’ll have a hard time arguing with him. But hopefully he’ll agree with my suggestion, and maybe someone, someday, will be spared falling into the same hole I’ve very recently fallen into.

Tags: ,

One Response to “How To Tell That CPAN Testers Is Doing Its Job”

  1. JoelNo Gravatar Says:

    I feel into the trap of running `make dist` without a MANIFEST file. While I would think that it should die and give a big warning, it instead produces and appropriately named .tar.gz file. Sadly its contents are only an error message.

    I know I have released at least one of these stunted dists, and I’m sure others have too. Grrrr.

    And yes now I know why `make disttest` exists, but when `make test` succeeds and `make dist` succeeds, I say “release”! Oh well.


Leave a Reply