…or, “How To Generate Well Over 100 Failure Reports With Just One Release.”
Ever wonder whether CPAN Testers was really gathering data for you? Ever wonder whether you’re getting the daily summary emails you signed up for? Here’s how you can tell, in just a few (moderately) easy steps:
- Start with a fairly-robust CPAN distribution. Let’s call it “RPC-XML“, just for argument’s sake.
- Have it include a test suite like this one, that test-loads all the classes.
- Remove one of the classes, let’s say, “RPC/XML/Method.pm“.
- This is the important part, don’t change the test suite.
- Dutifully run your tests, note 100% pass rate due to the fact that you have a perfectly usable RPC/XML/Method.pm from an existing installation somewhere under /usr/lib/perl or /usr/share/perl.
- Release, and wait. The reports will come rolling in within just a few short days.
- Now you know that the emails really do get sent reliably!
So, technically this is an issue with the use_ok() predicate in Test::More. And I’ve opened an issue on this, in GitHub. But this may very well be a case of, “Doctor, it hurts when I do this.” And if Schwern decides that it isn’t a bug, I’ll have a hard time arguing with him. But hopefully he’ll agree with my suggestion, and maybe someone, someday, will be spared falling into the same hole I’ve very recently fallen into.