| Subcribe via RSS

Perl Module Monday: Carp::Always

August 22nd, 2011 | 1 Comment | Posted in CPAN, Perl

(With special bonus-module this week!)

Ever run into an error or warning in someone else’s module, and wished you could get more information about it without having to wade through their code? It’s probably almost certain that you have, at least once. At least once, you’ve seen a warning or had a program die, and wished that you could see a stack trace without having to go in and edit the module. After all, the module is probably installed into a system location, and even if you don’t need root/sudo to edit it, you then have the hassle of going back and undoing the changes (or re-doing them, if/when you update the module in question) after your debugging has been done.

Enter Carp::Always. Carp::Always plays around with $SIG{__WARN__} and $SIG{__DIE__}, and gives you stack-traces on every die and warn that come from code. I was introduced to this module by a commenter on my “Chasing an Elusive Warning” post last week. Alas, it didn’t help in that case, as the warning was coming from within Perl itself. But I found the module to be a nice concept, and it’s easy to use on a only-when-you-need-it basis. Have a script that is generating warnings you want more information about?

perl -MCarp::Always script.pl

Carp::Always quietly slips in and make the necessary alteration to the die/warn handlers, and your script runs and does its thing. And when the warnings (or termination) come, you should have your stack trace.

The author does point out that this module may not play well with other modules that alter $SIG{__WARN__} and $SIG{__DIE__}, so there is that to be aware of. But that aside, hopefully this can be of great aid to your next debugging session!

(And as a bonus, here’s another module that does the essentially the same thing: Devel::SimpleTrace. I actually know much less about this one, except that it lists Data::Dumper as a dependency, which leads me to believe that it tries to do some pretty-printing with the data in stack-frames. But unlike Carp::Always, I haven’t used it. Still, if you like the concept, it’s worth checking out both modules and seeing which one you like more.)

Tags: , ,

Perl Module Release: RPC-XML 0.76

August 21st, 2011 | No Comments | Posted in CPAN, Perl, Software
Version: 0.76

Released: Saturday August 20, 2011, 06:30:00 PM -0700

Changes:

  • etc/make_method
  • lib/RPC/XML/Server.pm

RT #70258: Fixed typos in docs pointed out by Debian team.

  • lib/Apache/RPC/Server.pm

Better version of the fix for infinite loops. This is the patch originally suggested by Eric Cholet, who found the bug.

  • t/00_load.t

RT #70280: This test was still testing RPC/XML/Method.pm. Rewrote to remove that but include the (forgotten) XMLLibXML.pm module. That test has to be conditional on the presence of XML::LibXML.

  • Makefile.PL
  • t/51_client_with_host_header.t

Clean up test suite to work with older Test::More. Also specify a minimum Test::More that supports subtest(). This is also a part of RT #70280.

  • t/11_base64_fh.t
  • t/20_xml_parser.t
  • t/21_xml_libxml.t
  • t/40_server.t

These tests had failures when run as root. Permissions-based negative tests were incorrectly passing.

  • t/10_data.t

Moved the 64-bit “TODO” tests to a SKIP block. Non-64-bit systems will skip, rather than fail, these tests.

  • lib/RPC/XML/Server.pm

RT #65616: Fix for slow methods killing servers. Applied and modified patch from person who opened the ticket.

  • MANIFEST
  • lib/RPC/XML.pm
  • t/10_data.t
  • t/14_datetime_iso8601.t (added)

RT #55628: Improve flexibility of date parsing. This adds the ability to pass any ISO 8601 string to the RPC::XML::datetime_iso8601 constructor.

Tags: , , ,

Elusive Warning Found and Quashed!

August 17th, 2011 | No Comments | Posted in Perl

After several hours of puzzling through the XS code for XML::LibXML (and WOW is my XS-fu rusty!), I gave up and filed an RT ticket on it. I gave it all the info I had, including the specific test-suite that was generating the error. One thing I didn’t provide was the location within the test suite, because that seemed to change from platform to platform.

Well, great kudos to Shlomi Fish, who has already found and quashed the bug! It was definitely more complex, and buried more deeply, than I thought. It was related to my use of the XML::LibXML::CallBack class, usage of which was new in this release (there’s an issue with the slightly out-of-date libxml2 package that MacOS X Snow Leopard has, that interfered with a simpler solution, which forced me to use the callback class). Anyway, thanks to everyone who responded! You got me deep-enough into the code that I was able to craft a better bug-report than I might have otherwise. And hopefully, Shlomi is as happy to have the bug be found and dealt with as I am!

Tags:

Chasing a Very Elusive Warning

August 16th, 2011 | 7 Comments | Posted in Perl

I am getting a warning in one of my test suites for RPC-XML. I only get this warning when I run “make test“. I do not get the warning if I run the test directly with “perl -Mblib t/…“, or if I run it via prove. The test this occurs in is t/21_xml_libxml.t, which tests the XML::LibXML-based parser, and the warning I get is:

Use of uninitialized value in subroutine entry at .../site_perl/5.14.1/darwin-2level/XML/LibXML.pm line 843.

Because this is occurring within Perl itself, I cannot seem to get it to give me a stack-trace in place of the warning. My attempts to check for undefined values have not found any, yet I get the warning. Running in the debugger does not generate the warning.

Because my tests pass, I have confidence that this parser is fine, despite this only-in-test-harness warning. But the OCD side of me is going crazy over having this still show up when I run my tests locally (and yes, it shows up on all my platforms: Darwin, 64-bit Linux and 32-bit Linux). So, if anyone reading this can give me a helpful hint as to how I can get more/better diagnostic information on this, I would be truly grateful.

Tags: ,

How To Tell That CPAN Testers Is Doing Its Job

August 16th, 2011 | 1 Comment | Posted in CPAN, Perl

…or, “How To Generate Well Over 100 Failure Reports With Just One Release.”

Ever wonder whether CPAN Testers was really gathering data for you? Ever wonder whether you’re getting the daily summary emails you signed up for? Here’s how you can tell, in just a few (moderately) easy steps:

  1. Start with a fairly-robust CPAN distribution. Let’s call it “RPC-XML“, just for argument’s sake.
  2. Have it include a test suite like this one, that test-loads all the classes.
  3. Remove one of the classes, let’s say, “RPC/XML/Method.pm“.
  4. This is the important part, don’t change the test suite.
  5. Dutifully run your tests, note 100% pass rate due to the fact that you have a perfectly usable RPC/XML/Method.pm from an existing installation somewhere under /usr/lib/perl or /usr/share/perl.
  6. Release, and wait. The reports will come rolling in within just a few short days.
  7. Now you know that the emails really do get sent reliably!

So, technically this is an issue with the use_ok() predicate in Test::More. And I’ve opened an issue on this, in GitHub. But this may very well be a case of, “Doctor, it hurts when I do this.” And if Schwern decides that it isn’t a bug, I’ll have a hard time arguing with him. But hopefully he’ll agree with my suggestion, and maybe someone, someday, will be spared falling into the same hole I’ve very recently fallen into.

Tags: ,

No PMM Post This Week

August 15th, 2011 | 1 Comment | Posted in Perl

I haven’t had time to research a new module for Perl Module Monday this week, sorry. I have a few modules I’d like to write about, but just haven’t had the time to look at them in-depth. I am looking at doing a meta-PMM post on export/import modules, given that my last few PMM’s have generated a reasonable amount of feedback. But I’m not ready to do that one yet, either, and I’d prefer to not do something as a half-measure.

Tags: ,

Perl Module Release: RPC-XML 0.75

August 14th, 2011 | No Comments | Posted in CPAN, Perl, Software, XML

MetaCPAN.org: https://metacpan.org/release/RJRAY/RPC-XML-0.75

Version: 0.75

Released: Saturday August 13, 2011, 05:30:00 PM -0700

Changes:

  • MANIFEST

Somehow, t/13_no_deep_recursion.t never got added to MANIFEST.

  • lib/RPC/XML/Parser/XMLLibXML.pm

RT #65154: Fixed a cut/paste error in an error message.

  • lib/RPC/XML/Client.pm
  • t/51_client_with_host_header.t (added)

RT #68792: Merge pull request #2 from dragon3/master (https://github.com/dragon3). Allow setting of “Host” header, and test suite for it.

  • MANIFEST
  • t/51_client_with_host_header.t

Added new test suite to MANIFEST, fixed spelling. Also added “plan tests” line to the test suite.

  • lib/RPC/XML/Parser/XMLLibXML.pm
  • t/20_xml_parser.t
  • t/21_xml_libxml.t
  • t/41_server_hang.t

Merge pull request #3 from yannk/master (https://github.com/yannk). Expat parser subclass is protected against ext ent attack, libxml isn’t.

  • t/41_server_hang.t

Undo a change to this suite from yannk’s pull.

  • etc/make_method
  • lib/Apache/RPC/Server.pm
  • lib/Apache/RPC/Status.pm
  • lib/RPC/XML.pm
  • lib/RPC/XML/Client.pm
  • lib/RPC/XML/Function.pm
  • lib/RPC/XML/Method.pm
  • lib/RPC/XML/Parser.pm
  • lib/RPC/XML/Parser/XMLLibXML.pm
  • lib/RPC/XML/Parser/XMLParser.pm
  • lib/RPC/XML/ParserFactory.pm
  • lib/RPC/XML/Procedure.pm
  • lib/RPC/XML/Server.pm

More perlcritic-driven clean-up. This is mostly POD sections, but also includes heavy re-working of etc/make_method and parts of lib/RPC/XML.pm.

  • lib/RPC/XML/Parser/XMLLibXML.pm
  • t/21_xml_libxml.t

Fixed external entity handling on MacOS. Also made small change to the test suite to be cleaner.

  • lib/RPC/XML/Parser/XMLLibXML.pm
  • lib/RPC/XML/Parser/XMLParser.pm

Took out warnings on external entities blocking. Now it blocks silently. Also cleaned up some docs.

  • t/15_serialize.t

Additions to increase code coverage in XML.pm.

  • lib/RPC/XML.pm

Turns out this wasn’t exporting RPC_I8.

  • lib/Apache/RPC/Server.pm
  • lib/Apache/RPC/Status.pm
  • lib/RPC/XML.pm
  • lib/RPC/XML/Client.pm
  • lib/RPC/XML/Function.pm
  • lib/RPC/XML/Method.pm
  • lib/RPC/XML/Parser.pm
  • lib/RPC/XML/Parser/XMLLibXML.pm
  • lib/RPC/XML/Parser/XMLParser.pm
  • lib/RPC/XML/ParserFactory.pm
  • lib/RPC/XML/Procedure.pm
  • lib/RPC/XML/Server.pm
  • xt/02_pod_coverage.t

Made 5.8.8 the new minimum-required perl. Also dropped the utf8_downgrade hack, which affected an xt test.

  • lib/RPC/XML/Client.pm

Improved arguments-checking in send_request.

  • lib/RPC/XML/Client.pm
  • lib/RPC/XML/Parser/XMLLibXML.pm
  • lib/RPC/XML/Parser/XMLParser.pm
  • lib/RPC/XML/Server.pm

Fixed error-cases in usage of File::Temp->new(). File::Temp::new croaks on errors, doesn’t return undef like I thought.

  • MANIFEST
  • lib/RPC/XML/Function.pm (deleted)
  • lib/RPC/XML/Method.pm (deleted)
  • lib/RPC/XML/Procedure.pm

Roll Method.pm and Function.pm into Procedure.pm. Remove Method.pm and Function.pm from distro.

  • lib/RPC/XML/Parser/XMLLibXML.pm

Fixed regexp for methodName validation.

  • t/10_data.t
  • t/11_base64_fh.t
  • t/12_nil.t
  • t/15_serialize.t
  • t/20_xml_parser.t
  • t/21_xml_libxml.t
  • t/25_parser_negative.t (added)
  • t/29_parserfactory.t
  • t/30_method.t
  • t/40_server.t
  • t/40_server_xmllibxml.t
  • t/50_client.t
  • t/BadParserClass.pm (added)
  • t/meth_good_1.xpl
  • t/namespace3.xpl
  • t/svsm_text.b64 (added)
  • t/util.pl

First round of Devel::Cover-inspired improvements. These are the changes to the test suites to increase coverage of the code.

  • lib/RPC/XML.pm
  • lib/RPC/XML/Procedure.pm
  • lib/RPC/XML/Server.pm

Fixes and such from Devel::Cover analysis.

  • lib/RPC/XML/Procedure.pm
  • lib/RPC/XML/Server.pm
  • t/30_method.t
  • t/meth_good_1.xpl
  • t/meth_good_2.xpl (added)
  • t/meth_good_3.xpl (added)

Fixes for file-based method loading/reloading. New tests in the suite, and re-working of the ugliest hacky part of this package.

  • lib/RPC/XML/Procedure.pm
  • lib/RPC/XML/Server.pm
  • t/30_method.t
  • t/meth_good_3.xpl

RPC::XML::Procedure test-coverage improvement. Also removed some unneeded code.

  • lib/RPC/XML/Procedure.pm
  • lib/RPC/XML/Server.pm
  • t/30_method.t
  • t/40_server.t

Last round of RPC::XML::Procedure test coverage. This is mostly in t/40_server.t, though some bugs were found and addressed in the modules and in t/30_method.t.

  • lib/Apache/RPC/Server.pm
  • lib/Apache/RPC/Status.pm
  • lib/RPC/XML.pm
  • lib/RPC/XML/Client.pm
  • lib/RPC/XML/Parser.pm
  • lib/RPC/XML/Parser/XMLLibXML.pm
  • lib/RPC/XML/Parser/XMLParser.pm
  • lib/RPC/XML/ParserFactory.pm
  • lib/RPC/XML/Procedure.pm
  • lib/RPC/XML/Server.pm

Documentation clean-up and update.

  • lib/Apache/RPC/Server.pm
  • lib/Apache/RPC/Status.pm
  • lib/RPC/XML/Client.pm
  • lib/RPC/XML/Parser/XMLLibXML.pm
  • lib/RPC/XML/Parser/XMLParser.pm
  • lib/RPC/XML/Procedure.pm
  • lib/RPC/XML/Server.pm

Changes from new Perl::Critic::Bangs policies.

  • xt/01_pod.t
  • xt/02_pod_coverage.t
  • xt/03_meta.t
  • xt/04_minimumversion.t
  • xt/05_critic.t

Adjustments to reflect moving from t to xt. Also made changes to xt/02_pod_coverage.t to reflect changes to modules.

  • lib/RPC/XML/Client.pm

Removed some error checks that can never fail.

  • lib/RPC/XML/Server.pm
  • t/40_server.t

Code-coverage-driven changes and added tests.

  • etc/make_method

Fixes from new Perl::Critic::Bangs policies.

  • lib/RPC/XML/Server.pm

Removed usage of AutoLoader completely.

  • lib/RPC/XML/Server.pm
  • t/40_server.t
  • xt/02_pod_coverage.t

Removed some dead code and better did the aliases. This required a change in t/40_server.t for a private sub that no longer exists. Also updated xt/02_pod_coverage.t for private subs that have no pod.

  • lib/Apache/RPC/Server.pm

RT #67694: Fix a potential infinite-loop condition.

Tags: , , ,

Perl Module Monday: AutoRole

August 9th, 2011 | 4 Comments | Posted in CPAN, Perl

This week’s choice is a sort of follow-up to last week’s post. AutoRole is a module that lets you do run-time or compile-time loading of modules, along with potentially renaming what you import on the fly. It takes these two useful features and wraps them into one package.

I try not to do two such closely-related picks back-to-back, but I had actually noticed AutoRole about a year ago, and taken note of it. It was shortly after I had taken RJBS’s Moose tutorial at OSCON ’10, and I was looking for a way to do roles at my day-job without having the luxury of a full-on Moose install. Alas, as often happens with my ADD-addled brain, I promptly forgot about the module after a few days. Luckily, the module’s author mentioned it in a comment to last week’s post, and it reminded me to take another look at it.

And it looks quite flexible. You can specify one of three methods for loading the module (compile-time or two flavors of run-time loading), and you can both specify the methods/routines to load, and give them alternate names if necessary (or just desired). It’s also quite light-weight and has no dependencies. If I were to have any concerns about it, it would be that it seems to be fairly young code (the latest version at this writing being 0.03) and hasn’t been updated in just over a year. BUT, and this is important, these two facts don’t mean the code isn’t solid and usable. I, too, have had code that reached the point where I felt it was feature-complete and stable, while only having reached version 0.4 or so. So don’t let these two factors prevent you from at least looking over AutoRole and maybe giving it a try.

One thing I did notice that Exporter does, that neither Sub::Exporter or AutoRole seem to do, is export variables themselves. However, this may be considered a feature by some!

So here you go: Sub::Exporter for creating more flexibility in your exports as a module developer, and AutoRole for more flexibility on your imports (and to be fair, I’ll mention that RJBS also has a module called Sub::Import that provides a Sub::Exporter-sort-of-flexibility when importing from modules that don’t use Sub::Exporter). Different approaches to the same sort of problem, depending on the angle from which you are approaching it. Myself, I’ll be looking at both modules for my own use, at home and at work.

Tags: , ,

Perl Module Monday: Sub::Exporter

August 1st, 2011 | 5 Comments | Posted in CPAN, Perl

For this week’s entry, I’m taking a look at a module I first learned about in December  2009 while reading RJBS‘s Advent Calendar for that year: Sub::Exporter. I highly recommend his yearly calendar, as it is a great way to learn about new and interesting modules and features. Of course, it helps to actually use the things you find interesting: while I learned about this module over a year and a half ago, I’d completely forgotten about it until someone mentioned it during one of the talks as OSCON last week. Then I came across it in my notes on modules to consider for this series, and decided I’d best write about it soon, lest I forget it again!

Rather than going into an exhaustive explanation of what this module is, I invite you to take a quick look at the original advent calendar posting. Go ahead, I’ll wait. Done? Great!

Now, if you didn’t just check that out, or if you thought it was too long and just skimmed over it, here’s the short version: Sub::Exporter is a super-version of the core Exporter module. It allows other modules and scripts to import the routines you’ve chosen to export. But unlike Exporter, it gives both you and the user of your module a great range of flexibility in options and configuration of the routines that are exported/imported.

So, it can do everything that Exporter does, but it can also do a whole lot more. To me, the most useful feature of Sub::Exporter is the ability to rename an imported function when you import it. I have, in the past, had to opt to not import a given subroutine in order to avoid name-clashes between different modules. I would have to choose which one “wins”, and then use the full package name to call the other package’s routine. With Sub::Exporter, this is not only a fixable problem, it’s also the simplest of the examples of Sub::Exporter use.

The module is well-documented, coming not only with a basic manual page but also a tutorial page and a cookbook page. If only more modules did this! (I say that, but none of my modules do that, so I have no room to cast aspersions.) It is also fairly lightweight in both its own code and its dependencies (unlike last week’s PMM, which I later learned requires that you have Moose installed, even though it doesn’t use Moose directly).

I won’t necessarily replace all my usage of Exporter with Sub::Exporter— I think that some names are sufficiently unique as to avoid potential clashes— but I will certainly be using it in the future.

Tags: , ,

AutoLoader Considered Harmful?

July 30th, 2011 | 4 Comments | Posted in Perl

Lately, I’ve been doing a lot of work on some of my modules using the fine Devel::Cover package. And for about two years now, I’ve also been using Perl::Critic almost religiously (to the point where I’ve introduced it around $DAY_JOB, and even built them a snazzy web interface for it). But these two share a common failing: both fall down when they encounter a module that uses some form of external-file-based auto-loading.

First, let me explain what I’m referring to, for those who might not be familiar with auto-loading. Perl has this nifty feature called auto-loading, in which a function that doesn’t exist when it is called might have the chance to spring into life dynamically. Perl will look for and invoke a special subroutine called AUTOLOAD in your current namespace, giving it the name of the subroutine that was just requested in the special variable $AUTOLOAD (and passing that call’s args list in @_). You can get the routine by any means that Perl allows: you can create it on the fly, you can on-demand load a module and add it to your @ISA search path, etc. One of the most clever uses I saw early on in Perl 5′s life, was Lincoln Stein’s CGI module. In it, he used auto-loading to define all the HTML shortcut routines from a template, eval‘ing them on the fly as they were each first called.

In short, this amounts to Perl’s version of C’s dynamic loading, deferring the loading and compilation of code (well, in the C world you were just deferring the loading and linking, it was already compiled) until you needed it. Into to this we introduce the core modules AutoLoader and AutoSplit. AutoLoader provides for you an AUTOLOAD method that uses the name of the requested routine and the package it is in to look for a file by that name under a directory hierarchy within your @INC path. To compliment this, AutoSplit is used by the build/install process (be it ExtUtils::MakeMaker or Module::Build, etc.) to look at a module, determine if it uses AutoLoader, and splits out the routines that were defined as dynamically-loading into these per-name files. In fact, I got my break into the world of writing about Perl by writing an article explaining these two modules for the Perl Journal (alas, I cannot find a link to the article online). I also made my first contributions to Perl’s core by fleshing out the docs for these two, as I had been using them at my then-job to make a 20,000-line module more manageable.

But! I’m clearly not hear to sing the praises of AutoLoader, or I would have titled this post something else entirely. I don’t necessarily come here to bury Caesar in lieu of praising him, but introducing AutoLoader (or SelfLoader, for that matter) gums up the works when you use either of Devel::Cover or Perl::Critic. And while Devel::Cover might be able to fix this (I’ve filed a bug, but I don’t know the internals of D::C so I don’t know if it is addressable), Perl::Critic definitely won’t be able to.

See, AutoLoader/AutoSplit (and SelfLoader, though it uses a slightly different model) work by having you put the code you want to delay loading after the __END__ token in your module (__DATA__ in the case of SelfLoader). AutoSplit can then find it and split it out into individual files, and as the compiler stops compiling once it hits that token nothing at that point onward contributes to the start-up compilation phase. But while this successfully hides the code from the compiler, it also very successfully hides the code from Perl::Critic and Devel::Cover as well.

P::C is built on the PPI package by Adam Kennedy, and in terms of a Perl module as a document anything after __END__ is simply not code and not going to be processed as such. So your code that uses AutoLoader is not being fully analyzed by P::C. I had released a version of RPC::XML that I thought was critic-clean, without having thought about this caveat. When I later went and commented-out the “use AutoLoader” and “__END__” lines, I found a whole new set of violations to clear up.

Likewise, I had a similar problem with Devel::Cover. Only slightly worse, because I assumed that they had already dealt with this problem. The code gets loaded just like any other Perl module, so I assumed that when it loaded it then got instrumented. And the code uses “#line” directives to associate the code with the correct line in the original *.pm file, so I thought that they would be able to “translate” the line numbers of coverage statistics back to the originating file. But alas, no— code after __END__ is just as hidden from D::C as it is from P::C.

Which leaves me wondering whether AutoLoader (and __END__-based auto-loading in general) might not have run its course of usefulness. I’m left wondering what sort of hoops the authors of syntax-highlighting for Emacs had to jump through, to determine that content after __END__ was actually code rather than data, and to highlight it as such (vim doesn’t, it leaves it in the same font-face as data). Editors aside, here are two extremely useful development tools for Perl programmers (remember, I’m practically religious about P::C, spreading the gospel to my workplace and anywhere else I can), and both are hampered by the use of auto-loading. Given the leaps in memory and CPU over the last 15 years or so since I wrote that first article, do we really need AutoLoader anymore? I mean, I’m not saying do away with auto-loading itself, it has a truly useful place in Perl. I’m just not so sure whether we need AutoLoader (or SelfLoader) like we used to.