| Subcribe via RSS

Rant: On Website Passwords

February 27th, 2014 | 2 Comments | Posted in Rants, Security

This morning, my email included a now-familiar tune: A site that I occasionally visit may have been penetrated, so the company was informing the registered users that they needed to reset their passwords. Logging in would take you immediately to a page that let you request a link for password reset. Nothing new, and not that big of a deal. Like I said, it’s been happening quite a bit lately.

But then when I entered a new password, the form comes back immediately, with a message in red that is also becoming all-too-familiar: My password contained characters other than letters and digits. That is not allowed. Please choose another password.

Are you freakin’ kidding me? This is 2014, not 1993. Why in the world does your system limit passwords to just the [A-Za-z0-9] character range? What database, what content-management package are you using, that having a “!” or “$” in the password would break the application? This is just ridiculous. And surely you aren’t storing these passwords as plain-text, I hope? I recently read an excellent article on how one should store passwords in databases, using a combination of salting and one-way hashing functions. Unfortunately I’ve lost the link or I’d include it here. But the point is, you aren’t storing the passwords as plaintext, you shouldn’t be comparing them as plaintext, so why should you care what characters are in the passwords? And then, just to put icing on the cake, once I had selected a new (weak) password they emailed it back to me as part of the confirmation message.

They emailed it to me. In the clear. Not encrypted. They are not the only ones who have done that, either.

I use KeyPassX as a password store. I like it because it is cross-platform; I have it on my MacBook, on my Linux desktop machines, and I have apps that are compatible with the key-store that run on my Android phone and my iPad. All of the installations can configure the location of the key-store, so I have them all pointing to the file on my Dropbox account, thus keeping my passwords in sync across all devices. (Yes, Dropbox is not 100% secure, but the encryption on the key-store file is strong so I’m not worried about someone getting it.) When I looked at the list of sites and passwords this morning, as I was updating the entry for this site, I could tell by looking at the passwords which sites do or do not have this silly restriction on password characters. Among the offenders are two telecoms, a health insurance provider and no fewer than three financial institutions. And these are just the ones that I personally use. I would really have expected more from the financial places, because they have so much more on the line.

Look, I write software for a living, even if I don’t specialize in security. Characters are characters… your software doesn’t view “$” any differently than “Z“, unless the software is doing something horribly badly. Unless your stuff is written amazingly poorly, you can do better on the password front. And if it is written that poorly, consider finding a new vendor… you aren’t doing yourselves any favors with what you currently have.

Tags: ,

Dear LazyWeb: Good Key/Value Data Store for Perl?

February 24th, 2014 | 10 Comments | Posted in CPAN, Perl

I’m working on a web-based tool at my day job in which I will be caching a lot of data. Well, not a lot of data in the Facebook sense of “a lot”, but a non-trivial amount nonetheless.

Currently, for the prototype, I’ve been using a Berkeley DB file with DB_File::Lock to lock it for writes. While this is fine for one or two users, to make sure that my AJAX-driven concurrent data requests don’t trample on each other, it won’t even remotely scale. If five people hit the page at once, four of them will have to wait, and at least one will have a really long wait.

My data requirements are actually pretty simple: I’m storing data about directories, using the directory paths as keys. The values I’m storing are large-ish hashes of data frozen with the Storable module. For example, for a given top-level directory /A, with a few dozen directories simply numbered from 01 on up, I will end up with a collection of keys in the DB_File similar to:

/A        (meta-data, including the list of sub-dirs)
/A.tags   (some cross-reference tag data)
/A/01     (dir-specific data)

So, what should I use for this? The Perl process that creates and accesses the data will be running from an Apache/mod_perl environment, so file-permissions will have to work within that constraint. I haven’t looked at many of the cache-related modules on CPAN yet, but I have looked at MongoDB (and at the Perl driver MongoDB). But I feel like it is almost certainly overkill for this application, and I’m not sure how it handles binary data like what I am storing (I’m sure that it does handle it, I’m just not sure how).

Thoughts and/or suggestions?

Tags: ,

Test Suites: To Critic or Not?

February 19th, 2014 | 1 Comment | Posted in CPAN, Perl

Over the holiday weekend, I spent a good amount of time cleaning up my RPC::XML test suites with regards to Perl::Critic. It wasn’t quick work, either: Much of the suite was written when RPC::XML was first under development, which makes a lot of it well over 10 years old. But I had an extra day, and plenty of time to kill. So 25 files and one git commit later, my test are all clean with respect to my chosen Perl::Critic configuration. The exceptions are a couple of Modules::ProhibitExcessMainComplexity complaints, and most of them also chirp about Modules::RequireVersionVar because the policy doesn’t allow for exclusion of files like test suites.

But all through this, I was wondering: Does it really matter if your test suites are as clean as the code itself?

How often does someone else look at your test suites as closely as they look at the modules that those tests support? I’m not setting up a life-lesson here, I’m genuinely curious. I spent some time thinking about it, and I can say that the only times I’ve looked at test suites in other distributions has been when I was trying to figure out how they did something, or how a particular testing module is used. I’ve very rarely looked for examples of how to write my own tests, and I probably should. I’m pretty sure I could learn from some (most?) of them.

See, I look at Perl::Critic as at least partly geared towards the readability of code; using Perl::Critic helps me make my code more consistent in style and structure, which in turn makes it more readable. I introduced it into my day job (along with a web front-end) largely to try and encourage certain consistencies in style, uniformity even, in the name of corralling Perl coders who had no collective style. Yes, there are a lot of other benefits to critic that are there to be had. It has led me directly to bugs, or to code that could potentially fail, on numerous occasions. But for me, myself, style and structure are the order of the day. So, if few people look at your test suites, is there any reason to run them through the critic?

After a day or two of idle background-thought on it, I’ve decided that it is worth it. Even if the only person who looks at your tests is you. Because you do eventually have to look at those tests. And RPC::XML is not even remotely the oldest CPAN module, so there are distributions out there whose tests are much older than my 10+ years. And that’s plenty of time to forget WTF you were thinking when you wrote the test. The least you can do is try to make it readable for the next time you have to look at it.

Tags: , , ,

CPAN Testers Follow-Up: Progress Is Made

February 13th, 2014 | 3 Comments | Posted in CPAN, Perl

Or, How I Learned It (Mostly) Wasn’t My Fault

So, in my previous post I noted how my reward for getting back on the horse had been a slew of failure reports from the CPAN Testers service. This is a follow-up on what I’ve learned these last 6 days.

First, Do Not Panic

You’ve just gotten a barrage of FAIL reports from CPAN Testers. Don’t get yourself agitated over this. Think about it: You’ve just been handed (free of charge) the results of someone else taking the time to test your product for you, probably on a combination of architecture/OS/Perl that you didn’t have available to test yourself. This is a good thing for you. Once upon a time, you wouldn’t have had this until someone was actually using your code in the field. Then the report you got would have been from someone annoyed and/or angry over breakage. Instead, you’ve gotten some number of reports that look like this, which can hopefully help you zoom in on the issue.

I (kind of) panicked. Fortunately, after someone responded to my previous post I was able to calm down and start looking at the content of the reports more rationally.

Second, Figure Out What’s Gone Wrong

Look, you wrote the module in question. This means that you are almost certainly the best-suited to determine what went wrong. (Not always, as I’ll point out a little later, but probably.) Look at the reports carefully, see what tests failed. Look at those tests yourself and see what could have tripped them up. Look for counter-examples, or noteworthy differences. In my case, the majority of my test failures were caused by my server class (RPC::XML::Server) failing to instantiate because (it claimed) the address it was told to bind to was in use. This didn’t make sense, as I had code I was using that actually scanned for a port that wasn’t in use. The port that the server was trying to bind to had, just moments before, been judged available by virtue of the fact that a raw socket connection to it had failed. So what was wrong? Why was it failing? And more interestingly, why was it failing in these cases, but not in the dozens of PASS reports that I had also racked up?

I noticed that all the failing cases had something in common: a comment in the test output stream that a package called Acme::Override::INET was monkey-patching IO::Socket::IP in place of IO::Socket::INET. I had my first big clue.

Third, Prepare an Environment for Testing

If you aren’t using perlbrew, then either you aren’t testing as well as you could be or you’re spending way more effort on your testing than you need to be. I first started using it because the Perl that comes with Mac OS is older, and I wanted to be able to install and update CPAN modules more easily. Then I started keeping a half-dozen or so different additional installations of Perl around, at various versions with and without threading. If you’re developing modules, it’s an invaluable tool for testing your modules across different versions. It also meant that I could install Acme::Override::INET and IO::Socket::IP in a Perl other than my “main” one, so that I didn’t upset my primary work/development environment.

So I did just that; I picked a version of Perl that I had that was identical to one that had failed in one of the reports, and installed those two modules.

Fourth, Learn What You Can

Suddenly, I too was getting the exact same failures in the exact same places. This was great! I started throwing in some debugging statements, and verified that in each case a previous server allocation had succeeded but the next allocation (which was re-using the same port) would fail. For some reason, IO::Socket::IP seemed to be causing sockets (or at least the port allocations) to hold on beyond the life of the socket. I even managed to show that the error occurred over the process boundary, when I ran a test twice in rapid succession and the second run failed in the first server allocation! When I removed Acme::Override::INET and IO::Socket::IP, it immediately started working again. Same tests, same code, same Perl executable… the only difference was IO::Socket::IP.

(I should note here that un-doing the installation of Acme::Override::INET is tricky. It installs a “fake” IO/Socket/INET.pm file, but it uses Module::Build to do so. Unless you have M::B configured to use the same installation locations as ExtUtils::MakeMaker, M::B will install in a directory that is earlier in your search path than the directory that E::MM uses. So re-installing the IO module will not overwrite the IO::Socket::INET fake that Acme::Override::INET installed. I had to delete all four files that were installed.)

Fifth, Do Something with Your New Knowledge

So I opened up an RT ticket for IO::Socket::IP, and explained my situation at length in it. The module’s author replied and pointed out that the way I was selecting a port was much more work than I needed to do in that case. See, I’m not actually very adept at network programming. That’s actually an understatement… if it weren’t for HTTP in general, and HTTP-related Perl modules like LWP in particular, I would probably not have a CPAN module to my name that is so network-centric. Anyway, I at least felt at this point that there wasn’t anything wrong with RPC::XML::Server. But based on the suggestions from the IO::Socket::IP author, I changed most of my server-oriented tests to let the kernel allocate a random port instead of me selecting one. Now the tests all pass, even with the newer module and the monkey-patch installed. Mind you, there is still an issue with IO::Socket::IP, but I can’t help there. I can only hope that the information I’ve provided helps him out.

Sixth, and This Is Important, Be Content With It

It’s frustrating to know that there will still be some FAILs coming in, but I can’t help that. I can be happy with the fact that my tests are fixed, that there wasn’t actually anything wrong with my networking code (because I’d be close to lost if I had to go into a deep dive to troubleshoot that part). There are still things to fix and add to the RPC::XML package, but my server tests are all now a bit more robust than they were this time last week. I’ve found and reported an issue that will hopefully help another CPAN author. And, importantly for me, I’ve stayed engaged in my code for this last week. That’s been hard for me to do for a long time now, and it feels really good to have done it. Even though it’s only been about 5-6 hours of effort spread across that many days, I’m happy about it, and that will hopefully mean that the next 5-6 hours will come more easily.

Tags: , ,

CPAN Testers and RPC-XML: Well, Crap

February 7th, 2014 | 1 Comment | Posted in CPAN, Perl

So I’m starting to break my drought on personal-project coding, and yesterday I cut a new release of RPC-XML, the first one since September of 2012. I’m pretty happy about this; besides the fact that I was able to clear out my pull-request queue on GitHub, this means that I’m (slowly) getting my “old self” back. I’m (slowly) starting to work on my hobby-programming, and no longer seem to be limited to just doing my work for my day job.

Then this morning I checked my RSS feed for CPAN Testers Reports. NINE failures, all of them tests that passed just fine on my dev platform and on my Linux server. Tested with multiple Perl versions, etc. Granted, I don’t test other versions/platforms exhaustively, not compared to some people at least. But the things that seem to be failing are tests that have always passed before. Like, suddenly some of the tests that actually create server objects are failing to bind to their given address, saying that it’s already in use. Except that I programmatically select a port by first establishing that it is not in use. I suppose that theoretically it is possible that some other process stepped in between the time I do the port-detection and the time I actually allocate the server object. But like I said, these are tests that were passing just fine before. There are some others, as well, that seem to be consistently problematic for people who are not me. But I don’t get a lot of information from the CPAN Testers reports, just the output from “make test”, really. So until/unless I can replicate those failures, I’m not sure how to address them.

This isn’t really what I pictured, when I finally started getting back in the saddle…

Tags: , ,

And A Heavy Sigh Is Given

February 6th, 2014 | No Comments | Posted in Meta-Posts

I really don’t know which is worse; that it has been over a year since my last blog post, that it has been that long since my last CPAN upload, or that my last blog post was due to my last CPAN upload.

I need to work on this, somehow.

Tags: ,

Perl Module Release: RPC-XML 0.78

February 6th, 2014 | No Comments | Posted in CPAN, Perl, Software, XML
Version: 0.78
Released: Thursday February 6, 2014, 08:00:00 PM -0800


  • lib/RPC/XML.pm

A patch to loop detection in smart_encode from Dag-Erling Smørgrav. Some other minor bits.

  • lib/RPC/XML/Procedure.pm

RT #83108: Fixed a spelling error. Some other fixes, too.

  • lib/RPC/XML.pm

RT #86187: Force key-ordering in struct as_string and serialize. Was getting some intermittent bug reports of failures in t/15_serialize.t that amounted to the keys in a fault struct not being in consistent order.

  • lib/RPC/XML.pm
  • t/15_serialize.t

Undo the previous change and fix the test. The previous change didn't feel right, so this rolls it back and fixes the problem at the level of the test, instead.

  • Makefile.PL
  • lib/RPC/XML.pm
  • lib/RPC/XML/Client.pm
  • lib/RPC/XML/Server.pm

Replace direct evals for loading optional modules with Module::Load. Required adding this to Makefile.PL because Module::Load is not core in 5.8.8. Also did some slight doc tweaking.

  • lib/RPC/XML.pm
  • lib/RPC/XML/Client.pm

Merge pull request #5 from alexrj/utf8-encode. Use utf8::encode() instead of utf8::downgrade().

  • lib/RPC/XML.pm
  • lib/RPC/XML/Client.pm
  • lib/RPC/XML/Server.pm

Finish the uft8 encode vs. downgrade change from the previous commit. Changed in places that were overlooked, and adjusted the version number in all three modules.

  • lib/RPC/XML.pm
  • lib/RPC/XML/Parser/XMLLibXML.pm

Merge pull request #6 from dctabuyz/master. Added 'no_blanks' libxml option to skip blank XML::LibXML::Text nodes.

  • lib/RPC/XML/Server.pm

Merge pull request #7 from kvar/master. Initialize $do_compress in RPC::XML::Server between requests.

  • lib/RPC/XML.pm
  • lib/RPC/XML/Parser/XMLLibXML.pm
  • lib/RPC/XML/Server.pm

Bump version numbers on modules changed in github pulls.

Tags: , , ,