Fun with the each function in Perl

I just had a fun time figuring out an odd bug with some code at work.

There was a loop something like,

my $var = $obj->meta->relationships;
while(my ($key, $val) = each %$var)
{
	warn $key;
	...
}

For some reason I wasn’t even seeing the warn $key being hit. I then debugged the code, had a look at var and stepped through and it worked perfectly. What the heck. What I’d have called a heisenbug before I looked up the term on Wikipedia and realised it’s a bit of a misnomer.

Technically it didn’t always work when I debugged, but if I examined the $var variable first it did. I then tried doing a warn Dumper $var; before the while and that made the code work regardless of the debugger. Now I was wondering what magic the dumping code did to the variable to the variable. Those of you familiar with the each function can probably guess. I wasn’t though so I rewrote the loop using a for and keys since I am more familiar with them and that fixed the code, without the need for a dump of the variable.

With the each looking like the culprit I then went hunting for the documentation. It turns out it’s a function so a perldoc -f each brings up it’s documentation. It turns out that this function uses an implicit iterator tied to the hash and is reset when the iterator ends, or when you call keys or values. That answered the question about my viewing the variable fixing the code. Dumper and the debugger must be calling keys in order to iterate over the hash to display it.

Given a general idea of how the function works I was then able to figure out how I’d create that sort of bug. Logically the each function must not have reached the end and reset itself after a previous loop. The most logical way to do that would be a last statement assuming the standard while(... each ...) type pattern was used everywhere. Sure enough, a grep of the source revealed a call in a different module which iterated over that hash using each and then broke out of the loop using last, i.e.

my $var = $obj->meta->relationships;
while(my ($key, $val) = each %$var)
{
	if($key eq $match)
	{
		last;
	}
	...
}

To fix that I could have called keys or values afterwards, but it struck me as more logical to simply change it to a for loop using keys instead,

for my $key (keys %$var)
{
	my $val = $var->{$key};
	...

This was a real head banger until I figured it out. The each function looks quite neat but it’s use of an implicit iterator means that while you can potentially do some very clever code, it also allows you to royally screw yourself over. Since I’m good enough at that already I think I’ll be avoiding the each function in general.

vim tricks for perl

I’ve been making use of some helpful vim snippets posted on blogs.perl.org for a little while now and I thought they deserved a quick mention.  Until I started writing this blog post I hadn’t realised they were actually be the same author.  Marcel Grünauer has been writing some of the most interesting blog posts on the new(ish) perl blog site from my point of view.  The series on vim tricks is here.The first script that adds use statements makes use of expand('<cword>') which out of the box doesn’t pick up the whole package name, stopping at the :: instead.  The script makes a note that this can be ‘fixed’ by changing the iskeyword setting.  One of the commenters luckily makes a patch to make that change while also localising the effect, so that it’s only done for that call, rather than requiring you to fundamentally change your vims behaviour.

I only read that comment when I was constructing this blog post though and so I had a play at vim scripting myself.  Instead of using expand I used matchstr and gave it a hand coded regex to find from the cursor to the end of the package name.

    let p = matchstr(line, '[A-Za-z_0-9:]\+', col('.') - 1)
    let s:package = input('Package? ', p)

Having read the alternative fix for that I think the expand('<cword>') is still the better way to do it, but it at least allowed me to finally do a tiny bit of vim scripting.  As it happens I also noticed a minor bug in the use statement script that causes issues when you try to add a use statement for a package named something that is a subset of an existing use statement.  i.e. you have use A::B::C; and you try to add a use statement for A::B.

This can be fixed simply by adding a little to the end of the regex that checks the use statement doesn’t already exist,

   if (search('^use\s\+'.s:package.'[^A-Za-z_0-9:]', 'bnw') == 0)

I also modified my version of the script to use <Leader> instead of , as the command character, so you type \us instead of ,us to execute the command.  That makes it more inline with his second script.

I’ve stuck my vim config onto github, as much as a backup as to share it.  It’s not really very sophisticated.  https://github.com/colinnewell/colins-perl-vim-config/blob/master/vimrc

Now I’ve started to get more into using this vim scripting I really ought to go back and take a look at some of Ovids past blogs to see what useful things I can find there.  He has tended to blog interesting editor snippets.

Have a look at Marcel’s blog I realised he also blogged another really interesting thing, the dip script.  In some ways I’m impressed he’s managed to demonstrate it so succinctly here, but I also can’t help but think he ought to be shouting about it more because the potential of that is freaking awesome.

Working together in development

Sometimes I’m surprised that I’ve learnt to work in a team as a developer. As with most programmers when I started, even as a professional, I was mostly a lone wolf. Over the years I’ve gradually become more of a team player as my environment around me grew and as I grew. In some ways it’s slower, but on the whole my standard practices mean I find it a lot easier to collaborate with others, even on projects I hadn’t planned on sharing. Who’d have thunk I’d become a team player!

When it comes to developing in a team there are a few habits worth adopting.

  • Check in frequently, at points where things work, in small increments.  Don’t worry that things aren’t finished.  Waiting for big chunks of functionality to be complete before collaborating means you make yourself a blocker on the project.  If you are working on something major and it’s not ready for prime time stick it in a branch, but make sure it’s still checked in and on the server.  Sometimes colleagues need to work on the incomplete big stuff, and sometimes they don’t.  At least it’s there if they need to see it.  While you ultimately want to always check in working code, if you have to move onto something else for a period of time make sure you check in what you have.  Code is no use in your working directory on your machine only.  Even commented out code can be more useful.  Even if you’re the only one who comes back to it.
  • Don’t worry if the code is ugly, if it works check it in.  You can make it pretty next.  If making it pretty breaks it you can always roll back.  Working together means you need to tolerate ugliness, both other peoples code and your own.  Don’t be too precious.
  • Check in experiments.  They might not work right now but you often find yourself wanting to use something useful you developed as part of the experiment, months after.  It also makes it a lot easier to share ideas with other colleagues about the experiments and avoid duplicate work.
  • Make sure what you’re doing doesn’t just work on your machine.  Either have a 2nd machine or get someone else to periodically check your code at least builds, and ideally that the program works from time to time.  If you’re at a small company and have a laptop make sure things build on that too.  If you’re on a large team think about continuous integration. It’s often when you try to get things going on the second machine that you remember the dependencies you’ve added and that’s a good time to document them.
  • Make sure there is documentation for how to set things up / Build scripts.  Start with documentation and then codify it in scripts. Keep these up to date as you ensure it works on more than just your machine. Dependencies are the biggest pain when collaborating on projects. Remove that pain for your colleagues by at least documenting them.
  • Make sure there is nothing stored solely on your machine.
  • If you send an email to a colleague on ‘how to get started’ check it after you’ve sent it.  Often you’ll find there is information in the email that’s not in the project documentation.  Paste it into that documentation.
  • Store the documentation in source control with the source or in a wiki. Either way it’s version controlled and accessible and available from more than just your machine.
  • Make sure there are at least some basic tests against the library/project/application. They will help you check a development environment is working on a new machine, as well as the other obvious reasons for testing. It can be quite a good way of smoking out dependency issues when you’re first setting up a dev environment.

All of these habits actually turn out to be useful even if you’re a sole developer.  Making sure everything is checked in, documented etc. all means that after you’ve been distracted for a month from the project you can come back to it with a new laptop knowing you can get it going again with a minimum amount of fuss.

It does take a little extra time doing the things like making sure it will work on a second machine, but it means when you do need to deploy/share/move machine you can do it at a snap and whenever you want to, rather than having to schedule time for it at some point in the future. There’s not much more frustrating than blocking someone from being able to help you on your project simply because you don’t have time to hand it over because of all the other work you are doing on another project.