Perl QA Hackathon report (#perlqah2016)

Thank you to all the people who sponsored the Perl QAH hackathon and all those that provided their time. It was a very productive environment for a lot of projects.

I worked on 2 primary things relating to PAUSE permissions. PAUSE (indexing) itself and a new module for testing that permissions are consistent named Test::PAUSE::ConsistentPermissions.

The permissions of distributions on CPAN are generally something you don’t think about until you become a co-maintainer of a distribution and you see red on search.cpan.org or metacpan.org. Then you discover that there is a system by which each module in your distribution is effectively owned by somebody, and that some others might also have been granted permission to upload a file.

Let’s take Test::DBIx::Class as an example. I’m a co-maintainer, and JJNAPIORK is the owner. There is also another Co-Maintainer.

Using the script from Test::PAUSE::ConsistentPermissions I can look at the permissions currently assigned.

$ pause-check-distro-perms Test-DBIx-Class Test::DBIx::Class
Distribution: Test-DBIx-Class
Module: Test::DBIx::Class
Owner: JJNAPIORK
Comaint: NEWELLC, PHAYLON

When I upload a new release of Test::DBIx::Class to PAUSE any new modules added will be given to me. So I will be owner and no-one will be Comaint. I then need to grant PHAYLON comaint (which I can do as owner), and then pass ownership back to JJNAPIORK.

There is an alternative mechanism involving a bit of metadata named ‘x_authority’. With that we could ensure that JJNAPIORK retains ownership of all the modules within the distribution. The downside of that is that while I would also gain comaint on those new modules as the uploader, PHAYLON would not. Since I wasn’t the owner either, I wouldn’t be able to assign him Comaint, and I would have to ask JJNAPIORK to do that instead.

I believe there were historically alternative methods for managing this, but PAUSE has been through some rationalisation and simplification of some features and those features don’t exist anymore.

I came to this hackathon wanting to work on this problem having encountered Karen Etheridge (ETHER)’s prototype Dist::Zilla::Plugin::AuthorityFromModule which suggested a potential solution.

We had a meeting with the interested parties at the hackathon about how we could deal with this scenario better. Ricardo Signes (RJBS) suggested that we could make use of the fact that there is a designated ‘main module’ for permissions. We could  use that for the defaults. This is one step better than the previously suggested solution as it makes use of some of the previous rationalisation of PAUSE permissions and won’t require authors to provide extra metadata. This should mean that permissions are much more likely to just work out of the box.

With that potential solution suggested RJBS gave me some assistance to get working on the PAUSE indexer. I started with adding tests, then wrote the code.

The change made so far is very minimal, only affecting the indexing of new packages. No changes have been made to the user interface, or the complex permissions you can currently end up with. The pull request is here – https://github.com/andk/pause/pull/222. Note that it also benefits from Peter Sergeant (Perl Careers)’s work to hook the PAUSE tests into Travis giving it a green tick indicating the tests passed.

The other thing I worked on was Test::PAUSE::ConsistentPermissions to allow us to check that a distribution has consistent permissions. I created a script for checking a distro on PAUSE (not too dissimilar to Neil Bowers App::PAUSE::CheckPerms module) and a test function for using a distro’s release tests. This is a bit like Test::PAUSE::Permissions, but rather than check whether you have permission to upload the distribution, it checks whether the permissions are consistent. These two properties don’t necessarily coincide.

During the event I was also able to create a couple of minor pull requests.

Here’s the obligatory thank you to the full list of sponsors. Thank you all.

The sponsors for the Perl QA Hackathon 2016,

Advertisements

Getting out of CPAN hell

This is a quick and dirty way to reinstall/upgrade all your modules in cpan (assuming you have cpanm installed).


grep "C<Module> L<" `perldoc -l perllocal` | grep -v MyInternalModules | cut -c 46- | cut -d "|" -f 1 | sort | uniq | xargs cpanm --sudo

This is pretty brittle and only for emergencies.  Another trick pointed out by JJ who pointed me to perllocal is to grep the list further before the sort | uniq to just modules your likely to be interested in, to those starting Moose or Catalyst perhaps for example.

Another thing that I just realised is that while cpan* may not be perfect with spotting problem dependencies there are a lot of modules that make a fair effort to spot problems for you.  Moose and Catalyst both have checks in their makefiles to spot incompatble modules and warn you about them.  See the check_conflicts function in the Moose makefile for example.  This is the one area that running cpanm might bite.  The lack of spurious output also sometimes means lack of valuable output messages.  If you have run cpanm and are now experiencing problems you might want to check your logs to see if there were some conflicts,

find ~/.cpanm -name build.log -exec grep "conflicts with" {} \; -print

If you are upgrading a big module like Moose it might be worth flipping cpanm into verbose mode (-v) to spot those warnings when they happen.

* I don’t really mean the module CPAN.  Just the whole eco system of distributions and so on.  It’s a complex problem and not easily sorted in such a way that means it just works out of the box, although it’s pretty close…

 

Devel::Platform::Info::Win32

During my research of how to find out which operating system is running for the Devel::Platform::Info::Win32 module I found I could pull information from the following sources,

In practice with my initial implementation I haven’t pulled it from everywhere. Largely this is due to a lack of courage on my part. That and these concerns,

  • We need to make sure the module runs on as many machines as possible
  • Getting native win32 calls wrong seems to be a quick way to exit perl
  • 64 bit perl doesn’t seem to have all the same stuff loaded by default (at least the ActiveState version).

Should I fix this? It occurs to me that I’m probably being too cautious and if I add careful calls to check that modules exist before trying to use them and ensure I am checking everything (like the presence of Win32 API calls) then it ought to be able to continue to work on all platforms.

The thing that’s bugging me is that I know I need to use LoadLibrary and GetProcAddress. When I look those up in the Win32 help though they say that I should use Win32::API instead. Only that’s not present on my Perl 64 ActiveState distro making it not terribly useful. Given the simple nature of most of what I need to do I’d really like to call the original methods since they seem to be consistently there in most distributions. When I’ve googled the method though I’m coming up with very little by way of code that appears to use the calls so I’m really struggling to figure out what I should do. I’d normally expect some degree of prior art to be able to use as a guide with what to do.

I suppose I could create an XS module but I don’t think that’s an ideal option since we’d rather keep this module as lightweight as possible. I want it to run on any win32 perl with as little fuss as possible and an XS module isn’t likely to do that. This is a module for use across a lot of machines, not just for my own personal use where I could make compromises like that.

Of course you’re probably wondering what I’m complaining about. The net effect of what I haven’t done is that the module isn’t as exact at determining the OS version as it could be. It won’t get Windows Home Server. It doesn’t know the difference between Windows Server 2003 and Windows Server 2003 R2 or whether it’s a Small Business Server etc. It largely does get the major operating systems right though, at least as far as I’ve been able to test, from Windows 2000 onwards anyway.

(Incidentally I’m not trying to ding the ActiveState version of Perl. It’s my default Perl on Windows so I just have more of a clue of the out of the box behaviour).