I have tests and this is Perl so how hard can it be to convert over? With Catalyst::Action::REST indeed it is pretty simple to convert my calls, in fact it’s a case of changing the word PUT to POST in some of my function names. It’s the tests where things got interesting. I’m using Test::WWW::Mechanize variants to do my testing because it’s nice and simple. Unfortunately switching from put_ok to post_ok didn’t produce the desired results. When it came to the API it wasn’t reading the data at all. A bit of digging revealed that the post_ok call encoded the parameters in the application/x-www-form-urlencoded style before posting, where as the put_ok call just passed the json through raw.

Some digging into the Catalyst::Action::REST module revealed that they may well have had a similar issue because they created a little helper module called Test::Rest (not to be confused with Test::Rest on CPAN) which created the requests by hand for use in the test suite. Of course they may have simply been avoiding dependencies, and just been lucky to avoid the magic.

I didn’t manage to figure out a way to turn it off so in the end I did a similar thing. The fix for my test suite was to create a simple sub like this that rolled my own POST request with out any magic, then to call $mech->request to simply pass the request through like I was already doing with the DELETE’s. It’s basically a dumbed down version of a method from the Test::Rest from Catalyst::Action::REST.

use HTTP::Request;

sub construct_post
{
    my $url = shift;
    my $data = shift;

    my $req = HTTP::Request->new( "POST" => $url );
    $req->content_type( 'application/json' );
    $req->content_length(
        do { use bytes; length( $data ) }
    );
    $req->content( $data );
    return $req;
}

When it comes to testing it does not appear that the Catalyst::Test and Test::WWW::Mechanize::Catalyst modules automatically pick up your .psgi file when building the test server. This might be a feature for some tests, but sometimes you’ll definitely want to test the whole lot together. Luckily that’s fairly simple. When it comes to replacing Test::WWW::Mechanize::Catalyst you can switch to PSGI instead of Catalyst. Probably the closest thing to Catalyst::Test is Plack::Test. Both test modules require you to provide a $app object which Plack::Util makes it easy to load from your existing .psgi file.

Here’s a really simple test converted over to use a .psgi file.

use Test::Most;
use Test::WWW::Mechanize::PSGI;
use Plack::Util;

my $app = Plack::Util::load_psgi 'app.psgi';
my $mech = Test::WWW::Mechanize::PSGI->new( app => $app );

$mech->get_ok('/');

done_testing();

In this example I’ve created another report to use with sales orders. This is based on the sales order report so in actual fact I use some of the code and xml from that original report to build it up, then tweak them to produce the collection docket I want. I’m not actually going to reproduce the report here, since that’s a trivial thing to customise. The interesting thing is really how the module is packaged together and how to spot mistakes in the packaging.

The files/folders in the zip

module/__openerp__.py                # this contains the module info
module/report/collection_docket.rml  # the report
module/report/sale_order.py          # sets up the report parser
module/report/__init__.py            # just loads the code
module/reports.xml                   # registers the report
module/__init__.py                   # loads the code

__openerp__.py

{
   'name': 'Extra Sales Reports',
   'version': '0.01',
   'category': 'Extra reports for sales',
   'description': """
   The extra sales reports needed for our project,

   * Reports
     - Collection Docket

   """,
   'author': 'OpusVL',
   'website': 'http://www.opusvl.com',
   'depends': ['stock', 'procurement', 'board', 'sale'],
   'init_xml': [],
   'update_xml': [
       'reports.xml',
   ],
   'demo_xml': [],
   'test': [],
   'installable': True,
   'active': False,
}

__init.py__

import report

reports.xml

<?xml version="1.0" encoding="utf-8"?>
<openerp>
    <data>
        <report auto="False" id="collection_docket" model="sale.order" name="sale.collection_docket" rml="module/report/collection_docket.rml" string="Collection Docket" />
    </data>
</openerp>

report/__init__.py

import sale_order
report/sale_order.py
from report import report_sxw
import time


# this bit is basically a copy of the stuff
# in the regular sale order module.
# I can't just import that code because it causes
# the sale.order report to get registered again
# causing it to complain.
# otherwise I’d do this - from addons.sale.report import order 
class order(report_sxw.rml_parse):
    def __init__(self, cr, uid, name, context=None):
        super(order, self).__init__(cr, uid, name, context=context)
        self.localcontext.update({
            'time': time,
        })


report_sxw.report_sxw('report.sale.collection_docket', 'sale.order', 'addons/module/report/collection_docket.rml', parser=order, header="external")

report/collection_docket.rml

<?xml version="1.0"?>
<document filename="Sale Order.pdf">
... this is a copy of the addons/sale/report/sale_order.rml customised as necessary.

The module is then zipped up for distribution in a regular .zip file. This can either be imported directly into OpenERP or it can be unzipped manually into the addons directory.

Installation of the module via the OpenERP client

1. Go to Administration->Modules,
2. Select Import module and select the zip [1].
3. Select the module and mark it for install.
4. Now restart OpenERP server.
5. Now go back to the client and schedule the install of the module.

If you install your module this way you will actually find that the module is left in it’s zip file and the OpenERP server simply reads the files from the zip as if they were an extension of the addons directory.

Manual installation

1. Find the addons directory
2. Unzip the module into it.
3. Restart the openerp server
4. Go to Administration -> Modules
5. Select ‘Update Module List’.
6. Find the module and schedule it for install.

Use

The report can now be used programmatically using the standard report method and referencing it as sale.collection_docket. Alternatively you’ll find that a button has appeared on the sales order screen in the OpenERP client that allows you to print the collection docket alongside the button for printing the regular sales order report.

Trouble shooting

ZipImportError: bad local file header in /usr/share/pyshared/openerp-server/addons/myextra_reports.
zip

This normally indicates it is time to restart the server. If you have just imported the zip file of the module and tried to install it straight away you will often get this error.

ERROR:web-services:[01]: Exception: Report /usr/share/pyshared/openerp-server/addons/myextra_report/report/collection_docket.rml doesn’t exist or deleted :

This is generally caused by a typo in the parser python file where the report is registered with the report_sxw.report_sxw call.

ERROR:service:This service does not exist: ‘report.sale.collection_docket’
ERROR:web-services:[07]: KeyError: ‘report.sale.collection_docket’

The report hasn’t been registered. Has your module been installed and loaded against the current database? Remember that installing the module into the OpenERP server and installing it against your current database are two seperate steps.

ERROR:web-services:[01]: Exception: Start tag expected, ‘<' not found, line 1, column 1

This can be caused by a bad filename registered using the xml file. The path to the file should be in relation to the addons path. In other words, if the full path is /usr/share/pyshared/openerp-server/addons/module/report/docket.rml, the relative filename you need is module/report/docket.rml.
http://www.openerp.com/forum/topic26308.html

The alternative cause of this problem is that there is a unicode BOM indicator at the start of the file. One of the support entries appears to indicate that that will cause the parser to dislike the document.

https://bugs.launchpad.net/openobject-server/+bug/694409

[1] If you get a permissions error that’s normally because the addons
directory isn’t writeable by the user that openerp is running as.

Further reading

The OpenERP documentation regarding reports.

• Perl Advent Calendar
• Catalyst Advent Calendar

These are some of the essential posts to check out, if you only read a couple you should definitely have read these (they’re all good though).

• 2nd cpanm and cpanoutdated
• 4th Carp::Always
• 8th Perlbrew
• 10th List::Util
• 17th Try::Tiny
• 21st utf8::all
• 23rd Params::Util

Update: The 23rd’s item, Params::Util is another really handy tip.

Moose delegation can be read about here in the manual, Moose::Manual::Delegation. It allows you to have a property that you pass methods straight through to. Their example illustrates it simply,

 package Website;

  use Moose;

  has 'uri' => (
      is      => 'ro',
      isa     => 'URI',
      handles => [qw( host path )],
  );

moose-outdated is a tool that ships with Moose to help you spot dependency problems. It’s particularly useful after a major upgrade of Moose when you can run it to report any other modules that now require an upgrade as a result. It’s their way of helping you avoid CPAN hell.

$ moose-outdated 
$ 

If you’re lucky you get no output like this. Otherwise simply upgrade the modules it complains about.

        package SalesOrder;

        use Moose;

        has order_number => (is => 'ro', isa => 'Str', required => 1);
        has date_ordered => (is => 'ro', isa => 'DateTime', required => 1);
        has reference => (is => 'ro', isa => 'Str');
        has customer => (is => 'ro', isa => 'Customer', required => 1);

        1;

All those nice Moose constraints mean our objects have to get built with the correct information or there’s a blow up. Perfect. The problem I have, is that I want to use that same class to deal with defaults for that object. The defaults might be things like order numbers. In a lot of systems order numbers are generated by the system, like SO00001, then SO00002. I figured it would be neat if I could have a $sys->get_default_order() that returns me an instance of that class that has the default data. The problem is that it commonly won’t have all the information filled in. For instance, I’d probably get an order_number and date_ordered pre-populated but not the customer that we require. Attempting to construct a new sales order like this will blow up because of my constraints. As it was designed to.

    my $defaults = SalesOrder->new({
        order_number => 'SO0001',
        date_ordered => DateTime->today
    }); 

What I wanted to do instead was,

    my $new_class = MooseX::NotRequired::make_optional_subclass('SalesOrder');
    my $defaults = SalesOrder->new({
        order_number => 'SO0001',
        date_ordered => DateTime->today
    }); # it now works.
    # or even these examples,
    my $default = $new_class->new(); # no blow up
    my $default2 = $new_class->new({ reference => undef }); # fine too

With a start from doy on #moose and some help from my colleague JJ I was able to come up with this function that makes that work. It’s not perfect but it does a good enough job for now. It removed any required constraints from the attributes and changes simple isa definitions like 'Str' to 'Maybe[Str]' to allow undef.

sub make_optional_subclass 
{
    my $class = shift;

    my $meta = Moose::Meta::Class->create_anon_class(superclasses => [$class], weaken => 0);
    for my $att ($meta->get_all_attributes)
    {
        my $name = $att->name;
        my $options = {};
        if($att->is_required) {
            $options->{required} = 0;
        }
        my $type = $att->{isa}; # FIXME: this is ugly
        unless (ref $type) {
            unless($type =~ /Maybe/)
            {
                my $new_type = "Maybe[$type]";
                $options->{isa} = $new_type;
            }
        }

        if(scalar keys %$options)
        {
            $meta->add_attribute("+$name", $options);
        }
    }
    return $meta->name;
}

I am using this in practice for a system we are building at OpusVL. This code creates a new class, based on the old one, where the attributes are now optional. It takes care to ensure the original class isn’t changed.

duplicate key value violates unique constraint "ir_property_pkey"
DETAIL:  Key (id)=(181) already exists.

I know how it’s possible to create them, setup a database, then import data from a dump file that has setval statements like this, but I thought I’d learnt from that mistake and I’ve generally avoided that particular method of screwing up my database integrity.

SELECT pg_catalog.setval('account_account_id_seq', 203, true);

Anyhow, it’s become boring manually fixing them so I’ve created a quick script to spot the problems. It probably makes some naive assumptions but it seems to be working for me and making life easier for the right now.

#!/usr/bin/perl
use strict;
use warnings;

my %sequences;
my %ids;
my $current_table;
my $in_copy = 0;
while(<>)
{
    if(/pg_catalog\.setval\('(\w+[^']*)_id_seq',\s*(\d+),/i)
    {
        $sequences{$1} = $2;
    }
    if(/COPY (\w+)\s*\(id/i) 
    {
        $in_copy = 1;
        $current_table = $1;
    }
    elsif($in_copy)
    {
        if(/\\\./)
        {
            $in_copy = 0;
        }
        else
        {
            /^(\d+)\t/;
            my $id = $1;
            $ids{$current_table} = $ids{$current_table} && $ids{$current_table} > $id ? $ids{$current_table} : $id;
        }
    }
}

for my $table (keys %ids)
{
    my $id = $ids{$table};
    my $sequence = $sequences{$table} || 0;
    if($sequence < $id)
    {
        print "The sequence for $table is too low ($sequence), it should be set to $id\n";
    }
}

It is designed for the non trivial command line app which has multiple ‘tasks’. Take a look at programs like git and svn for an example. Dist::Zilla is the most well known of application I know of that has actually been implemented using the library. It makes it trivial for you to push those tasks into separate modules and builds the help system automatically. It deals with all the command line parsing but you can override that if you want to. I like it’s default behaviour though because it integrates nicely into the help system and seems generally powerful enough.

The only negatives I’ve found are trivial, and they’re really just things to avoid rather than something to worry about.

• You do want to make sure that you are not loading heavy weight modules in your commands immediately by requiring them rather than using them, otherwise the help system gets bogged down. (This is noted in the documentation)
• The task names are based on the module names so your tasks have to obey the same punctuation rules as Perl module names.

There were a couple of things that made my progress with the module a little slower than they should have been. Until I can figure out how to turn my feedback into a a contribution back to the project I am noting it here so that I don’t forget them.

• Filling in the abstract is necessary for the help when you run your app with no commands. There are two methods, provide an abstract function in your command which returns the abstract, or setup your pod correctly. The bit of the pod it’s looking for is the bit after the – in the =head1 NAME section. i.e. “App::Command::task – abstract goes here”. See the ‘ABSTRACT_FROM’ setting in ExtUtils::MakeMaker for a better description of the Abstract. I guess this exposes the fact that I haven’t been in the habit of filling in the abstract in my pod!
• You need to look at the documentation for App::Cmd::Command in order to figure out what your commands can override and use. This is not really made clear in the documentation and the code because you use App::Cmd::Setup and then use your own app with -command to actually inherit from it. That means the module name appears nowhere in your own code, and should really be referenced in the documentation somewhere.
• Using global options isn’t really touched on much in the documentation. That would be really useful since an app commonly deals with a common theme. I need to figure out what is possible with the module because I missed that capability when I constructed my first app.

As you can see, this is basically just niggles with the documentation. The main reason I haven’t simply contributed a doc patch already is that I haven’t figured out the best way to express this. Hopefully elucidating these points helps me figure out how to document them better.

It’s a fine module and I can definitely see myself using it a lot in the future.

#     Tried to use 'Net::SSL'.
#     Error:  Can't load '/home/user/.cpanm/work/1319487963.639/Crypt-SSLeay-0.58/blib/arch/auto/Crypt/SSLeay/SSLeay.so' for module Crypt::SSLeay: /home/user/.cpanm/work/1319487963.639/Crypt-SSLeay-0.58/blib/arch/auto/Crypt/SSLeay/SSLeay.so: undefined symbol: SSLv2_client_method at /home/user/perl5/perlbrew/perls/perl-5.14.1/lib/5.14.1/x86_64-linux/DynaLoader.pm line 190.

Luckily there is a patch attached to the case, nossl2.patch which can be applied to make it just work.

If you’re using cpanm, run it in prompt mode and look at the errors when they occur. That will allow you to patch the module and get it working again.

$ cpanm Crypt::Sleay --prompt
--> Working on Crypt::SSLeay
Fetching http://search.cpan.org/CPAN/authors/id/N/NA/NANIS/Crypt-SSLeay-0.58.tar.gz ... OK
Configuring Crypt-SSLeay-0.58 ... OK
Building and testing Crypt-SSLeay-0.58 ... FAIL
Testing Crypt-SSLeay-0.58 failed.
You can s)kip, r)etry, f)orce install or l)ook ? [s] l
Entering /home/user/.cpanm/work/1319488454.2107/Crypt-SSLeay-0.58 with /bin/bash
$ patch < ~/Downloads/nossl2.patch
patching file SSLeay.xs
$ make
$ make test
$ exit
You can s)kip, r)etry, f)orce install or l)ook ? [s] r
Successfully installed Crypt-SSLeay-0.58

The failures aren’t really any ones fault, it’s simply that things move on. I’m assuming the module will be patched soon and this blog post will become redundant. Until then hopefully this helps people find the RT case with the solution.

The more I use those sites, the more I realise they have a lot of intelligent ideas demonstrated.