Tag Archives: cpan

Dist::Zilla, cpanhacker, do you use it??!

Dist::Zilla, cpan hacker, do you use it?

If not, you should. Full stop.

So I've been reading about Dist::Zilla (dzil) for a while now. The first articles didn't really convince me too much. Yes, it's a nice tool, but why would I want to use it? Ok, to save time. Right. But how do I use it the first time? I need a config file. Ok where do I get (a sane) one?

Then I read a clear walk-through example of a Dist::Zilla config by Dave Rolsky. So, the quick start route was clear. Copy/paste the dist.ini file and try it.

Except, the file in that blog post contains so much stuff (read: additional plugins required) that is not practical to use as a first dist.ini config.

Basic premise

It's that you have to have a module/class that you want to upload to the CPAN, and you want to save yourself all the "package-wrapping" activities that a CPAN distribution needs. That is were dzil shines.

Start-using-Dist::Zilla-in-2-minutes guide

Good, you're on your way to spend countless hours less on maintaining your CPAN stuff.

  • First things first: write the thing that you want to package and upload to CPAN. Fatto? Bene.
  • So you probably have a lib folder with your class and a t folder with the test scripts for your class. Good.
  • Then, usually if you want to package everything for CPAN, you need to add a MANIFEST, a README, INSTALL, Changes, possibly a LICENSE, etc… Forget about those!
  • Copy paste the following into a dist.ini file:
    ; Everything starting with ';' is a comment
    
    name    = Your-Awesome-Module
    author  = Joe J. Hacker <jjhacker@example.org>
    license = Perl_5
    copyright_holder = Joe J. Hacker
    copyright_year   = 2010
    
    version = 0.01
    
    [@Basic]
    [InstallGuide]
    [MetaJSON]
    
    [MetaResources]
    bugtracker.web    = http://rt.cpan.org/NoAuth/Bugs.html?Dist=Your-Awesome-Module
    bugtracker.mailto = bug-your-awesome-module@rt.cpan.org
    
    ; If you have a repository...
    ;repository.url    = git://github.com/jjhacker/your-awesome-module.git
    ;repository.web    = http://github.com/jjhacker/your-awesome-module
    ;repository.type   = git
    
    ; You have to have Dist::Zilla::Plugin::<Name> for these to work
    ;[PodWeaver]
    ;[NoTabsTests]
    ;[EOLTests]
    ;[Signature]
    ;[CheckChangeLog]
    
    [Prereq]
    ;DBI = 1.600
    ;SomeOtherModule = 0
    
    [Prereq / TestRequires]
    Test::More = 0
    
    ; If you're using git, this is interesting
    ; You need to install Dist::Zilla::PluginBundle::Git
    ;[@Git]
    
    
  • Add the following line at the top of your main class (.pm) file:
    # ABSTRACT: Supercool generator of twisty little passages, all alike
    
  • Run dzil build

That's it! The build command will do everything for you. When I realized how quick this tool makes my CPAN activities, I decided I had to try to convince some more people out there. Until I didn't really try it, I wasn't convinced 100%. Now I definitely am. Life's just too short. Use DZIL.

There's also a release command, that will also send your generated distribution archive to the CPAN directly. I couldn't get that to work. I remember there was some kind of dot file I had to write my credentials to before I could release, but I couldn't find the details today. If anyone knows, please comment here… :-)

Mocking Cache::Memcached

Today I've been working on some elaborate unit tests that require a database and a memcached object. In My Opera, like probably everywhere else, we use our own classes for both DBI and memcached access. The memcached class in particular is just a subclass of Cache::Memcached, so nothing special there.

I was looking at already existing modules that would mock Cache::Memcached without requiring a running memcached server. The only relevant one I could find is Test::Memcached.

However, Test::Memcached interacts with the memcached binary, trying to start/stop it. This is not really what I want to do. I could do that, but it would be slightly complicated because we have lots of test installations, and we'd have to install or require a memcached daemon running everywhere.
A single shared memcached daemon wouldn't be so smart either since the different installations would interfere with each other. We could probably use key namespaces for that. Mmh.

Anyway, I decided that having a mock object, something that could be named Cache::Memcached::Mock, or Test::Memcached::Mock could be simpler and more easily testable as well. So I wrote a prototype that subclasses Test::MockObject that works fine for now and covers all my needs. It uses just a regular hash as memory storage, and supports get(), set(), flush_all(), and delete().

Not sure if I should upload it to CPAN…

Dependencies suck

We love dependencies. For example, in the CPAN universe. They make our job so damn easier. Thousands of production quality, unit tested modules at your fingertips.

But dependencies also suck really badly, for example when you're using a Linux distribution that has packages that are just too old to be useful. Hey, but they are stable! More stable-as-dead or more stable-as-production-quality? You decide.

It's been many months since I installed a local instance of Transifex, a Django application that allows translators to easily contribute to projects. We're using it for My Opera, but also trying to get other internal projects to use it.

So far, it has worked nicely. I think Transifex is a really good application, its feature set is just right for what we need etc… Last week I decided to upgrade our Transifex instance from v0.8.0-devel to 0.9.0-devel. The improvements were really nice and needed, so I just decided to go for it. I had been upgrading in the 0.8.0 series from their repository (aka HEAD, aka master, aka trunk).

This time though, the list of dependencies was a bit more specific than usual. Also, please note that 0.9.0 is a **BLEEDING EDGE** development version as of June 2010.

Anyway, first dependency listed was "Django = 1.1.2". I think I started going down the wrong path when I upgraded Django with:


$ sudo easy_install 'Django>=1.1.2'

Here you can see that my mind is somewhat hardwired to the Perl culture, where backward compatibility is of paramount importance. I wrote code 10 years ago, using perl 5.005, that it's still in production, unmodified, with perl 5.10, and I'm talking about commercial stuff, not silly home projects. The terrible mistake here is to think that this also applies everywhere else. Forget it. It's not true.

In fact, easy_install picked up Django 1.2.1, which is an entirely different beast that breaks at least a couple of assumptions that Transifex was making. I don't remember exactly now, but one had to do with the automatic export of email.MIMEBase into django.core.mail and another I only remember it broke horribly.

So, a couple of hours later, thanks to the guys on the #transifex channel, I figured out that what I really needed to write was:


$ sudo easy_install 'Django==1.1.2'

This forces to install the given version instead of any later one. So far so good. Then I had another problem, completely unrelated, the required me to strace the ./manage.py Django script, to figure out that it was using a totally screwed up sqlite database coming from a year old test version of transifex I had installed through easy_install and was completely ignoring my local settings that went to a MySQL db. How nice.

So, yes, we always complain about CPAN, dependencies, Module::Install, ExtUtils::MakeMaker and whatnot, but a look at other worlds (easy_install, ruby-gems anyone?) can remind Perl people of the fantastic toolchain and especially culture "we" have built, and that's still kicking everyone else's ass, on any platform.

So, regarding the debate in the Perl community, my vote goes to keeping Sane(tm) backward compability standards, as we always did. It matters, especially for commercial software companies!

My silly twitter OAuth command line client

I had the need to test a new My Opera API module that is soon coming out. Since this module, to be named Net::MyOpera, is modeled exactly after Net::Twitter, I tried changing my example script replacing all Net::MyOpera occurrencies to Net::Twitter. And there you go, a Twitter command line client was born.

I know, there's plenty of them already, and as I said, I didn't really need one, but since it's there already, it's nice to have it. So I saved it into my ~/bin folder, and aliased to tw, so whenever I feel the urge to communicate stupid things to the universe, I can now do that. Ehm wait… :-)

The code is out on Github.

As always, there's a hidden (poor) excuse for this. And it's that I'm working to port this Twitter command line client to Perl6. OAuth support needs a good deal of modules that are not immediately available for Perl6, so it's going to be exciting.

First we're going to need are Digest::SHA1, and Digest::HMAC for the HMAC-SHA1 signatures. These modules are not impossible to write, except currently there's a problem using Parrot libraries from Perl6.

I'm trying to do the same for my Perl6 Digest::MD5 module, but I'm stumbling on the following error:

t/perl5-compat.t ... Null PMC access in find_method('signature')
  in 'Digest::MD5::md5_hex' at line 11:lib/Digest/MD5.pm
  in main program body at line 17:t/perl5-compat.t
t/perl5-compat.t ... Dubious, test returned 1 (wstat 256, 0x100)

I will need some help on this :-)

Disassembling a real world Plack PSGI application

After I started playing with Plack, I tried to evaluate whether to continue using it for our mission-critical production stuff or give up, going back to the same techniques we already use (successfully).

I think it's time to develop and deploy a Plack based application. In my grand plan, :-), I'd like to deploy nginx with PSGI support, or even more ambitiously, nginx or apache with Starman as "backend" http server. We'll see…

In the meantime, I'd like to write here a couple of niceties about Plack and Starman, showing some real code I wrote when I started.

A real world PSGI application

Here's a sample PSGI application currently under development:

#!/usr/bin/env perl
#
# Sample PSGI application
#

use strict;
#se warnings;
use constant ENVIRONMENT         => 'development';
use constant APACHE_DEPLOYMENT   => (ENVIRONMENT eq 'production');
use constant ENABLE_ACCESS_LOG   => (ENVIRONMENT eq 'development');
use constant ENABLE_DEBUG_PANELS => (ENVIRONMENT eq 'development');

use Plack::Builder;
use AuthOpera;
use AuthOpera::Account;

my $app = AuthOpera::Account->new(); 

builder {

    enable_if { not APACHE_DEPLOYMENT }
        'Plack::Middleware::Static', 
        path => qr{^/(bitmaps/|images/|js/|css/|downtime/|favicon.ico$|ping.html$)},
        root => '..',
        ;

    mount "/account" => builder {

        enable_if { ENABLE_DEBUG_PANELS } 'StackTrace';
        enable_if { ENABLE_DEBUG_PANELS } 'Debug';   # panels => [ qw(DBITrace Memory) ];
        enable_if { ENABLE_DEBUG_PANELS } 'Lint';
        enable_if { ENABLE_DEBUG_PANELS } 'Runtime';
        enable_if { ENABLE_ACCESS_LOG   } 'AccessLog';

        $app;

    }

}

Of course, the main application code is not here, but in the AuthOpera::Account class. That's not really relevant to what we're discussing here. Let's just say that any class, to be a valid and complete PSGI application, has to:

  • subclass from Plack::Component
  • have a call() method
  • the call() method must return a valid PSGI response. Example:
    package MyPSGIApp;
    
    use strict;
    use Data:: Dumper ();
    use parent 'Plack::Component';
    
    sub call {
        # $env is the full PSGI environment
        my ($self, $env) = @_;
    
        return [
    
            # HTTP Status code
            200,
    
            # HTTP headers as arrayref
            [ 'Content-type' => 'text/html' ],
    
            # Response body as array ref
            [ '<!DOCTYPE html>',
              '<body><h1>Hello world</h1><pre>',
              Data:: Dumper:: Dumper($env),
              '</pre></body></html>',
            ],
        ];
    }
    
    1;
    

That's it, this is a full PSGI application that does dump all its PSGI environment.

Of course in a real example, you probably want a template engine to return the page content, etc… That's what we are building for our applications. Actually just assembling the components we already have developed during these years, so we have template classes, config classes, localization, database access, etc…

So we're basically just gluing these ready made components inside the PSGI application, and then using them. I don't think this is particularly original, but it allows us to quickly "port" our code to PSGI and thus run anywhere we want to.

app.psgi in detail

Now, let's see the PSGI app in more detail.

use constant ENVIRONMENT         => 'development';
use constant APACHE_DEPLOYMENT   => (ENVIRONMENT eq 'production');
use constant ENABLE_ACCESS_LOG   => (ENVIRONMENT eq 'development');
use constant ENABLE_DEBUG_PANELS => (ENVIRONMENT eq 'development');

These constants are used to turn on and off certain features mentioned later in the builder {} block. I just found out the other day that these constants are near to useless. That is because plackup and starman already provide a -E environment switch. If you start your application with:

starman -E development myapp.psgi     # same with plackup, the default server

then Plack will by default enable the debugging panels and the Apache-style access log. I found out about this after having written that file. This means that the following enable_ifs are unnecessary:

mount "/myroot" => builder {
    enable_if { ENABLE_DEBUG_PANELS } 'StackTrace';
    enable_if { ENABLE_DEBUG_PANELS } 'Debug';   # panels => [ qw(DBITrace Memory) ];
    enable_if { ENABLE_DEBUG_PANELS } 'Lint';
    enable_if { ENABLE_DEBUG_PANELS } 'Runtime';
    enable_if { ENABLE_ACCESS_LOG   } 'AccessLog';
    $app;
}

I think Plack enables by default at least StackTrace, Debug, and AccessLog. In my case, however, I'm also enabling RunTime and Lint. But more importantly, I need to differentiate between Apache deployment and Starman deployment. That affects the way static files are served.

When deploying under Apache, I don't need the following:

enable_if { not APACHE_DEPLOYMENT }
    'Plack::Middleware::Static',
    path => qr{^/(bitmaps/|images/|js/|css/|downtime/|favicon.ico$|ping.html$)},
    root => '..';

because my PSGI application is enabled in an Apache <Location> block, as in:

<Location /myroot/>
    SetHandler perl-script
    PerlResponseHandler Plack::Handler::Apache2
    PerlSetVar psgi_app /my/path/to/app.psgi
</Location>

So Apache already takes care of serving the static files for me. However, when running completely under Starman, I need to tell it which folders or paths need to be served as static files, and where they are located. This is the purpose of the Static middleware:

enable_if { not APACHE_DEPLOYMENT } 'Plack::Middleware::Static',
    path => qr{^/(images/|js/|css/|favicon.ico$)},
    root => '/var/www/something';

If you're always deploying through plackup or starman, then, again, you don't need any enable_if, just enable. Maybe it's also a good idea to put everything under /static. For me that wasn't possible, since I already had existing content:

enable 'Plack::Middleware::Static',
    path => qr{^/static/},
    root => '/var/www/something';

Plack::Builder

About the Plack::Builder bit, and the related builder function. That is a function that helps you specify what you want Plack to run and how. Example:

builder {
    enable 'StackTrace';
    enable 'Debug';
    enable 'AccessLog';
    $app
}

where StackTrace, Debug, and AccessLog are all middleware classes, so causes Plack to wrap your final $app application first with the AccessLog middleware, then Debug and then StackTrace. I didn't check the code, but I believe this creates 3 different PSGI applications that are meant to fiddle with the response that your own application generates.

PSGI makes this possible, and it's just great. More middleware means easier and faster development. And ultimately, very good middleware makes for great reuse too.

The mount wrapper

I used mount in my example very basicly, but you can use mount to assemble compounds of applications in a very simple way. The same thing you do, for example, with Django and urls.py, except that, if you have seen a non-trivial urls.py, it looks like spaghetti after a while. Compare with this:

my $app1 = MyApp->new();
my $app2 = MyApp2->new();
#...

builder {

    enable 'Plack::Middleware::Static', 
        path => qr{^/static/},
        root => '/var/www/something';

    mount "/path1" => builder {
        enable 'StackTrace';
        $app;
    }

    mount "/path2" => $app2;

    mount "/path3" => builder {
        enable 'SomeMiddleware';
        $app3;
    }

}

Of course, then you have to add some dispatcher logic to your applications, but in the Plack world, we don't lack good dispatchers.

Plack rocks.

LWP::Simple for Perl 6

During the last Perl 6 hackaton in Oslo, I got to meet in person some very cool folks from the Perl Community, and we had a lot of brainstorming fun, as usual.

I went there with the ambitious (and out of my skills, most probably) goal of implementing a Socket interface for Perl 6. Talking to the various smart folks there, I realized that we didn't need to write that much, because Parrot, on which the current Perl 6 implementation is based on, already had sockets support.

After much nagging, I wrote a quick & dirty wrapper that mimicked the existing Perl5's IO::Socket library, that Carl and Martin improved. And on top of that, we were able to write a really tiny LWP::Simple-like class for Perl 6.

Now it's on up on github, go fetch it!, before it's too late :)

Win32::API v0.52 released to CPAN

I just released version 0.52 of Win32::API. It should be available at your local CPAN mirror very soon.

This version should play nicer with CPAN testers, since it checks if the OS makes sense before going through Makefile.PL, so no more failures with Solaris, Linux, FreeBSD, and such are expected.

Please note that I published 0.51 one hour ago, but it's broken, so I have already scheduled its deletion. Please don't use 0.51.

Enjoy and please report to me any problems you have.

UPDATE: it seems that I screwed up again the META.yml and the dist resulted as UNauthorized because of my inclusion of Devel::AssertOS. Oh well…

Here's a list of latest changes:

2008-03-01 Win32::API v0.52 Cosimo

– Devel::AssertOS was not properly set up in 0.51.

2008-03-01 Win32::API v0.51 Cosimo

– Cleaned up API.pm pod docs and clearly stated Win32::API license
– Fixed $$/pid tests for Cygwin
– Now uses Devel::AssertOS to check that we are on a Win32 or Cygwin system. This should ease the work of CPAN testers.

2008-02-23 Win32::API v0.50 Cosimo

– Fixed RT#31702 http://rt.cpan.org/Public/Bug/Display.html?id=31702
Thanks to RUrban for supplying the fix.

2008-02-20 Win32::API v0.49 Cosimo

– Fixed the stack cleanup assembler statements for GCC in API.xs, I hope.
– Fixed a Borland C macro definition in API.xs
– Fixed META.yml (stupid me)

2008-02-20 Win32::API v0.48 Cosimo

– Finally applied the cdecl/stdcall patch available since long time
from http://www.xs4all.nl/~itsme/projects/perl/.
Now Win32::API *can* work with cdecl DLLs.
Fixes RT#32424 http://rt.cpan.org/Public/Bug/Display.html?id=32424
and RT#24685 http://rt.cpan.org/Public/Bug/Display.html?id=24685
Thanks to Willem Jan Hengeveld (itsme at xs4all.nl) for his great work. Thanks to JimK for a test case and to BrowserUk and others at PerlMonks for providing me useful information and complaints. :-)

However, still missing some GCC assembler magic for stack cleanup. Any help?

– Clarified licensing info. Yes, Win32::API is available with GPL 2 / Artistic license.

Win32::API v0.48 adds cdecl function call support

In my (really little) spare time, I enjoy hacking on fun projects.
Lately, I have been struggling to get Win32::API Perl module to work on Vista with various MSVC++ and GCC compilers.

Since many have asked for it, and some brave soul had an old patch available on the net, I decided to give it a shot. I had already tried 2 or 3 times to get this patch (originally for v0.41) integrated in the new Win32::API versions, but always failed at it.

Today I decided to take the full journey, and seems I succeeded. Probably I broke something else… :-)
Please bare with me. Anyway, I have 2 things left in this release that I want to address in the new one:

– There's a Borland C macro def that is completely broken (thanks to BrowserUk for noticing)
– I need to complete the stack cleanup assembler code at the end of Call() function in the XS code. This should be easy, for some definition of "easy", since I already have some examples of MSVC/GCC assembler code in my RT queue.

Stay tuned for further updates…

Have you ever heard “Bon Digi”?

It's a game. A crazy one. A really crazy one.
We were doing it friday evening, drinking like nuts, and something in my mind popped up, and I thought:

We should code the algorithm for this game

In fact, it seemed really stupid and obvious. But thinking about it more and more, I discovered that it's not so simple at all…

And if you want to code it elegantly and concisely, you have to think about it for some time.
So, I took this -Ofun opportunity, and turned it, as I promised to Zoso, into a Games::BonDigi CPAN module.

Enjoy!