Tag Archives: perl6

Perl6 LWP::Simple now uses the URI module for added awesome

My little LWP::Simple module for Perl 6 is growing beyond my control. I just merged a a pull request with a patch to remove all existing URL parsing to replace it with the new URI class.

That's fantastic, because:

  • it reduces the amount of code in LWP::Simple
  • the URI module is based on the actual grammar for URIs and IPv6 addresses. Plain and simple. It doesn't adhere to the standard. It is the standard. Isn't that amazing?

So, just clicky clicky and the pull request was merged. Running the test suite confirmed that everything still works fine passing all tests. Can it get much better than this? :)

If you want to have a look at Perl 6 code, modules.perl6.org is the place to go for inspiration. Have fun!

Facter ported to Perl 6

A few days ago I wrote about my fun experiment trying to port facter to rakudo Perl 6. I said it was "almost completely functional".

Well, I was a bit optimistic, it seems :) It took me a few more nights of hacking, but in the end now it's almost completely functional :) It basically runs, I have ported a few facts from Facter's original ruby code too, like the kernel fact, or the physicalprocessorcount fact and other simple ones.

What's missing to declare the experiment successful is the implementation of confines. A confine in facter speak it's a specific restriction that applies to a fact. For example, the physicalprocessorcount fact reads some files from /proc, and that is only available on Linux. So, in this case, the confine rule for physicalprocessorcount is that the fact kernel must have "Linux" as its value. In code that becomes:

Facter.add("physicalprocessorcount", sub ($f) {
    $f.confine("kernel" => "Linux");
    $f.setcode(block => sub {
        Facter::Util::Resolution.exec('grep "physical id" /proc/cpuinfo|cut -d: -f 2|sort -u|wc -l');
    });
});

which is pretty similar to the Ruby counterpart:

Facter.add("physicalprocessorcount") do
    confine :kernel => :linux

    setcode do
        ppcount = Facter::Util::Resolution.exec('grep "physical id" /proc/cpuinfo|cut -d: -f 2|sort -u|wc -l')
    end
end

The Ruby version is still more elegant, but all in all I'm very happy with the outcome so far. It could probably be improved a lot too. Perl 6 is awesome. Get the code from http://github.com/cosimo/perl6-facter/ and feel free to ping me or comment if you want to know more.

Porting Facter from Ruby to Perl6

I've been playing with Puppet for a while now. One of the most interesting (and simple!) components of Puppet is Facter.

Facter is a small software that reports "facts" about your computer. When you run facter, its output looks like the following:

architecture => x86_64
facterversion => 1.5.6
fqdn => cd01.localdomain.lan
hardwaremodel => x86_64
hostname => cd01
id => cosimo
interfaces => eth0,pan0
ipaddress => 10.0.0.1
...
uptime_hours => 422
uptime_seconds => 1519256
virtual => physical

It is interesting also because it's extensible with your own custom plugins. A custom plugin is just a Ruby file with usually a call to Facter.add:

Facter.add(:kernel) do
    setcode do
        require 'rbconfig'
        case Config::CONFIG['host_os']
        when /mswin|win32|dos|cygwin|mingw/i
            'windows'
        else
            Facter::Util::Resolution.exec("uname -s")
        end
    end
end

As a little experiment to become more familiar with Ruby and at the same time to enjoy writing some Perl 6, I decided to study the facter project and then port it to Perl6.

That took me a few hours over a couple of weekends, and it's almost completely functional. It is a straight port from the Ruby code, so it doesn't really use the magic powers of Perl6 yet. The idea is to use a different branch, now that I know it well under the hood, and to rewrite it from the ground up in Perl6.

I found out that Ruby code maps very closely to Perl6, apart from the yield instruction and a different model for static/instance variables. For yield, something that would be close in Perl6 is gather/take, but I'm not really sure that is the appropriate statement to use. yield is used in the fact value resolution algorithm, and that is currently the only thing that doesn't work properly in my port. Everything else is in place. Regarding static/instance variables, Ruby uses @attribute for instance variables, and @@attribute for static variables. In Perl6, instance variables are denoted with $.attribute if public, while static variables are package globals, so you can use our $attribute.

Of course, there's a lot more! If you're interested, take a look at the code on Github, the URL is http://github.com/cosimo/perl6-facter.

More Perl 6 development, String::CRC32

A couple of days were plenty enough to whip up a new Perl 6 module, String::CRC32.

This is a straight conversion from the Perl 5 CPAN version of String::CRC32. That one uses some C/XS code though. I wrote the equivalent in Perl 6. Was quite simple actually, once I learned a bit more on numeric bitwise operators.

The classic || and && for bitwise "OR" and "AND" are now written as:


Bitwise OR  '+|'
Bitwise AND '+&'
Bitwise XOR '+^'

So the following code:


$x &&= 0xFF;

becomes:


$x +&= 0xFF;

The reason behind this is that now there are not only numeric bitwise operators, but also boolean ones. You can read more in the official specification for Perl 6 operators.

You can download the code for the module on http://github.com/cosimo/perl6-string-crc32. Take a look at various other Perl 6 modules on modules.perl6.org.

Perl 6 LWP::Simple gets chunked transfers support

With this one I think we're basically done!

Perl6 LWP::Simple gets chunked transfers support. It's probably not excellent or universally working, but for the examples I could try and test, it's totally fine. If you find some URLs where it's broken, please tell me.

I also threw in the getstore() method to save URLs locally.

So, LWP::Simple for Perl 6 is here and it's working. It's not yet "complete" compared to the Perl 5 version, but now that I got the hard bits working, and the internals can perform a full http response parsing, I'll try to reach a 100% API compatibility with the Perl 5 one (where it makes sense).

Try it and let me know. Have fun!

Digest::MD5 for Perl 6 finally works!

It took me an awful long time, and lots of help from the folks on #parrot and #perl6 but in the end, it's done!

It needs a tiny patch to Parrot, but I believe it will be added to the next parrot release. I tried to documented the fixes to the code to help others that might have the same problems.

So now Digest::MD5 for Perl 6 works as good as the Perl 5 one. I'm too tired to say anything else :)

Good night!

LWP::Simple for Perl 6, now with (partial) BasicAuth support and getstore()

I just pushed out another update for the LWP::Simple module for Perl 6. This time, the main work was:

  • refactoring the code and adding unit tests for the URL parsing (that might even grow into a Perl 6 URI module
  • adding partial basic auth support. To be complete and working, it needs to base64 encode the user/password pair. Not implemented yet. I'll see if I get around to it, or if someone has done it already.
  • adding a getstore() method, that writes on to disk the downloaded content. Unfortunately that needs to strip the HTTP headers and undestand chunked transfers for it to be remotely useful.

It was nice to see the module grow in both functionality, code and unit tests coverage. I had to workaround a couple of problems I couldn't understand. I was extremely lazy and I didn't even look up Synopses, so I assume it's my fault. However.

The first is the use of .match() and ~~ to match against a regular expression. I found that the following code:

my $hostname = 'cosimo:eelst@faveclub.eelst.com';
if $hostname.match('^cosimo') {
    # Doesn't enter here
}

doesn't trigger a match. However, this other here:

my $hostname = 'cosimo:eelst@faveclub.eelst.com';
if $hostname ~~ /^cosimo/ {
    # Does match
}

And, in the same way, something similarly surprising. The following code correctly matches:

my $hostname = 'cosimo:eelst@faveclub.eelst.com';
if $hostname ~~ /^ .+ : .+ @ .+ $/ {
    say '(user:pass@host) matches';
} else {
    say '(user:pass@host) does not match';
}

but adding captures makes the same exact regex fail:

my $hostname = 'cosimo:eelst@faveclub.eelst.com';
if $hostname ~~ /^ (.+) : (.+)  @ (.+) $/ {
    say '(with captures) matches';
} else {
    say '(with captures) does not match';
}

There are also very nice things about programming in Perl 6 that are slowly sucking me in this fantastic language. This is part of a test script for LWP::Simple:

#
# Test the parse_url() method
#
use v6;
use Test;
use LWP:: Simple;

my @test = (
    { User-Agent => 'Opera/9.80 (WinNT; 6.0) Version/10.60' },
    "User-Agent: Opera/9.80 (WinNT; 6.0) Version/10.60rn",

    { Connection => 'close' },
    "Connection: closern",
);

for @test -> %headers, $expected_str {
    my $hdr_str = LWP:: Simple.stringify_headers(%headers);
    is($hdr_str, $expected_str, 'OK - ' ~ $hdr_str);
}

Note how in the for statement we can "extract" the hash and string from the @test array with:

for @test -> %headers, $expected_string {
    # Loop body
}

It's not a big deal, other languages have it, but Perl 6 is filled with this small niceties that make the resulting code still feel like Perl, but also, don't know exactly, more robust perhaps?

So, to conclude:

  • Anyone with a Perl 6 implementation of MIME::Base64 ? Speak up before I create a monster :)
  • Anyone cares enough to take on the chunked transfer encoding support?

My silly twitter OAuth command line client

I had the need to test a new My Opera API module that is soon coming out. Since this module, to be named Net::MyOpera, is modeled exactly after Net::Twitter, I tried changing my example script replacing all Net::MyOpera occurrencies to Net::Twitter. And there you go, a Twitter command line client was born.

I know, there's plenty of them already, and as I said, I didn't really need one, but since it's there already, it's nice to have it. So I saved it into my ~/bin folder, and aliased to tw, so whenever I feel the urge to communicate stupid things to the universe, I can now do that. Ehm wait… :-)

The code is out on Github.

As always, there's a hidden (poor) excuse for this. And it's that I'm working to port this Twitter command line client to Perl6. OAuth support needs a good deal of modules that are not immediately available for Perl6, so it's going to be exciting.

First we're going to need are Digest::SHA1, and Digest::HMAC for the HMAC-SHA1 signatures. These modules are not impossible to write, except currently there's a problem using Parrot libraries from Perl6.

I'm trying to do the same for my Perl6 Digest::MD5 module, but I'm stumbling on the following error:

t/perl5-compat.t ... Null PMC access in find_method('signature')
  in 'Digest::MD5::md5_hex' at line 11:lib/Digest/MD5.pm
  in main program body at line 17:t/perl5-compat.t
t/perl5-compat.t ... Dubious, test returned 1 (wstat 256, 0x100)

I will need some help on this :-)

Perl6 hacking, grammars, Digest::MD5 and caffeine levels

I'll be brief. Need some sleep. :)

Perl 6 is here. Now. And there's an immense work waiting to be done: rewriting Perl5's CPAN. Ain't that easy? :)

Anyway, during last couple of weeks, I spent most of my spare time playing with Perl 6:

Grammars

My poor excuse to (try to) learn grammars was to build a Perl6 class that could parse a puppet module and build some documentation for it. Puppet itself uses a grammar to parse its modules, so I thought it wouldn't be impossible to port it to Perl6 and use it to parse puppet code.

Well, turns out it's not so easy, but at least I'm learning how grammars work and having fun.

Perl6 Digest::MD5

This is extremely fascinating, because it's touching the Parrot core. In Parrot, there's already a Digest::MD5 module, so all you have to do (but again, not so easy), is to write a Perl6 "wrapper" around the Parrot code.

And how do you do that? With PIR blocks. This stuff is great. Seriously. It's like going back to Assembly, in some sense(tm). Here's an example of this glue PIR code:

class Digest::MD5 {

    multi method md5_hex (Str $message) {

        pir::load_bytecode('Digest/MD5.pbc');

        my $md5_sum = Q:PIR {
            .local pmc md5sum, md5_sum_get
            md5sum = get_root_global ['parrot'; 'Digest'], '_md5sum'
            $P0 = find_lex '$message'
            $P1 = md5sum($P0)
            md5_sum_get = get_root_global ['parrot'; 'Digest'], '_md5_hex'
            %r = md5_sum_get($P1)
        };

        return $md5_sum;
    }

    multi method md5_hex (@message) {
        my Str $message = @message.join('');
        return Digest::MD5.md5_hex($message);
    }

}

Even if you don't understand Perl 6 or PIR, you can probably recognize a class definition, and polimorphic methods. md5_hex() is in fact defined twice:

  • multi method md5_hex (Str $message)
  • multi method md5_hex (@message)

You don't have to write polimorphic methods, but you can do it if you want. Yes, there's more than one way, and there always will be. I don't like dictators, even if they are benevolent, and Python code looks so flat and dull, seriously. There's no personality in Python code. Yes, sigils are great.

Digest::MD5 is also using alien technology (UFO).

Synopsis documents

Nothing fancy there, just improved the existing CSS. For an example, go read Synopsis 03 about operators.

Good night!

A working bogosort in Perl6

It's not even mine actually. @moritz helped me with that. I couldn't remember the syntax for the reduce operator (and it's [<op>]), so here it is, in its full wikipedia glory.

@list .= pick(*) until [<=] @list

Here's a somewhat more complete example, that you can actually run:

my @list = 1, 2, 7, 4, 3;
@list .= pick(*) until [<=] @list;
say @list.perl;

This will bogosort the array (wikipedia), that is, shuffle it until it's ordered. Look at the Java version for a nice comparison.

Here's the explanation:

my @list = 1, 2, 7, 4, 3;

Declares the list. You don't need parens around the elements. If that was a deck of cards, you could write it as:

my @list = <A 2 3 4 5 6 7 8 9 J Q K>

The angular brackets <> are the equivalent of qw() in Perl 5. Following line is quite dense:

@list .= pick(*) until [<=] @list;

This can be written also as:

while not [<=] @list {
  @list .= pick(*)
}

As you can see, you don't need parens around the while condition. The [<=] @list bit is IINM a reduce operator for a list. For example, to sum all elements of a list, you'd use:

my @num = 1, 2, 3, 4;
say [+] @num;

So the [<=] operator applied to a list compares each element with the next one, then the result with the next one. Assuming @list = 2, 4, 8, 5, that is how I think it runs:

2 <= 4 (true, go on, last evaluated value is 4)
4 <= 8 (true, go on, last value is 8)
8 <= 5 (false, stop here, return value is false)

So, if the array is ordered, the while/until block is skipped. If the array is not ordered, execute the following:

@list .= pick(*)

.= is an operator for in-place method call. Writing that is equivalent to:

@list = @list.pick(*)

That in Perl 5 would be written as:

$list = SomeListObject->new(1, 2, 3, 4, 5);
$list->pick(???);

Actually, there wouldn't be any Perl 5 direct equivalent code, since the magic bit here (the *, aka "whatever") it's not even there. "whatever" is magic for "anything that's applicable here, until there's an end to it", or at least this is my ignorant interpretation of it… :)

So in this context, pick() is a method that returns a random element from a list. Together with *, it means pick a random element from any of the original @list. Mmmh. But then it should be equivalent to:

@list .= pick(@list)

But it's not, so pick(*) knows the right thing to do…
And it's clear that I don't know how pick() works :)
Anyway…

BTW, this is runnable code. If you want to try it, clone the rakudo repository, and run perl Configure.pl --gen-parrot to build it. Have fun!