Tag Archives: rakudo

Perl6 LWP::Simple now uses the URI module for added awesome

My little LWP::Simple module for Perl 6 is growing beyond my control. I just merged a a pull request with a patch to remove all existing URL parsing to replace it with the new URI class.

That's fantastic, because:

  • it reduces the amount of code in LWP::Simple
  • the URI module is based on the actual grammar for URIs and IPv6 addresses. Plain and simple. It doesn't adhere to the standard. It is the standard. Isn't that amazing?

So, just clicky clicky and the pull request was merged. Running the test suite confirmed that everything still works fine passing all tests. Can it get much better than this? :)

If you want to have a look at Perl 6 code, modules.perl6.org is the place to go for inspiration. Have fun!

More Perl 6 development, String::CRC32

A couple of days were plenty enough to whip up a new Perl 6 module, String::CRC32.

This is a straight conversion from the Perl 5 CPAN version of String::CRC32. That one uses some C/XS code though. I wrote the equivalent in Perl 6. Was quite simple actually, once I learned a bit more on numeric bitwise operators.

The classic || and && for bitwise "OR" and "AND" are now written as:


Bitwise OR  '+|'
Bitwise AND '+&'
Bitwise XOR '+^'

So the following code:


$x &&= 0xFF;

becomes:


$x +&= 0xFF;

The reason behind this is that now there are not only numeric bitwise operators, but also boolean ones. You can read more in the official specification for Perl 6 operators.

You can download the code for the module on http://github.com/cosimo/perl6-string-crc32. Take a look at various other Perl 6 modules on modules.perl6.org.

Perl 6 LWP::Simple gets chunked transfers support

With this one I think we're basically done!

Perl6 LWP::Simple gets chunked transfers support. It's probably not excellent or universally working, but for the examples I could try and test, it's totally fine. If you find some URLs where it's broken, please tell me.

I also threw in the getstore() method to save URLs locally.

So, LWP::Simple for Perl 6 is here and it's working. It's not yet "complete" compared to the Perl 5 version, but now that I got the hard bits working, and the internals can perform a full http response parsing, I'll try to reach a 100% API compatibility with the Perl 5 one (where it makes sense).

Try it and let me know. Have fun!

Digest::MD5 for Perl 6 finally works!

It took me an awful long time, and lots of help from the folks on #parrot and #perl6 but in the end, it's done!

It needs a tiny patch to Parrot, but I believe it will be added to the next parrot release. I tried to documented the fixes to the code to help others that might have the same problems.

So now Digest::MD5 for Perl 6 works as good as the Perl 5 one. I'm too tired to say anything else :)

Good night!

LWP::Simple for Perl 6, now with (partial) BasicAuth support and getstore()

I just pushed out another update for the LWP::Simple module for Perl 6. This time, the main work was:

  • refactoring the code and adding unit tests for the URL parsing (that might even grow into a Perl 6 URI module
  • adding partial basic auth support. To be complete and working, it needs to base64 encode the user/password pair. Not implemented yet. I'll see if I get around to it, or if someone has done it already.
  • adding a getstore() method, that writes on to disk the downloaded content. Unfortunately that needs to strip the HTTP headers and undestand chunked transfers for it to be remotely useful.

It was nice to see the module grow in both functionality, code and unit tests coverage. I had to workaround a couple of problems I couldn't understand. I was extremely lazy and I didn't even look up Synopses, so I assume it's my fault. However.

The first is the use of .match() and ~~ to match against a regular expression. I found that the following code:

my $hostname = 'cosimo:eelst@faveclub.eelst.com';
if $hostname.match('^cosimo') {
    # Doesn't enter here
}

doesn't trigger a match. However, this other here:

my $hostname = 'cosimo:eelst@faveclub.eelst.com';
if $hostname ~~ /^cosimo/ {
    # Does match
}

And, in the same way, something similarly surprising. The following code correctly matches:

my $hostname = 'cosimo:eelst@faveclub.eelst.com';
if $hostname ~~ /^ .+ : .+ @ .+ $/ {
    say '(user:pass@host) matches';
} else {
    say '(user:pass@host) does not match';
}

but adding captures makes the same exact regex fail:

my $hostname = 'cosimo:eelst@faveclub.eelst.com';
if $hostname ~~ /^ (.+) : (.+)  @ (.+) $/ {
    say '(with captures) matches';
} else {
    say '(with captures) does not match';
}

There are also very nice things about programming in Perl 6 that are slowly sucking me in this fantastic language. This is part of a test script for LWP::Simple:

#
# Test the parse_url() method
#
use v6;
use Test;
use LWP:: Simple;

my @test = (
    { User-Agent => 'Opera/9.80 (WinNT; 6.0) Version/10.60' },
    "User-Agent: Opera/9.80 (WinNT; 6.0) Version/10.60rn",

    { Connection => 'close' },
    "Connection: closern",
);

for @test -> %headers, $expected_str {
    my $hdr_str = LWP:: Simple.stringify_headers(%headers);
    is($hdr_str, $expected_str, 'OK - ' ~ $hdr_str);
}

Note how in the for statement we can "extract" the hash and string from the @test array with:

for @test -> %headers, $expected_string {
    # Loop body
}

It's not a big deal, other languages have it, but Perl 6 is filled with this small niceties that make the resulting code still feel like Perl, but also, don't know exactly, more robust perhaps?

So, to conclude:

  • Anyone with a Perl 6 implementation of MIME::Base64 ? Speak up before I create a monster :)
  • Anyone cares enough to take on the chunked transfer encoding support?

My silly twitter OAuth command line client

I had the need to test a new My Opera API module that is soon coming out. Since this module, to be named Net::MyOpera, is modeled exactly after Net::Twitter, I tried changing my example script replacing all Net::MyOpera occurrencies to Net::Twitter. And there you go, a Twitter command line client was born.

I know, there's plenty of them already, and as I said, I didn't really need one, but since it's there already, it's nice to have it. So I saved it into my ~/bin folder, and aliased to tw, so whenever I feel the urge to communicate stupid things to the universe, I can now do that. Ehm wait… :-)

The code is out on Github.

As always, there's a hidden (poor) excuse for this. And it's that I'm working to port this Twitter command line client to Perl6. OAuth support needs a good deal of modules that are not immediately available for Perl6, so it's going to be exciting.

First we're going to need are Digest::SHA1, and Digest::HMAC for the HMAC-SHA1 signatures. These modules are not impossible to write, except currently there's a problem using Parrot libraries from Perl6.

I'm trying to do the same for my Perl6 Digest::MD5 module, but I'm stumbling on the following error:

t/perl5-compat.t ... Null PMC access in find_method('signature')
  in 'Digest::MD5::md5_hex' at line 11:lib/Digest/MD5.pm
  in main program body at line 17:t/perl5-compat.t
t/perl5-compat.t ... Dubious, test returned 1 (wstat 256, 0x100)

I will need some help on this :-)

Perl6 hacking, grammars, Digest::MD5 and caffeine levels

I'll be brief. Need some sleep. :)

Perl 6 is here. Now. And there's an immense work waiting to be done: rewriting Perl5's CPAN. Ain't that easy? :)

Anyway, during last couple of weeks, I spent most of my spare time playing with Perl 6:

Grammars

My poor excuse to (try to) learn grammars was to build a Perl6 class that could parse a puppet module and build some documentation for it. Puppet itself uses a grammar to parse its modules, so I thought it wouldn't be impossible to port it to Perl6 and use it to parse puppet code.

Well, turns out it's not so easy, but at least I'm learning how grammars work and having fun.

Perl6 Digest::MD5

This is extremely fascinating, because it's touching the Parrot core. In Parrot, there's already a Digest::MD5 module, so all you have to do (but again, not so easy), is to write a Perl6 "wrapper" around the Parrot code.

And how do you do that? With PIR blocks. This stuff is great. Seriously. It's like going back to Assembly, in some sense(tm). Here's an example of this glue PIR code:

class Digest::MD5 {

    multi method md5_hex (Str $message) {

        pir::load_bytecode('Digest/MD5.pbc');

        my $md5_sum = Q:PIR {
            .local pmc md5sum, md5_sum_get
            md5sum = get_root_global ['parrot'; 'Digest'], '_md5sum'
            $P0 = find_lex '$message'
            $P1 = md5sum($P0)
            md5_sum_get = get_root_global ['parrot'; 'Digest'], '_md5_hex'
            %r = md5_sum_get($P1)
        };

        return $md5_sum;
    }

    multi method md5_hex (@message) {
        my Str $message = @message.join('');
        return Digest::MD5.md5_hex($message);
    }

}

Even if you don't understand Perl 6 or PIR, you can probably recognize a class definition, and polimorphic methods. md5_hex() is in fact defined twice:

  • multi method md5_hex (Str $message)
  • multi method md5_hex (@message)

You don't have to write polimorphic methods, but you can do it if you want. Yes, there's more than one way, and there always will be. I don't like dictators, even if they are benevolent, and Python code looks so flat and dull, seriously. There's no personality in Python code. Yes, sigils are great.

Digest::MD5 is also using alien technology (UFO).

Synopsis documents

Nothing fancy there, just improved the existing CSS. For an example, go read Synopsis 03 about operators.

Good night!

A working bogosort in Perl6

It's not even mine actually. @moritz helped me with that. I couldn't remember the syntax for the reduce operator (and it's [<op>]), so here it is, in its full wikipedia glory.

@list .= pick(*) until [<=] @list

Here's a somewhat more complete example, that you can actually run:

my @list = 1, 2, 7, 4, 3;
@list .= pick(*) until [<=] @list;
say @list.perl;

This will bogosort the array (wikipedia), that is, shuffle it until it's ordered. Look at the Java version for a nice comparison.

Here's the explanation:

my @list = 1, 2, 7, 4, 3;

Declares the list. You don't need parens around the elements. If that was a deck of cards, you could write it as:

my @list = <A 2 3 4 5 6 7 8 9 J Q K>

The angular brackets <> are the equivalent of qw() in Perl 5. Following line is quite dense:

@list .= pick(*) until [<=] @list;

This can be written also as:

while not [<=] @list {
  @list .= pick(*)
}

As you can see, you don't need parens around the while condition. The [<=] @list bit is IINM a reduce operator for a list. For example, to sum all elements of a list, you'd use:

my @num = 1, 2, 3, 4;
say [+] @num;

So the [<=] operator applied to a list compares each element with the next one, then the result with the next one. Assuming @list = 2, 4, 8, 5, that is how I think it runs:

2 <= 4 (true, go on, last evaluated value is 4)
4 <= 8 (true, go on, last value is 8)
8 <= 5 (false, stop here, return value is false)

So, if the array is ordered, the while/until block is skipped. If the array is not ordered, execute the following:

@list .= pick(*)

.= is an operator for in-place method call. Writing that is equivalent to:

@list = @list.pick(*)

That in Perl 5 would be written as:

$list = SomeListObject->new(1, 2, 3, 4, 5);
$list->pick(???);

Actually, there wouldn't be any Perl 5 direct equivalent code, since the magic bit here (the *, aka "whatever") it's not even there. "whatever" is magic for "anything that's applicable here, until there's an end to it", or at least this is my ignorant interpretation of it… :)

So in this context, pick() is a method that returns a random element from a list. Together with *, it means pick a random element from any of the original @list. Mmmh. But then it should be equivalent to:

@list .= pick(@list)

But it's not, so pick(*) knows the right thing to do…
And it's clear that I don't know how pick() works :)
Anyway…

BTW, this is runnable code. If you want to try it, clone the rakudo repository, and run perl Configure.pl --gen-parrot to build it. Have fun!

Communities in Action 2010 in Oslo

Last Monday, 10th of May, the regular Oslo.pm (Oslo Perl Mongers) meeting was a little special. In fact, there wasn't any Oslo.pm meeting, but we went to a free, one evening mini conference sponsored by several norwegian companies: "Communities in Action".

The basic idea was to put together several different communities in the Oslo area, so the conference program was diverse and exciting. There were 7 different tracks. Among them, the most interesting for me, apart from Oslo.pm, were:

  • XP (extreme programming, agile, etc…)
  • scalabin (about the scala language)
  • OWASP
  • Oslo C++ users group

Other tracks were by the NNUG, Norwegian .NET users group and IASA, some norwegian association about something…

There were a couple of colleagues from Opera, and other guys I know from Oslo.pm, but being there alone, I couldn't follow more than one track, so I tried jumping a bit between different rooms. I found the XP talk a bit boring, so I settled on the Oslo.pm track. There was a talk on rakudo * by Karl Rune Nilsen and a talk about meta-object programming in Perl vs Ruby by Matt Trout.

Both talks rocked. I think I had attended a very similar rakudo talk before, but the Ruby one was entirely new. It was funny when Matt, just before starting, tried to attract Ruby folks screaming and shouting
throughout the hotel hall :-)

After the conference, a small group of us gathered and headed over to tilt, a nice pub/pinball place, where I ended up making the day record even if I suck at pinballs…

LWP::Simple for Perl 6

During the last Perl 6 hackaton in Oslo, I got to meet in person some very cool folks from the Perl Community, and we had a lot of brainstorming fun, as usual.

I went there with the ambitious (and out of my skills, most probably) goal of implementing a Socket interface for Perl 6. Talking to the various smart folks there, I realized that we didn't need to write that much, because Parrot, on which the current Perl 6 implementation is based on, already had sockets support.

After much nagging, I wrote a quick & dirty wrapper that mimicked the existing Perl5's IO::Socket library, that Carl and Martin improved. And on top of that, we were able to write a really tiny LWP::Simple-like class for Perl 6.

Now it's on up on github, go fetch it!, before it's too late :)