Perl Noise Aggregator

Perlbuzz news roundup for 2011-07-31

nobody@example.com (Andy Lester) — 2011-08-01T02:47:19Z

These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.

Hello, World in Place 2: InteractiveDebugger (blogs.perl.org)
Giving ack a little bit extra do-what-I-mean (shuttlethread.com)
Reducing documentation's technical debt (blogs.perl.org)
YAPC::NA attendees, please respond to survey (blogs.perl.org)
File::UStore, a UUID-based file storage abstractor (blogs.perl.org)
Protecting against cross-site scripting attacks with Template::AutoFilter (blogs.perl.org)
Screencast: How to contribute to a Perl module using vim and Github (szabgab.com)
Avoid unwanted bitwise operators with Perl::Critic (szabgab.com)
Just discovered rjray's Perl Module Monday feature: (dereferenced.com)
Booking.com donates $10K to support Perl 5 maintenance (news.perlfoundation.org)
Expect a flurry of Rakudo releases soon (rakudo.org)
cPanel is also sponsoring Perl 5 Maintenance Fund (news.perlfoundation.org)
YAPC::NA 2012 gets away from RTFM marketing (perlbuzz.com)
About the Perl 5 Core Maintenance Fund (perlfoundation.org)
Upgrading from Perl 5.8 to a less-than-newest version (blog.timbunce.org)
Using Acme::CPANAuthors::India to link with other authors (blogs.perl.org)
YAPC::Asia 2011 call for papers now open (yapcasia.org)
YAPC::Asia 2011 and the webapps behind it (blogs.perl.org)
Going to OSCON next week? Here are six tips for making the most of it with the least amount of hassle (petdance.com)
Portuguese Perl Workshop is Sep 22-23, 2011 (workshop.perl.pt)
Tool to move tickets from RT to Github (pythian.com)
Now you need LWP::Protocol::https (blogs.perl.org)
Perl program to back up Google contacts/calendar (preshweb.co.uk)
"What is MetaCPAN" screencast http://youtu.be/7N1lZY5E5gg (now with English subtitles)
Can't let #oscon pass without getting the scoop on cpangloss.com from (
"Perl 6 is solid enough now. Start thinking about porting modules. Start thinking about reinventing modules." -- Damian at #oscon

RT @pghphw @mjdominus will be teaching a course and giving a talk at PPW 2011. Watch for details.
Notes on Morphing Graph::Easy into Graph::Easy::Marpa (savage.net.au)
A new vim plugin to work with ack and iTerm: (vim.org)
Holy cow does a lot get done on MetaCPAN in a week (blogs.perl.org)
MetaCPAN::API gets updated (blogs.perl.org)
Rakudo Star 2011.07 released (rakudo.org)
Would you bring your kids to YAPC? (blog.yapcna.org)
Recollections of YAPC::NA 2011 (activestate.com)
Understanding autovivification (effectiveperlprogramming.com)

Perlbuzz news roundup for 2011-07-31

nobody@example.com (Andy Lester) — 2011-08-01T02:47:19Z

These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.

Hello, World in Place 2: InteractiveDebugger (blogs.perl.org)
Giving ack a little bit extra do-what-I-mean (shuttlethread.com)
Reducing documentation's technical debt (blogs.perl.org)
YAPC::NA attendees, please respond to survey (blogs.perl.org)
File::UStore, a UUID-based file storage abstractor (blogs.perl.org)
Protecting against cross-site scripting attacks with Template::AutoFilter (blogs.perl.org)
Screencast: How to contribute to a Perl module using vim and Github (szabgab.com)
Avoid unwanted bitwise operators with Perl::Critic (szabgab.com)
Just discovered rjray's Perl Module Monday feature: (dereferenced.com)
Booking.com donates $10K to support Perl 5 maintenance (news.perlfoundation.org)
Expect a flurry of Rakudo releases soon (rakudo.org)
cPanel is also sponsoring Perl 5 Maintenance Fund (news.perlfoundation.org)
YAPC::NA 2012 gets away from RTFM marketing (perlbuzz.com)
About the Perl 5 Core Maintenance Fund (perlfoundation.org)
Upgrading from Perl 5.8 to a less-than-newest version (blog.timbunce.org)
Using Acme::CPANAuthors::India to link with other authors (blogs.perl.org)
YAPC::Asia 2011 call for papers now open (yapcasia.org)
YAPC::Asia 2011 and the webapps behind it (blogs.perl.org)
Going to OSCON next week? Here are six tips for making the most of it with the least amount of hassle (petdance.com)
Portuguese Perl Workshop is Sep 22-23, 2011 (workshop.perl.pt)
Tool to move tickets from RT to Github (pythian.com)
Now you need LWP::Protocol::https (blogs.perl.org)
Perl program to back up Google contacts/calendar (preshweb.co.uk)
"What is MetaCPAN" screencast http://youtu.be/7N1lZY5E5gg (now with English subtitles)
Can't let #oscon pass without getting the scoop on cpangloss.com from (
"Perl 6 is solid enough now. Start thinking about porting modules. Start thinking about reinventing modules." -- Damian at #oscon

RT @pghphw @mjdominus will be teaching a course and giving a talk at PPW 2011. Watch for details.
Notes on Morphing Graph::Easy into Graph::Easy::Marpa (savage.net.au)
A new vim plugin to work with ack and iTerm: (vim.org)
Holy cow does a lot get done on MetaCPAN in a week (blogs.perl.org)
MetaCPAN::API gets updated (blogs.perl.org)
Rakudo Star 2011.07 released (rakudo.org)
Would you bring your kids to YAPC? (blog.yapcna.org)
Recollections of YAPC::NA 2011 (activestate.com)
Understanding autovivification (effectiveperlprogramming.com)

Weekly Perl News - first issue sent out

nobody@example.com (Gabor Szabo) — 2011-08-01T02:02:08Z

You are busy churning out code or managing the developers. You care about Perl but don't have time to go through tens and hundreds of articles and blog posts every day. You want to keep an eye on the development of Perl without drowning in a sea of blog posts. You need someone to point out the most important news and articles in the Perl World.

That's how I opened the first issue.[1]

With this newsletter I'll try to be your guide.

The newsletter won't provide original content. I am just going to send out links with a few words of comment to the items that I found most important.

If you are a Perl junkie like me, reading all the posts the Perl community and ecosystem produces, then this newsletter will just anger you. Why did I include X and not Y? You are welcome to let me know your opinion.

If you are like the silent majority, having little time but still wanting to be up to date then, I hope, you will find this newsletter useful.

My hope is that this newsletter will serve both "simple users" of Perl and managers who run teams of Perl developer.

Here you can see the first issue and you can sign up to the free Perl Weekly.

Enjoy!

[1] Well, except a typo that was quickly caught and reported by MDK, thanks!

Turn off autovivification when you don’t want it

nobody@example.com (brian d foy) — 2011-07-31T22:01:20Z

Autovivification, although a great feature, might bite you when you don’t expect it. I explained this feature in Understand autovivification, but I didn’t tell you that there’s a way to control it and even turn it off completely.

The autovivification pragma, which you can get from CPAN, lets you decide how autovivification works, or doesn’t work. As you saw in Understand autovivification, there are several ways that autovivification comes into play, specifically when you dereference an undefined value to:

fetch a value
store a value
test an undefined value with exists
delete an element

The pragma lets you control each of these independently.

The simplest use is simply unimport the module with no. This gets a bit tricky because this pragma is more interesting for what you disallow rather than enable:

no autovivification;

With no arguments, this turns off autovivification for fetching a value, or using delete or exists. This does not turn off autovivification for storing a value. You’re more likely to make mistakes with those first three, and most likely to get what you want with the last one.

None of these work, in that they don’t create any more of the data structure:

no autovivification;

my $cats;
delete $cats{'Buster'};
my $foo = $cats{'Buster'}; 

if( exists $cats{'Buster'} ) { 1 }

use Data::Dumper;
print Dumper( $cats );

You don’t get an error or a warning, but nothing happens to $cats, which is still undefined at the end. The pragma just leaves the undefined value as it is and your program continues. In those cases, you still get the answers that expect. When the element isn’t there, you still get undef when you try to access it. It stills returns false when you try exists and the element isn’t there after a exists. Your program continues without a warning.

If you want to know when you’ve skipped at possible autovivification, you can also tell autovivification to warn you, or if you’re really paranoid, kill your program.

Including the warn option gives you warnings. As with any import list, when you specify something you override all defaults. If you still want those defaults, you have to list them explicitly:

no autovivification qw(fetch delete exists warn);

use Data::Dumper;

my $cats;

delete $cats->{Buster};

print Dumper( $cats );

Now you get a warning:

Reference was vivified at auto.pl line 7.
$VAR1 = undef;

If you want to stop the program, you use strict instead of warn:

no autovivification qw(fetch delete exists strict);

use Data::Dumper;

my $cats;

delete $cats->{Buster};

print Dumper( $cats );

Your program stops at the point that Perl tries to autovivify something and gives you an file and line where autovivification stopped your program:

Reference vivification forbidden at auto.pl line 7.

When you store a value, however, autovivification doesn’t get in your way, even if you create a deep one:

no autovivification;

use Data::Dumper;

my $cats;

$cats->{Buster}{count} = 9;

print Dumper( $cats );

You still autovivified the structure:

$VAR1 = {
          'Buster' => {
                        'count' => 9
                      }
        };

If you want to turn off autovification for storing a variable, you have to explicitly disallow it. As before, you have to provide the other default options that you’d like to keep:

no autovivification qw(fetch delete exists store strict);

use Data::Dumper;

my $cats;

$cats->{Buster}++;

print Dumper( $cats );

Now, when you try to autovivify a hash reference to store a value in the Buster key, your program stops (because you are also using strict:

Reference vivification forbidden at auto.pl line 7.

Lexical scope

You can limit the effect of autovivification pragma to part of your program by using it in only in the scope where you need it. There are two ways that you can do this, depending on what you need.

First, you can turn off all autovivification for the entire file (which is itself a scope, see Know what creates a scope), but re-enable it for the bits where you need it:

no autovivification qw(fetch delete exists store strict);

# can't do it in here

{
use autovivification qw(store);

...; # allow some of it in here
}

# can't do it in here, either

Second, you can leave Perl’s normal autovivification features in place for the entire file, but turn it off in the sensitive parts of the program, just as in Item 100. Use lexical warnings to selectively turn on or off complaints (but in this case, engaging safety features):

...; # program with possible autovivification

{
# can't do it in here
no autovivification qw(fetch delete exists store strict);

}

...; # program with possible autovivification

Things to remember

The autovivification pragma lets you disable Perl’s automatic data structure creation
The defaults do not prevent you from assigning to an autovivified reference
You can enable warnings or errors to catch autovivification events

GSoC – The podparser branch has landed

nobody@example.com (ttjjss) — 2011-07-31T19:14:11Z

Just a few minutes ago, with a total count of 28 files changed, 1627 insertions and 41 deletions, the podparser branch in which I’ve been doing my GSoC work has been merged into nom, the current development branch of Rakudo and the soon-to-be master. So, what do we get?

Pod is now parsed, not ignored. The whole parse tree is available in runtime in the $=POD variable
A basic support for documenting classes and subroutines with #= comments, and the .WHY method on each object returning the attached documentation
A –doc switch, a not-really-replacement for perldoc
A simple Pod::To::Text output generator
and more, of course :)

So, how does it look like in action?

$=POD

All the Pod we have written in our code is now available in the $=POD variable. Example?

=begin pod
Some documentation
=end pod

say $=POD[0].content[0].content;

Results in

Some documentation

That’s not very useful per se, so how about this one:

=begin pod

=head1 NAME

A basic pod document

=head2 Running Perl programs

To run a Perl program from the Unix command line:

    perl progname.pl

=head2 Things on my desk

There are the following things on my desk right now:

=item A cup of tea
=item A couple of pens
=item A stereo
=item A couple of books
=item A laptop, obviously

=end pod

DOC INIT {
    use Pod::To::Text;
    say pod2text($=POD);
    exit;
}

say "I'm just a simple program";

Now what’s the DOC INIT thing? Let’s see. How about we run the above program:

$ perl6 foo.pl
I'm just a simple program

No suprises. Let’s introduce the –doc switch then:

$ perl6 --doc foo.pl
NAME

A basic pod document

  Running Perl programs

To run a Perl program from the Unix command line:

    perl progname.pl

  Things on my desk

There are the following things on my desk right now:

 * A cup of tea
 * A couple of pens
 * A stereo
 * A couple of books
 * A laptop, obviously

That’s right. The DOC blocks are executed only when the –doc command line option has been given. At the moment you have to write them yourself, but maybe even in the nearest few days there will be a default DOC INIT block doing What You mean all by itself. There we go, a perldoc-alike built-in :)

WHY

That’s probably the biggest killer feature in the whole project. Although it’s not yet fully implemented in Rakudo (suprise segfaults here and there, don’t worry, they’re not permanent :)), it looks pretty much like this:

#= our zebra class
class Zebra {
...
}

say Zebra.WHY # prints: "our zebra class"

Yes yes, a documentation inspection in runtime. See the potential?

# what was that sub again?
&sort.WHY.say # get the documentation for the sort() builtin

That opens a way for lots of awesome userspace tools too.

So, what’s still not quite there?

.WHY (some cases)
Formatting Codes (C<code>, B<bold> etc)
DOC use (that can be written as DOC INIT { use … } for now)
Default DOC INIT (as mentioned above)
A Pod::Parser as a standalone module

With that done, my GSoC project will be finished, and I’ll hopefully have time to hit some things I was planning to poke in some spare time, for example:

HTML output generator
Extending the parser so Synopse 26 can be fully parsed
Other fancy userspace tools; I don’t want to spoil the suprise :)

What can I say? Pull it, compile it, play with it, report bugs and have the appopriate amount of fun :)

How to integrate a Mailman signup with your web site?

nobody@example.com (Gabor Szabo) — 2011-07-31T10:49:17Z

There are many people who prefer to read selected news items over following 10-30 sources of information. Lot's of us prefer e-mail alerts over RSS feeds. During the week-end I setup two newsletters.

One is called Perl Weekly sending out a list of the most important news items and articles regarding Perl.

The other one is an announcement list for my own website. This is for people who would like to get notified when I upload a new post.

For now they are both Mailman based mailing lists as that was the easiest to setup without giving up control to a mailing company.

The problem with Mailman is that I need to direct people to a separate subscribe page that - at least by default - looks very different from my web site:

It could be customized but it still means people need to do extra steps before they can sign up to the newsletter.

The first thing I did for better integration is to embed the HTML form that is on the mailman page into my web site but only show one field for the e-mail address.

This is the HTML:

  <form action="http://mail.szabgab.com/mailman/subscribe/update" method="POST">
  E-mail: <input name="email" /><input type="submit" value="Sign Me Up!" />
  </form>

And this is how it looks:

If you try that form an click on the button you will see that the result page is still what Mailman provides.

What about changing that too?

I changed the form so the form will have a class called subscribe <form class="subscribe" and added the following JQuery code:

 $(document).ready(function() {
   $("form.subscribe").submit(function() {
     var url = $(this).attr('action') + '?' + $(this).serialize();
     $.get(url, function(r) {
         window.location = "http://szabgab.com/thankyou.html";
     });
     return false;
   });
 });

Now when the user clicks on the Sign Me Up! button, the above script will serialzie the data in the form and send a GET request to the mailman subscribe form. When that replies the JQuery code will load the thank you page with further instructions.

See how it works on my site:

It still leaves the ugly e-mail that Mailman sends out and the verification form but at least the first step is more integrated in my website.

I am not a JQuery or HTML expert. I am probably better described as a newbie for life. There might be much better ways to make this happen.

How do you integrate e-mail signup to your website?

AutoLoader Considered Harmful?

nobody@example.com (rjray) — 2011-07-30T18:31:51Z

Lately, I’ve been doing a lot of work on some of my modules using the fine Devel::Cover package. And for about two years now, I’ve also been using Perl::Critic almost religiously (to the point where I’ve introduced it around $DAY_JOB, and even built them a snazzy web interface for it). But these two share a common failing: both fall down when they encounter a module that uses some form of external-file-based auto-loading.

First, let me explain what I’m referring to, for those who might not be familiar with auto-loading. Perl has this nifty feature called auto-loading, in which a function that doesn’t exist when it is called might have the chance to spring into life dynamically. Perl will look for and invoke a special subroutine called AUTOLOAD in your current namespace, giving it the name of the subroutine that was just requested in the special variable $AUTOLOAD (and passing that call’s args list in @_). You can get the routine by any means that Perl allows: you can create it on the fly, you can on-demand load a module and add it to your @ISA search path, etc. One of the most clever uses I saw early on in Perl 5′s life, was Lincoln Stein’s CGI module. In it, he used auto-loading to define all the HTML shortcut routines from a template, eval‘ing them on the fly as they were each first called.

In short, this amounts to Perl’s version of C’s dynamic loading, deferring the loading and compilation of code (well, in the C world you were just deferring the loading and linking, it was already compiled) until you needed it. Into to this we introduce the core modules AutoLoader and AutoSplit. AutoLoader provides for you an AUTOLOAD method that uses the name of the requested routine and the package it is in to look for a file by that name under a directory hierarchy within your @INC path. To compliment this, AutoSplit is used by the build/install process (be it ExtUtils::MakeMaker or Module::Build, etc.) to look at a module, determine if it uses AutoLoader, and splits out the routines that were defined as dynamically-loading into these per-name files. In fact, I got my break into the world of writing about Perl by writing an article explaining these two modules for the Perl Journal (alas, I cannot find a link to the article online). I also made my first contributions to Perl’s core by fleshing out the docs for these two, as I had been using them at my then-job to make a 20,000-line module more manageable.

But! I’m clearly not hear to sing the praises of AutoLoader, or I would have titled this post something else entirely. I don’t necessarily come here to bury Caesar in lieu of praising him, but introducing AutoLoader (or SelfLoader, for that matter) gums up the works when you use either of Devel::Cover or Perl::Critic. And while Devel::Cover might be able to fix this (I’ve filed a bug, but I don’t know the internals of D::C so I don’t know if it is addressable), Perl::Critic definitely won’t be able to.

See, AutoLoader/AutoSplit (and SelfLoader, though it uses a slightly different model) work by having you put the code you want to delay loading after the __END__ token in your module (__DATA__ in the case of SelfLoader). AutoSplit can then find it and split it out into individual files, and as the compiler stops compiling once it hits that token nothing at that point onward contributes to the start-up compilation phase. But while this successfully hides the code from the compiler, it also very successfully hides the code from Perl::Critic and Devel::Cover as well.

P::C is built on the PPI package by Adam Kennedy, and in terms of a Perl module as a document anything after __END__ is simply not code and not going to be processed as such. So your code that uses AutoLoader is not being fully analyzed by P::C. I had released a version of RPC::XML that I thought was critic-clean, without having thought about this caveat. When I later went and commented-out the “use AutoLoader” and “__END__” lines, I found a whole new set of violations to clear up.

Likewise, I had a similar problem with Devel::Cover. Only slightly worse, because I assumed that they had already dealt with this problem. The code gets loaded just like any other Perl module, so I assumed that when it loaded it then got instrumented. And the code uses “#line” directives to associate the code with the correct line in the original *.pm file, so I thought that they would be able to “translate” the line numbers of coverage statistics back to the originating file. But alas, no— code after __END__ is just as hidden from D::C as it is from P::C.

Which leaves me wondering whether AutoLoader (and __END__-based auto-loading in general) might not have run its course of usefulness. I’m left wondering what sort of hoops the authors of syntax-highlighting for Emacs had to jump through, to determine that content after __END__ was actually code rather than data, and to highlight it as such (vim doesn’t, it leaves it in the same font-face as data). Editors aside, here are two extremely useful development tools for Perl programmers (remember, I’m practically religious about P::C, spreading the gospel to my workplace and anywhere else I can), and both are hampered by the use of auto-loading. Given the leaps in memory and CPU over the last 15 years or so since I wrote that first article, do we really need AutoLoader anymore? I mean, I’m not saying do away with auto-loading itself, it has a truly useful place in Perl. I’m just not so sure whether we need AutoLoader (or SelfLoader) like we used to.

Perl 6 Compiler Feature Matrix

nobody@example.com (Moritz Lenz)

We now have a nice table that tells you which Perl 6 compiler implements what..

Such a thing was long overdue. When the topic came up in the past, people have suggested mostly automated solutions that compared test coverage of compiles to generate such a table. Nothing came out of it, it would have been a rather large endeavor. Now Eevee blogged about the lack of some easy overview that tells you what is implemented in Rakudo., and I thought it was time to tackle the problem.

Instead of some advanced automated system, we now have a simple text file, and a short perl script that converts it to a HTML page.

I'd like to thanks Will Coleda, Patrick Michaud and Stefan O'Rear for their contributions, and encourage everybody to keep the data up to date.

How fast is Rakudo's "nom" branch?

nobody@example.com (Moritz Lenz)

Nearly one year ago, the Rakudo Perl 6 developers proudly released the first Rakudo Star, a distribution aimed at showing the world what Perl 6 can look like, and in turn get feedback from more early adaptors.

And feedback we got. While the overall response was very positive, people had one main concern: it was too slow. That didn't come as a surprise, considering that we had focused on features first. Now it was time to change that, and work on massive performance improvements.

That is easier said than done. One of the reasons is that Rakudo is tightly coupled to the parrot virtual machine, but there is a lot of mismatch between the two. For example parrot provides multi dispatch built-in, but not quite with the semantics that Perl 6 needs. Same for parameter binding, objects and a number of other areas.

In the following year, parrot got a new, faster garbage collector, and Jonathan Worthington came up with a cache for type checks at routine call time.

This sped up this simple mandelbrot fractal generator at size 201x201 from 18 minutes to 16 minutes 14 seconds. Actually the speedup was better than that, but we paid a performance penalty for new features, bug fixes and parrot performance regressions.

But it was clear that more substantial improvements where needed. One of the most promising candidates for speedups is a complete redesign of the object model, resulting in the "nom" (new object model) branch of Rakudo. Additionally to providing much more well suited OO primitives than parrot can offer right now, it also allows to share more information between compile time and run time, making a lot of optimizations possible.

Yesterday I sped up some operations on Complex numbers, and implemented a built-in that was missing to run the mandelbrot script. And today I timed it: 3 Minutes. From originally 18 Minutes.

Now that's a speedup by more than a factor of 5. I'm not sure if it will extend to other operators, but it sure is encouraging.

And that's without the optimizations that will now be possible, for example inlining operators. So after a literally slow start, Rakudo Perl 6 has a bright and fast future ahead. And it's already here, just not evenly distributed.

Thoughts on masak's Perl 6 Coding Contest

nobody@example.com (Moritz Lenz)

Masak's Perl 6 Coding Contest (short p6cc) is now in its final stage - the public commentary of solutions. So far masak has commented on p1, p2, and p3. I enjoyed the reviews and explanations so far, and look forward to more (except to the p4 review, because I botched the solution to this one).

The reviews made a good read, and here are only a few very minor points that I find worth mentioning. No criticism intended (neither to the author nor the reviewer).

Matthias' p1 solution uses side effects in subroutine named to-string. I would avoid that in "production" code (for whatever I might mean with that word...), since it's not what I would expect from the name. Instead of a counter, the array indexes could be used to identify which matrix to stringify.
Masak's review of fox' p1 solution contains the question I wonder why the @items array deserved a plural but the @matrix only a singular.... I'd say that's because @matrix holds one matrix, but @items holds many items. Speaking of which, I don't like the name @matrix - it describe a structure rather than the contents. The structure becomes pretty obvious through the access, but what is in the matrix?
After viewing my own p1 submission without syntax hilighting, I wonder why I chose so many double blank lines. My vim color scheme uses a dark blue for comments, which means that comment blocks need more visual distance from code, in my personal opinion. But when sharing code, I shouldn't make layout decisions based on non-shared syntax hilighting.
colomon's p2 submission uses complex numbers for coordinates. I considered that myself, and there's nothing wrong with it. Just strange that he then reinvented subtraction of two complex numbers in sub lines-intersect. Maybe he didn't want to come up with names for the intermediate results, $slope and $axes-intercept might have been viable ones.
Matthias could have simplified comb: /<&number>/ to just comb: &number -- a regex is just a callable, and when it's explicit in the current scope, you can just use a hard reference to it.
colomon's p3 solution could, as far as I can tell, replace all regexes with tokens - less backtracking, fewer surprises.
Reviewing my own p3 solution, masak asked I wonder what stopped moritz from, rather than doing $_ && .including on line 39, doing .?including instead. Either he considered that too cute, or he didn't consider it. I did consider it, but I decided against it, because the two things are subtly different. Mine only calls the method if the topic is true, whereas .?including always tries the call - even on a type object, where accessing an attribute leads to a fatal error. I think in my code that case doesn't show up, but it did appear during debugging in earlier versions.

Introducing my new project: Quelology organizes books

nobody@example.com (Moritz Lenz)

For about half a year I've been working on a website called quelology, which collects book series and translations.

It is intended to answer questions of the form: I've now read "Harry Potter and the Order of the Phoenix", which is the next book in that series? or What's the name of the French translation of that book?

The website and data mining behind it are written in Perl, and it is based on book meta data by isfdb, amazon and worldcat.

I'm working on importing data from more sources, next up will be the Swedish National Library.

After completing the data mining stage, I'll add an interfaces that allows the visitor to edit the book, series and translations data, so that users can extend the data body.

A Foray into Perl 5 land

nobody@example.com (Moritz Lenz)

Usually I use Perl 6 as my programming language of choice. But about a week ago I started to plan and code an application for which the richness of the Perl 5 ecosystem outmatch the design weaknesses of Perl 5, compared to Perl 6.

I have quite a bit of Perl 5 experience too, but so far I mostly used it for smallish tasks, and didn't have too much use for many frameworks.

Here is a list of some tools I used, and my experience with them. I hope that my experiences can help the Perl 5 community to improve some edges here and there, and serve as an inspiration for Perl 6 modules authors what problems to approach.

cpanminus - painless module installation.

cpanminus gives easy access to the wealth of the CPAN. The module installation worked perfectly without any configuration or interaction.

Just when a XML parser had a dependency on an external C header file (expat.h), a module installation failed. Since Perl modules don't have a way to declare external dependencies, that's probably as good as a module installer can work. Kudos! The only improvement I can think of is directly showing the error message, instead of having it to dig up from a log file. I don't know if there's an easy way to automate that though.

DBIx::Class for the database

I store data in a postgresql database. From lowest to highest abstraction, the Perl modules involved are DBD::Pg (the database driver), DBI (the driver-independent database interface) and DBIx::Class, an object relational mapper.

Postgres itself is fantastic, and DBD::Pg and DBI look rock solid to me. I've worked with DBIx::Class before, and liked it. Upon rereading the documentation, I found that since my last usage the recommended usage pattern has changed. Writing result classes into the Result::MyClass namespace and result set into ResultSet::MyClass has made the result sets more accessible. Since they are a key feature of DBIx-Class, I welcome this change, and adopted it very naturally.

A small left-over from the previous scheme made the transition a tad harder than it had to, but upon reporting it on the #dbix-class IRC channel (on irc.perl.org), I immediately got commit access, and fixed it in the source.

Since I deal with trees, I was happy to discover a DBIx::Class plugin for nested sets. I was less happy to discover that it broke basic object creation, and had a bug that prevented merging of trees, a feature advertised in the documentation. Luckily both were very easy to patch, the patches now live in the bug tracker. I hope the maintainer applies them soon.

The nested set extension comes with a good test suite, but it seems it hasn't had much real world usage. I think that with some more usage (and maybe a few more bug fixes), it'll turn into a very good module.

Mojolicious for the web frontend

While waiting for the Catalyst dependencies to install, I decided on a whim to try out Mojolicious, a new-ish web framework. Or more precisely Mojolicious::Lite, a simplified API that lets you keep the whole application in a single file.

Now there were a lot of small, rough spots in Mojolicious. The community on IRC was very helpful, and asked me to record my findings on a wiki page - which I did.

What really bugs me about Mojolicious is that the built-in template system produces very incomprehensible error messages. It uses a mixture of verbatim text and perl code, separated by tags with various semantics (for example tags that just execute code, those that execute and insert the result, optionally with HTML/XML escaping).

Unfortunately that means that the code you write differs from the code that perl executes, which makes the error messages pretty useless.

My first suggestion to improve that situation is to display the generated code in the error message, in addition to the template code (and make the generated code as simple as possible.

If the generated code is non-trivial, it would help to add some markup to distinguish the user-written code from the code that the generator adds around it. I have no idea how easy or hard that would be to implement, though.

Conclusions

The Perl 5 modules were mostly very easy to use, and the corresponding communities very attentive and helpful.

If there's something the authors can learn from Perl 6, then it's a love for better error messages.

The Perl 6 world can aspire for such a rich and easy-to-use module system.

Perl 6 in 2010

nobody@example.com (Moritz Lenz)

2010 has been a busy year for the Perl 6 developers, and came with a noticeable distribution release, many new modules and much fun for the people involved. Here's a short, subjective reflection of this year's Perl 6 events.

Specification

While some specification changes had substantial impact on the compiler writers (and were usually in turn triggered by their worries), the user mostly saw maturing of experimental features, smoothed APIs and some few new features.

Lists, lists, lists

Just as last year, there were a lot of discussions on how lists and related types worked. Much of it was driven by the efforts to implement proper lazy lists in Rakudo.

The result is a much more solid list model, which uses immutable iterators under the hood - a fact that is hidden quite well from the user.

The sequence operator (previously known as "series operator") is a powerful tool for creating lazy lists. It has been extensively refactored to solve problems both with its implementation and usage. It now takes an optional closure left of the ... operator, and a limit that terminates the sequence if it matches true:

# Lazy list of Fibonacci numbers up to (but excluding) 100:
my @fib := 1, 2, *+* ...^ * >= 100;

Date and Time

After several iterations, excessive bikeshedding and serious hacking, Perl 6 now has built-in classes for handling times and dates. They are inspired by the DateTime and Date::Simple Perl 5 modules. The biggest difference is probably that DateTime objects are immutable in Perl 6.

This part of the specification is implemented completely in Rakudo.

Zip meta operator

The Z zip operator can now also act as a meta operator. Thus an easy way to add two lists pairwise and lazily is now

my @sum = @list1 Z+ @list2;

Other changes

The .pick method, which randomly takes one or more elements from lists and hashes, has been split up into two: @a.pick(3) returns 3 distinct, random items from array @a, while @a.roll(3) does three independent random choices, resulting in possible duplicates in the result list.
The scoping of lexical multi routines and their protos has been clarified and overhauled (see this discussion of what was wrong previously, and the resolution).
The numeric roles, buffers and Whatever type have received significant updates

Community

In 2010 we had a remarkable influx of friendly, interested, skilled and enthused newcomers to the Perl 6 community. This is the result of increased marketing outside the Perl community, well publicized releases, great technology and a friendly community.

Community expansion

Two challenges or contests have been announced this year.

Moritz Lenz published a series of "weekly" challenges, guided tasks to implement something that the Perl 6 community needs: A website for the ecosystem, a feature in a compiler and other small things that could be tackled without much prior knowledge.

The overall response was very good, and several people used it as a quick start into the Perl 6 community, and stayed.

Towards the end of the year, Carl Mäsak announced his Perl 6 coding contest. The submitter with the best solutions to five well known programming tasks is to win 100€ worth of books.

Another far-reaching project was the Perl 6 advent calendar for 2010, which attracted more than forty thousand visitors .

In an attempt to make Perl 6 compilers easier available to the masses, John Harrison implemented a web frontend to the Rakudo REPL and made it available at try.rakudo.org.

Conferences

There was a big Perl 6 hackathon (though we had more discussions than hacking) at YAPC::EU 2010 in Pisa. Many Perl 6 contributors, compiler writers and users met and discussed pressing topics in the realms of specification, implementation roadmap, measuring progress and community management. See the meeting notes for details.

Of course there were also some Perl 6 talks at YAPC::EU, many of which seemed well received by the audience.

Perl 6 talks were also held at the Netherlands Perl Workshop, YAPC::Russia, Norwegian Unix User Group and OSDC France, as well as many other conferences which the author forgot :-).

Repository changes

Due to neglected maintenance, the Pugs repository had to be shut down. It has been migrated to git, and split up into several repositories under the perl6 organization on github. Notable parts include:

roast: the Perl 6 test suite
specs: the specification
perl6.org: the main Perl 6 website
modules.perl6.org: the Perl 6 modules website
ecosystem: the module list repository
mu: the remnants of the old pugs repository

While the transition was mostly ad-hoc and not really planned for, most of the resulting confusion could be resolved fairly quickly.

Module ecosystem

While we still lack a proper module distribution system, we now have a website of known Perl 6 modules and a module installer.

But most importantly the number of modules and module authors is steadily increasing (82 known Perl 6 modules at the time of writing, compared to 45 last year). While we still lack the wealth of the Perl 5 ecosystem, there are now database modules, HTTP client and server modules, serialization, file handling tools and so on.

Implementations

Rakudo

Most importantly, this year saw the first Rakudo Star release. Rakudo star is a distribution of the Rakudo compiler, modules and documentation. While it is still a kind of preview release, some few production usages of the Rakudo Perl 6 compiler and distribution have been spotted in the wake of this release.

Also a good part of the Rakudo code based has been replaced during a major refactor, which bases Rakudo on top of a new grammar engine.

Major improvements to the compiler include

an implementation of lazy lists
lexical classes and roles
Perl 6 level stack traces
much more solid meta object model, which allows the user to create and modify classes programmatically at run time
implementation of the s/search/replace/ and s[search] = replace() syntactic features, along with several new regex adverbs and variable interpolation into regexes
improved interpolation in double-quoted strings: array and hash variables now properly interpolate when the expression ends in a bracketing construct
an improved read-evaluation-print loop, which now remembers variables from previous lines, and also automatically prints the result if no output was produced
multi level array and hash autovivification
binding and read-only binding of variables
a solid implementation of the DateTime and Date classes
MAIN and USAGE subroutines
the magic $*ARGFILES file handle and get (comparable to while (<>) { ... } in Perl 5)
an implementation of basic feed operators

During YAPC::EU the Rakudo contributors decided to target multiple virtual machines: besides the current parrot backend we want to support at least the CLR (.NET).

With this goal in mind, and the need for major performance improvements, Jonathan Worthington prototyped a new, efficient meta object model for parrot in C#, and used that as a base for the new CLR backend. He got help from Matthew Wilson, and Martin Berends started porting the effort to the JVM. Jonathan explained some of his work nicely on the 6guts blog.

In 2011 we will likely see a port of the meta object implementation to parrot, and the beginnings of a Rakudo port to the CLR and JVM.

Niecza

In June, Stefan O'Rear started taking notes on how to compile Perl 6 to the Common Language Runtime (CLR). In November he announced the Niecza Perl 6 compiler, focused on the generation of efficient code.

It already has an impressive list of features, including proper Longest Token Matching, a feature of regexes and grammars that no other Perl 6 compiler has implemented so far.

Summary

2010 was a very rewarding year for the Perl 6 community. With Rakudo there was a compiler available, with which small and medium scale projects can be fun to write. Niecza is quickly catching up.

People experiment with Perl 6, join the community and bring fresh ideas. There is still a long road ahead of us, but the author feels that this road is getting broader and more accessible with each step.

An offer for software developers: free IRC logging

nobody@example.com (Moritz Lenz)

The Perl 6 developers communicate a lot through IRC. Some of the conversation is still valuable later on, so we have public IRC logs.

The software powering these logs was written especially for #perl6, but works fine for other channels too. Among the other users are TestML, CDK (Chemistry Development Kit), darcs, mojo, Padre, the Perl IDE, Parrot and Rosetta Code.

If you are also developing software, and would like public logs for your channel (either on freenode or irc.perl.org; other servers might be added on demand), feel free to contact me (moritz on freenode, or per email: moritz at faui2k3.org)

Features include: linking to individual lines, permanent URLs and volatile URLs for the current day, automatic linking of URLs and readable color schemes.

A current limitation is that you can't have two channels with the same name from different networks, in case of conflict "first come, first served" holds.

First Grant Report: Structured Error Messages

nobody@example.com (Moritz Lenz)

My Hague Grant proposal for designing, implementing and testing structured error messages for Perl 6 has been acceepted, and I've started my work on it in my copious free time.

Before the grant started I've unified the error messages of several compilers to use "Cannot" instead of a wild mixture of "Cannot", "Can not" and "Can't".

In the past week I created a repository for the initial work on the error message spec, and added a list of existing error messages across different compilers, and some notes regarding the upcoming spec.

So far I've outlined some thoughts about separation of concerns, classification of the error messages, testing error messages for certain properties, and calling syntax for die() and fail().

Any constructive feedback on it is very welcome.

Thanks go to Ian Hague and The Perl Foundation for supporting my work financially.

Perl 6 notes from February 2011

nobody@example.com (Moritz Lenz)

Lately real life has prevented me from blogging, so here are just a few random notes from the Perl 6 developers:

The Perl bug tracker now has tags testneeded and testcommitted, which can mark tests that need or have tests in the spectest suite. Since the URLs for querying these tags are unwieldy and non-obvious, I've created some aliases: http://rakudo.de/testneeded and http://rakudo.de/testcommitted.

Development of the new nqp and rakudo-on-the-new-object-model is progressing nicely. I had some fun porting some PIR code to NQP, and writing some new code. Most interesting to read is the source of the new meta model, much of which is written in a subset of Perl 6 (so quite readable, if you happen to know Perl 6. For example you can see how the method resolution order for multiple inheritance is calculated.

There is a parrot branch that adds a generation garbage collector to parrot. Its release is planned for shortly after the 3.1.0 release due tomorrow. Initial benchmarks show that Rakudo is between 25% and 30% faster on that parrot, as measured by a spectest run. I very much look forward to having that in the parrot main line.

Writing code for niecza is quite a nice experience. It still has a big startup cost, but then runs much faster than rakudo (at least it feels that way). There are still lots of features missing (for example non-integer number literals), but feature requests are usually implement quite quickly.

PVC - Perl 6 Vocabulary Coach

nobody@example.com (Moritz Lenz)

Hej!

I'm trying to learn Norwegian these days, which is eating many of my round tuits that were previously available for Perl 6 hacking.

On the other hand I'm using Perl 6 to help me learning; I've written PVC, the Perl 6 Vocabulary Coach. It uses a word list, and keeps track of how many times in a row I entered the Norwegian translation of a word correctly.

The better I know a word, the less frequently I'm being asked for it. Not quite the same as spaced repetition learning, but still it feels quite well.

The Real World Strikes Back - or why you shouldn't forbid stuff just because you think it's wrong

nobody@example.com (Moritz Lenz)

tl;dr;; version: arbitrary API limitations do more harm than good, even if meant well in the first place.

Most advanced libraries that help you with date calculations have separate data types for a point in time, and a time span. That's because those two concepts actually have different semantics. It doesn't make sense to add two points in time, but it does make sense to add two durations, or add a duration to a point in time.

In Perl 6, those two types are called Instant and Duration. And obviously it makes sense to multiply a Duration with a number, but it doesn't make sense to multiply two Durations, or take a power of a Duration. Right?

That's the opinion that both the Perl 6 specification and the implementation in Rakudo reflected. Until recently, when somebody started to actually use it.

That's when the real world struck back. Carl Mäsak did some timings, and then calculated averages and standard deviations. And for calculating standard deviations, you actually have to square those durations, add them up, and then calculate the square root.

So this perfectly legitimate use case shows that multiplication (and also exponentiation) are perfectly fine operations on Durations. Likewise the current specification disallows division of two Durations. Why? It's perfectly fine to ask for the ratio of two time spans. How much longer (or shorter) are my meetings with my current boss, compared to those with my previous boss? That's the question that Duration / Duration answers.

So, the real world taught me that putting restrictions on the allowed operations is a bad idea. It was meant well, it was supposed to catch operations that don't made sense to the designer, and presumably would catch some error that a confused beginner might make. But in the end it did more harm than good.

Currently the Duration class stores a number, and re-dispatches all operations to that number, forbidding some of them. Having learned my lesson, I suggest we get rid of it, and have Instant - Instant return that number straight away. If some day we want to add functionality to Duration, we can still create that class as a subclass of the number.