Patrick Michaud is truly earning the Mozilla Foundation’s Perl 6 on Parrot Grant. He’s just published a Perl 6 on Parrot Roadmap for December 4, 2007, which describes the progress the project has made in the past couple of weeks.

In particular, he and Jerry Gay just switched the Perl 6 compiler in languages/perl6/ from the old compiler toolkit to the new compiler toolkit. This sounds like a small thing, but it’s actually a very large step.

The previous version of the tools used TGE (in Parrot at compilers/tge/) directly. TGE, in Parrot-land, is the Tree Grammar Engine. TGE processes attribute grammars, which describe transformation rules between two types of trees. For example, all of the compiler tools use PGE (see compilers/pge/) to parse source code into a parse tree. The compiler tools use one or more TGE transformation to turn the parse tree into something that Parrot can execute natively.

Previous versions of the Perl 6 compiler used PIR (Parrot’s native programming language, a sort of high-level assembly) to specify the TGE transformation. The topmost rule, which applies to Perl 6 programs as a whole, looked something like:

transform past (ROOT) :language('PIR') {
    node = node['program']
    .local pmc outerblock, past
    outerblock = get_global '$?BLOCK'

    past = new 'PAST::Block'
    past.'init_perl6'('node' => node, 'name' => 'anon')
    past.'init_lexicals'()

    set_global '$?BLOCK', past

    .local pmc cnode, cpast
    cnode = node['statement_list']
    cpast = tree.'get'('past', cnode, 'Perl6::Grammar::statement_list')

    past.'push'(cpast)
    set_global '$?BLOCK', outerblock

    .return (past)
}

That’s not awful as far as assembly language goes; it has subroutines, calling conventions, objects, method calls, pairs, aggregate variables, namespaces, and labeled registers, for example. It’s just one rule in a 700-line file though.

Compare that to the corresponding rule in the new compiler tools. This version uses NQP (compilers/nqp/), which is a minimal version of Perl 6 that’s much easier to parse as it lack some of the more powerful features of Perl 6. It’s a much higher level language than PIR is though:

method TOP($/) {
    my $past := $( $<statement_block> );
    $past.blocktype('declaration');
    make $past;
}

There are three interesting puncutation clusters that need explanation here. The first is that $/ represents the match object. Where you read node in the first example, you have $/ here. This is a Perl 6 feature. The second is :=, which is the binding operator. For now, think of it like assignment; it assigns the result of an expression to a name. The third is the $( ... ) construct, which here processes a subrule of the TOP rule called statement_block. There’s another method in this grammar called statement_block, and it returns a PAST node. This method only sets a property on it with the method call.

That’s basically what the earlier PIR code did too, though it took a lot more code to perform the same behaviors. Right now, the NQP code is half the size of the PIR code, though it doesn’t quite perform everything the previous version did. The NQP code is much more maintainable and expandable too, though, so where adding a feature to the PIR code might take an hour, adding a feature through NQP could take less than half that time.

Currently there’s documentation in PDD 26 (found at docs/pdds/pdd26_ast.pod) about PAST and the abstract syntax trees implemented and supported in the compiler tools. One of the next and most important steps is writing more documentation about how the compiler tools work and how to add your own features to compilers written on Parrot.