Perl 5 change #33858 adds three new features from Perl 6 to Perl 5.12. With recent interest in implementations of dynamic languages on various virtual machines, I thought it interesting to discuss how a feature comes about and how it works — even if you’re a very happy user of Perl or another dynamic language with no desire ever to look inside your favorite implementation, the details can be enlightening.

Here’s how it happened.

The three features of Perl 6 are collectively the “yada yada yada” operators, as described in List Prefix Precedence in Perl 6 Synopsis 3. They allow you to write stub code that compiles but doesn’t do anything in particular. This is useful for exploration, example, and bottom-up programming. You can write:

sub main
{
    do_this();
    do_that();
    finish_up();
}

sub do_this   { ... }
sub do_that   { ... }
sub finish_up { ... }

… and the program will compile and run. That’s valid Perl 6 code. (It’s also now valid Perl 5 code.) Of course, when Perl encounters ... at runtime, it will throw an exception that you’re trying to execute unimplemented code. It works at the statement level, so you can also write:

sub do_this
{
    my $arg = shift;
    return do_something_fun( $arg ) if defined $arg;
    ...;
}

… and stub out error handling, if you’re not ready to make this code completely robust yet. That’s a very cheap error handling mechanism.

Adding a new syntax feature to a language on a VM generally requires two parts. First, you need to modify the parser to support the new feature. If you’re fortunate, this will be easy and you won’t have any parse conflicts, as they’re tricky to resolve. Then you need to modify the generated bytecode or optree or AST to produce the desired effect. If you’re fortunate, you can implement the behavior in terms of operations the VM already supports.

I was halfway lucky. Modifying the parser was easy, though there are potential conflicts with really ugly Perl 5 code. Modifying the optree was fantastically easy.

If you want to skip ahead and read the patch, you can ignore everything except for the chunks for perl/perly.y and perl/toke.c. Everything else is either the generated parser or documentation and tests. This only took eighteen lines of code, counting whitespace and braces. I’ll explain how it works tomorrow.