One of the greatest unsung benefits of the Pugs project (an experimental implementation of Perl 6) is that it has created, last I checked, some 15,000 test cases for Perl 6, with thousands more on the way.
I’ve realized something as I’ve worked on the Parrot part of Perl 6….
Parrot rebooted somewhat around the end of June 2006, during the hackathon after YAPC::NA 2006. Since that time, it’s attracted a handful of really great new committers and has had a tremendous amount of new development and improvements.
Though Parrot started out as the intended production platform for Perl 6, another goal was always to make a great platform for any dynamic language. Right now, there are a couple of dozen languages that target Parrot in whole or in part, with maybe half of them in good shape — Tcl, Perl 1, Perl 6, Scheme, Ruby, APL, bc, HQ9+, and more.
Part of the reason for the renewed interest in Parrot is because it has
impressive compiler tools. Allison Randal, the new Parrot chief architect and
the driving force behind most of these compiler tools, claims that people
think writing compilers is difficult because they have to use clunky and
painful tools. (It’s time to bury flex, lex,
yacc, and bison.)
Making it easier to target Parrot means that more people can target Parrot with new and existing languages. For example, today in #parrot on irc.perl.org, a couple of people helped solve a problem with Cardinal (the Ruby port) and a few others discussed plans to get JavaScript working on Parrot in the near future.
I should note that all of these languages can interoperate in process.
All of this work and planning made me realize something important that I’ve never put into words before. Open languages need open, independent test suites.
It’s good to have a language specification, but you can’t beat a comprehensive, executable specification that anyone can download and read and run. Perl 1 had one way back in 1987, and Perl 5.9.x has a huge number of tests.
One of the greatest unsung benefits of the Pugs project (an experimental implementation of Perl 6) is that it has created, last I checked, some 15,000 test cases for Perl 6, with thousands more on the way.
Imagine how much easier it would be to make Cardinal or Pecan (hey, no one has named JavaScript on Parrot yet; it looks a little like ECMA and starts with “P”) work effectively if it were possible to run just the test suite with the language itself, starting from the barest possible sanity tests and working up to the complete, full language.
Obviously there are licensing concerns and trademark concerns — it seems to me that allowing people to redistribute and run the test suite and, if unmodified, call their implementations “compatible” is good, though distributing modified versions of the suites should not lead to confusion. (The Artistic License 2.0 seems reasonable here.) As well, it’s important to have the legal power to distinguish a compliant implementation from a non-compliant implementation.
Still, having a good test suite helps the language itself and it helps interoperability, especially in this new age of languages running on several good virtual machines and platforms.
Now how do language developers and implementors and designers and users encourage this?
Ruby implementers from JRuby Ruby.Net, MetaRuby, and Cardinal are trying to work together to get a ruby conformance test suite. http://groups.google.com/group/RubyTests
Kevin
This style has been employed with great success for Parrot's Parser Grammar Engine (PGE.) PGE copied Perl 5 regex tests verbatim from bleadperl, and uses a custom harness to run them using the Perl 5 Regex interface to PGE on Parrot.
The PGE Perl 6 regex tests have been written in a similar form. This allows the Pugs team, and anybody else writing a Perl 6 regex implementation, to steal hundreds of existing regex tests. All they need to do is write their own harness to run them, as well as track which tests are classified as 'skip' or 'todo' as their implementation develops.
I find that designing tests for sharability has greatly benefitted PGE. During early development, we were able to make use of a great body of quality tests (perl5.) Now that PGE has matured, we are able to share our tests with others. I expect that as other Perl 6 Regex implementations develop, they will extend the existing body of tests, which will lead to improved quality in all the implementations.
I think this transitions well into a question I asked a long time ago on the perl6 mail list (or was it a newsgroup?) about visibility into the virtual machine for administration.
Newer java's have a lot of cool stuff for monitoring memory, cpu, existing routines, etc. So.. wouldn't it be nice to monitor what your ruby/python/perl6/perlvascript was doing during runtime inside of parrot?
Matthew, that's a great idea. Norman Nunley and I looked into it a little bit at the post-YAPC hackathon. There is a profiling runloop in Parrot right now, but there's not much visibility from outside or inside yet.
What kind of interface would you expect to see and what kind of information would you want?
Basically, the ability to run 'ps', and 'vmstat' inside of the virtual machine is what unix admins like me look for. It's what we do on (nonvirtual) machines, after all. :)
As far as an interface, that doesn't really matter. Obviously, the holy grail is to get this kind of info on-demand without performance impacts. Hooking into snmp also makes life easier for profiling with rrdtool and the like, but dumping to stderr (think java kill -3) is also pretty handy.
Off-topic but,
All these project names are confusing. Why not steal a page from Microsoft and start calling things JavaScript.Parrot, Ruby.Parrot, Perl6.Parrot, etc. It would be less confusing and better marketing, it would be easier to explain to people what's going on when they can see the similarity to .Net.
miguel, I can see your point. The reason I haven't used the "official" name for any of the language ports I've worked on (Punie and Pheme) is that they're not far enough along for me to feel comfortable advertising them as compliant implementations.
If they had comprehensive test suites and some sort of specification to pass (yes, I'm aware of S5R5), I might change my mind when they reach the point of conformance.