I am working on finding a way to enable developers working in a wide variety of languages to directly access computationally-intensive libraries written in C++, C, and Fortran. The libraries will have been multithreaded using Threading Building Blocks (TBB), the open source project for which I’m “community manager.” TBB is a C++ template library (like STL). I don’t expect to have much of a problem calling C and Fortran libraries from C++/TBB code. But, what’s the best path to enable someone writing in Perl or Python or Ruby or — whatever — to call these multithreaded libraries?
This search has led me to reinvestigate some techniques I’ve looked at in the past — for example, Perl’s XS — but the idea of having to create an interface for each individual calling language is unappealing. I looked at, and did some experimenting with, SWIG (Simplified Wrapper and Interface Generator). But before I got very far, Parrot was suggested to me by some people on the #tbb IRC channel (on FreeNode.net).
During my initial investigation of Parrot, I wrote a blog about my research. Parrot looked promising to me:
Hence, if we can wrap C++ libraries threaded using TBB, then the Parrot NCI should make it possible for all the languages that have Parrot support to call those libraries. Then, high level scripting languages such as Ruby, Python, and Perl will have convenient access to computationally-intensive libraries that have been threaded for optimal performance on multicore processors.
This post elicited an interesting response on another site: “Will Parrot Ever Truly Deliver?” The author acknowledges that “Parrot does sound like an interesting piece of technology”, but wonders “will it ever be a platform suitable for serious, production usage?” The author’s concerns include the length of time Parrot has been in existence (quite a long time), the instability of the code base (lots of significant changes), and the incompleteness of the support from other languages.
Does multicore change the Parrot equation?
Sometimes a technology is invented, and the time simply isn’t right, the need at the moment for solutions that apply that technology is nearly non-existent, though many people readily admit it’s a “wonderful” technology. I wonder if this might apply to a certain extent to Parrot prior to the age of many-core computing?
In a few years, inexpensive PCs will have 8, 16, or more processing cores. Some people doubt that the average home or office user is going to have any use for all these cores. I think that’s like saying “no one will ever need more than 640K of RAM.” Once it’s possible for the average home or office user to apply algorithms and image analysis and video processing and stock market simulators that were previously available only on high-end workstations in data centers, you cannot tell me they won’t want to do this.
It’s going to take programming techniques like Threading Building Blocks, OpenMP, perhaps new languages such as Erlang, or Transactional Memory applied in Haskell, to multithread these computationally-intensive libraries. I doubt that applying conventional low-level threads is going to be an efficient way to accomplish this in terms of programming time (I’ve worked at this level for a long time).
But on the other side: no one is going to want to convert the mass of existing software platforms/applications that could potentially apply these computation libraries, into C++ or C. A convenient means to enable a broad spectrum of languages to call multithreaded C++, C, and Fortran libraries is going to be needed. Otherwise, again we face enormous software development inefficiency, as a separate interface has to be constructed for each library for each calling language. That’s not a solution that is going to fly, in my opinion.
It seems to me that Parrot is an excellent candidate for addressing this problem. If this is the case, the Parrot team may soon find itself lent increasing support from independent developers, and possibly from companies who recognize the need for this capability with respect to their own applications.
I don’t think this need was really there when PC performance could be improved simply through ever-increasing clock speeds. Single-threaded software that did a few simple calculations was fine then. Multicore, however, changes everything. As highly-scalable multithreaded computation / simulation libraries become available, and people realize they want them, and developers realize they need to be able to call these libraries from every language platform, Parrot’s time may arrive.
