A lengthy thread on the Perl 5 development mailing list has asked if there’s self-generated FUD about the future of Perl worth debunking.
I’m not the person to ask about the viability of using Perl 5 for new development projects in your own environment (and I can’t wait for Perl 6), but I had an interesting thought about how to gauge the suitability and evolution of a language today….
How Does a Programming Language Stagnate?
A new question arose yesterday in the monster Perl 5 Porters thread (see my Come Back, Zinc!). Given the increasing length of time between stable major releases, do users think Perl 5 is stagnating?
The question is more complex than it sounds at first. For example, who is this mythical pluralized user? What kind of work does he or she do? Does this user follow p5p or Perl Monks or read Perl.com? What kinds of features does he or she use or want to use? What level of involvement does this user have in the system?
(The always-imminent Perl 6 has a further effect on the question. Will Perl 5 be a legacy system soon? For how long? Then again, given how much Apache 1.3.x and PHP 4 and MySQL 3.23.x and even Perl 5.004 and 5.005 are still around, perhaps that legacy question isn’t as important to the users we ought to be considering. Maybe this is the most important question, though.)
Which Part of the System Actually Matters?
I immediately objected to the idea. My brain immediately said, “You’re a smart man, person who posted this thesis, but there’s something really wrong about it.”
I had to think about it for a while. (My poor roommates heard the whole story though, but that’s what you get for walking around in my house.)
Of course, everything in programming is about creating, using, and managing levels of abstraction. That’s the platform, the core language (syntax and semantics, possibly even idioms — but nothing more), the libraries, and the tools.
Striating your language into these levels brings up several questions:
- Who develops at which level?
- At which level do most users concentrate their use?
- Which is the most appropriate level for a new feature?
- How well integrated with the entire system is a feature at each level?
I suspect that the level at which most developers write the most code also dictates at least some of the utility of the entire stack as well. (That is, if everyone’s building developer tools, who’s building end-user applications?) That argument might go the other way too — the level at which most users need to use the stack dictates the level at which they need to work. It’s difficult to discount the abstraction capabilities of the language, though. (Would Java have as much tool support if its syntax were more malleable?)
You can’t discount the purpose of the stack either. The GNOME and KDE and XFCE projects have different types of developers and different types of users than do programming languages. Yet I digress.
Where Does a Language Grow?
I’ve long argued that abstraction and extensibility are two of the most important characteristics of any whole programming language. (Often I’ve argued this in terms of maintainability, so the point there is indirect.)
Consider C. The language has a small syntax and a reasonably well-defined semantic model (measure K&R versus Java in a Nutshell part n). There’s little evolution in the language or its semantics (okay, C89 and C99, but how long until you can rely on working cross-platform and cross-compiler implementations of the latter?). The only reliable (macros don’t count) extension mechanism is creating shared libraries.
Fortunately, C is portable enough that well-written shared libraries are reasonably easy to rely on. Besides that, if your users are “people who care about programming in C”, you can assume that they have the ability at least to call C. (It gets better that C is easy enough to link to from almost any other language. That’s further growth.)
Languages such as Perl and Python and Ruby have it both a little easier and a little more difficult. These languages don’t need a specific compilation step in the sense that, once you have the runtime on your system, you have what you need to run most of the code written in those languages. There’s no distinction between having the Perl compiler and the Perl runtime, as there is with C and C++ and even Java.
That has two opposite design effects. First, it’s easier to push some of the complexity and extensibility of the whole language into libraries. Second, it’s tempting to pull that complexity into the core language layer to make further extensions easier or even possible to produce. It’s probably even the right approach, much of the time.
Distributing pure-language applications can be easier because, in theory, the runtime abstracts away platform-specific behavior behind a useful interface and, also in theory, pure-language code can run anywhere. There are obvious exceptions, but I suspect this is frustrating enough when it doesn’t work because it works most of the time.
Non-pure extensions are much worse. If users only need the language
runtime, most of the time, can you rely on them having the compiler for
another language available? It depends on the user’s strata. Perl 5 porters
have a working C compiler set up to compile Perl, or they can’t do much
useful. (Okay, I exaggerate slightly. They at least have a
make utility and sane shell settings. That’s the requirement
for being able to patch and test the core libraries.)
I wonder. Does a language grow despite the difficult parts? Even though distributing compiled extensions is difficult in comparison to pure-language extensions, are people using them?
That’s mostly rhetorical.
When and Where Do Users See Stagnation?
If I submit a patch to p5p to add a new feature, when can I rely on that in code I want to distribute to users? It depends on the users… but it will be a couple of years. Maybe the bleeding-edge Linux distributions will pick up Perl 5.10 within several months of its release. Maybe not. The BSDs will come along soon.
Yet will people running stable applications that need to stay up and supported for months and months upgrade any time soon?
My experience says no. (It’s a pity too, because I want to use that new feature now.) Again, I point to Apache 2, PHP 5 and PHP 6, and even Perl 5.8.8.
Could the idea behind the levels of abstraction work for measuring the level of involvement in the language? Some users always stay on the latest stable version, but my theory is that these levels form an inverted pyramid. At the base (or tip) are people such as the various Perl 5 pumpkings and Larry. As you move up the pyramid, each level has more users until you reach the wide-open Internet, with millions and millions of people using sites and applications built with Perl. They just don’t know it.
If the propagation of a new version of Perl throughout the world reflects the distance of a user from the tip of the pyramid. I expect someone still running Perl 5.004 to know and care very little about new features of Perl. If there has been no reason to upgrade in the past decade, why upgrade now?
It’s not just people tied to legacy platforms, though. I installed Wine on my desktop machine a couple of years ago and haven’t used it since. I don’t use it often, I don’t use it heavily, and I don’t have any real reason to upgrade. I suspect that applies to a lot of languages as well. (I’d have used Java as an example, but people might figure out it’s my whipping boy.)
Do new features in Wine matter to me? Not until they matter to my use of Wine. It’s a fine project and I’m glad it exists, but even if the pace of change did slow down for a while (and my impression is that it hasn’t), it wouldn’t affect me greatly.
More importantly, my impression of Wine isn’t that important, especially to the Wine developers. I’m not a target user.
The People Whose Perceptions Matter
Whose impressions matter? Target users.
Who are those target users? That’s a big question.
If my theory of layers is right, far fewer people will care about the core language than the libraries. In a well-designed system, you also need far fewer people to maintain the core language than the libraries. (That may be why we call such a system well-designed; it meets the reality.)
Attracting people to do that is useful and necessary. Yet for every one person you can attract to hack on and maintain the core language, you probably can attract a hundred or a thousand people interested only in doing useful things with the whole language and libraries and the platform and tools.
I strongly suspect the health of a language and its perceptions depends more on those thousand people than the one, though without the core hackers, a language must eventually converge to a stable implementation or just die. (There may even be a difference.)
When someone talks about a language stagnating, though, there’s an implied audience. Who thinks it’s stagnating? Why? Do their impressions matter? Is the impression accurate? Is it worth contesting?
I’m not sure how to answer those questions With regard to Perl 5. Most of the evolution of the whole language takes place on the CPAN, and the rates of development and invention and change keep increasing every year.
That doesn’t matter much if people at levels above the libraries can’t or won’t use them though.
There are still language changes and enhancements going into Perl 5.9.x (which will become Perl 5.10) and I’m anxious to use them… but I’m not sure they will make the people above the library layer rush out to upgrade. That’s not bad, they’re just not fundamentally different features. They’re mostly high-end enhancements, with a couple of notable exceptions.
These are all good things, though. Perl’s evolution shouldn’t have to wait for Perl 6 or the release of Perl 5.10 or any other single point of failure or control.
Yet there must be mechanisms for users who don’t want to have to understand how to set up a compilation environment on their platforms and to slog through several different but similar modules to find the right one for their purposes, mechanisms by which they can get the benefit of this evolution without having to manage it themselves.
The same goes for any other language.
If there’s a perception problem, perhaps it’s because people are looking at the wrong thing and expecting the wrong answers. Perhaps Perl end-users need to concentrate on the work of distributors and framework and tool builders, rather than the comparatively raw, low-level output of the core builders.
If that’s so, maybe there should be a way for these groups to communicate. Maybe the core language is converging on its ultimate set of features and its ultimate steady state of stability. Maybe it’s temporarily reached a false maxima. I don’t know. I do know that evolution and growth is not slowing at the edges.
Is Perl dying? There are far more and often better tools and libraries available right now than there were a year ago. That seems like progress to me.
Now the trick is demonstrating that — and, just maybe, moving a few people to deeper layers, bit by bit.