Here’s my wrap-up of the Open Source convention. I published an earlier blog on it as well.
Philip Rosedale and Rishab Aiyer Ghosh on the economics of open sourcing
Second Life’s client, the world’s most popular 3D social experience, is now open source and is receiving both bug fixes and exciting new functionality from the community. Founder Philip Rosedale explained in his keynote that the main thing holding up release of the server is troubling gaps in security that Linden Lab wants to fix first.Theorizing, Rosedale advised any company that hopes to benefit from network effects (such as companies with social aspects) to open source its software from the beginning. His reasoning starts with the common observation that network effects lead to a power law distribution, where the leader in each field gets the bulk of the traffic and potential revenue. Therefore, contributing to the whole–putting your innovations into the common pot as open source–increases your own take even more than that of your competitors.
In a presentation of a large European study about free and open source software, long-time researcher Rishab Aiyer Ghosh repeated an assertion that can also be found in the writings of Eric Raymond, that only a few percent of programmers earn their living from creating packaged software. In other words, the vast majority would have jobs even in a world that moved entirely to free software.
Other findings included:
The majority of firms in every industry surveyed said they include free software in their products.
When free software is included, estimates of the value of software to the economy are approximately doubled.
The time spent by Europeans contributing to free software in 2006 added up to the equivalent of 26,000 full-time programmers.
The percentage of free software code contributed by companies is gradually rising and now stands at nearly 20%. I noticed in the chart a slight decline during the dot-com bust of 2001-2002.
SourceForge commits from Europe have recently surpassed those from the U.S. and Canada for the first time. Commits from Japan and China are gradually increasing but still form a very small part. When the results are scaled to take average national income into account, India and China rise to join the most active committers.
Both developers and companies agree that work on free software can compensate for the lack of a degree in hiring.
In any five-year period, half of the code on any free software project is replaced.
People matter: the new obsession
The Asperger’s phase of computing is over. Everyone at the Open Source convention wants to learn how to work with people. The most salient difference between conferences of earlier years and this one is the proliferation of sessions about how to organize a team, how to handle disagreement, how to motivate developers, and so on. Some of the tutorials dissect these activities in a meticulous fashion (so maybe Asperger’s is in effect after all) but for the most part, the insights have been known in the business world for decades. However, the books and presentations circulating in the computer field tune the doctrines to the pressing needs of the second half of the first decade of the century.Intel contributes
There was a lot of participation by Intel on the convention stages, and because it’s one of the major employers in the computer field here in Portland, where the convention is held, no one was surprised that a lot of Intel staff were in the halls as well.I had a very pleasurable experience working with Intel engineer and evangelist James Reinders on the book Intel Threading Building Blocks: Outfitting C++ for Multi-core Processor Parallelism . It was really a team effort by a bunch of wonderful folks at Intel (whom I finally met here in Portland this week), coordinated expertly by James. The development process was insane, because Intel proposed the book–along with an urgent requested to get it released at to the Open Source convention–some three weeks before the production deadline.
Should anyone think that the hallmark O’Reilly quality may have suffered because of the rush, I encourage you to pick up the book and judge it on its merits. It’s first chapter is a superb justification of the subject matter, in the context of modern multicore computing (and there’s no reason the library couldn’t run on AMD cores as well). The last chapter is a history of research in high-level concurrency. In short, Threading Building Blocks is a very elegant way to hide the difficult and error-prone aspects of parallel programming.
I thought that Intel’s concern for keeping secret that they were open-sourcing Intel Threading Building Blocks was a bit old-fashioned, and that making a big splash about open-sourcing it at OSCon would be anti-climactic. But in fact they generated a bunch of nice news articles that way, and James greatly entertained us at his keynote, deliberately making himself the butt of laughter by pretending to make a sales pitch for TBB.
I was nearly as much impressed by Intel’s release of its Linux-ready Firmware Developer Kit, as described by Rolla Selbak. The pressing problem addressed by this kit is the difficulty of vendors in making sure their BIOS works with Linux. They check it against Windows, but the Microsoft testing tools are lax and don’t enforce specifications. ACPI problems in the firmware can lead not only to easily anticipated problems on Linux (such as with suspend and resume) but a host of other surprises, such as in fan operation and overheating. By inserting a CD with the developer kit, anyone can check adherence to specifications with a two-minute automated test. The output is easy to read, and developers can separate tests for open and proprietary systems.
Where have the database programmers gone?
Attendance at sessions about databases were down this year. The problem affected not only MySQL sessions (which could be explained by the strength of the annual MySQL Conference and Expo) but PostgreSQL as well. On the other hand, I noticed good attendance at sessions about data mining, such as Toby Segaran’s Data-mining from Open APIs and Lino Ramirez’s Machine Learning Made Easy with Perl. Perhaps the growth of public data sources has raised developers’ interest in making creative combinations of other people’s data, as I reported in my earlier blog. But sites are collecting more and more of their own data as well, and a good understanding of database topics–such as storage engines and schema management–could help them use it much more efficiently. One interesting talk along those lines was Mike Solomon’s Super-sizing YouTube, where he described the incremental ways that company reorganized its storage and access methods as it grew into one of the Internet’s most popular sites.Odds and ends
OSCon was a very big show this year: over 2,500 attendees, and 15 parallel tracks. My own talk about my research on free documentation was up against some famous people doing interesting presentations at the same time, which probably accounted the drop in attendance relative to a talk I gave at a former OSCon. But the dozen people who came gave me some excellent feedback, which I’ll describe in another posting next week.A star-studded panel at Powell’s Technical Bookstore celebrated the release last month of O’Reilly’s best-selling Beautiful Code. The panel consisted of Karl Fogel, Greg Kroah-Hartman, Simon Peyton Jones, and Perl/Parrot developer chromatic, moderated by Ward Cunningham. About 50 people attended, and the conversation raised interesting controversies over such things as what makes one language more or less appropriate and whether genetic programming can create beautiful programs.
Most companies will not release their software into the open source for one simple reason: return on investment. When you've invested 20 person-years of development into a product, tweaking and grooming it so that it is stable and user-friendly - then you're not going to let someone else copy the thing, paste their logo onto it and sell it while keeping you out of the profit loop.
Large companies that earn their money from consulting are able to give software away because they have a regular stream of incoming money. Small shops cannot afford to give away that source of income, as they have nothing to compensate for the loss of profit.
The license restrictions are all well and good, but very few have actually been tested in court. Not to mention that a small shop probably doesn't have the resources to actually defend its rights in court - especially not when the adversaries are large corporations.
Reality is that somewhere down the line, someone has to pay for the time and investment, as programmers need to put food on their tables as well.
@OSsceptic, most companies in the world by far never sell software. It's not their business at all. Nick Carr goes into far more detail on this.
@chromatic
Obviously I was referring to software companies, who are not all eager to open up their source code. Nick Carr has a good point that for most companies on the planet (except for software companies themselves IMHO), software and information technology are a commodity. For software companies, however, it's their life artery.
Everyone is free to choose open- or closed-source software to use in their company. Likewise, every software company is free to make their source public or keep it private. Titles like this article's make it sounds like a software company has to apologize for not sharing the source, regardless of the resources (time, money, blood, sweat and tears) that have been poured into it.
You have got the wrong end of the stick here. What Skype has done, is take someone else's open source code, modify parts of it and not make those modification available under the same GPL terms they received the bulk of the code they use which is released under GPL by others. If Skype wanted to not GPL their modified code, that is their right, but they would then have to rewrite the bulk of the code they use, which I suspect would not be a viabile exercise compared with getting it as a GPL freebie.
In any case I don't think the spat is a major one. Skype wants to GPL the code. It is simply the means by which the code is made available that is being disputed. The whole thing seems to be a case of splitting hairs.
"Percentage of programmers that earn their living writing packaged software."
Andy, Statistics are made to contribute to manipulation and I may be doing that inadvertently because I have not read the reports you talk about by Ghoush and ESR. But from my former work on the Worldwide Programmer Census published by IDC, I can point your readers to a publicly available source on the subject (http://www.bls.gov/oes/current/oes151021.htm.) Based on that U.S. Department of Labor source, the statement in your story that "only a few percent of programmers earn their living from creating packaged software" is not correct, at least in the U.S.
The government says there are 400,000 programmers in the U.S and that more than a third of them are in the industry "Computer Systems Design or Related Services" and/or work for "Software Publishers." Even if the government is off by half, that's hardly "a few percent." If you count what the government calls Computer Software Engineers for both Applications and Systems Software, which I would recommend, you see similar percentages.
Again, your sources could be using other data and other methodologies but I tend to believe the Department of Labor numbers at least for the U.S. In addition, when you think about it, even if you're not creating packaged software you're working with it and could not be accomplishing your job without it. So the idea that everyone can create their own wheel and that that would be good for economic growth doesn't compute.