You can hardly peek into the discussions of operating systems today
without the claim being thrust in your face that virtualization is the
most important technology in the field. That should bother people who
are interested in operating systems. I say this because, from what I
hear, the value of virtualization is directly proportional to failures
on the part of operating systems.

Virtualization is said to ensure security by walling off applications.
It’s being promoted to improve performance with large server
applications. And it multiplies the range of available applications
by letting a user run different operating systems.

Therefore, I conclude, virtualization would be almost unnecessary if
operating systems were sufficiently secure, if they could efficiently
support several large applications at once, and if they offered all
the features and applications people wanted.

As an IBM VP told me, the original impetus behind virtualization was
to let mainframe programmers run test and production versions of their
software on a single system for cost savings. I can understand why
developers want to isolate test systems, and why a hosting service
might want to provide the illusion of a fully dedicated system to
different clients. But virtualization is going way beyond such uses,
and being touted for benefits that a good operating system should
offer.

Having vented all this, I’ll move on to discussing two of the many
virtualization companies that came to LinuxWorld Expo in San
Francisco.

Virtualization

VMWare is touting a new level of
control to help virtualization work cooperatively with clusters. If a
server needs more resources than its current host can offer, VMWare HA
can restart it on another node or start a new instance (without
preserving all the state information). It can also keep the allocation
of resources to different instances proportional while adding new
instances.

At the most recent Ottawa Linux Symposium, a number of virtualization
developers met with Linux kernel developers and agreed to develop a
series of hooks (known as paravirtualization) in the operating system
so that the virtualization layer can be more intelligent. For
instance, if the Linux kernel makes the decision to do a massive
read-ahead on behalf of a process (anticipating the reading of a large
number of pages at once) it can indicate this to VMWare or another
virtualization product.

VMWare is also working hard to be an efficient user of multiple core
processors, and to optimize the use of NUMA architectures.

They recently concluded a contest for best virtual appliance: a
software system that combines virtualization with some set of
applications. One of the winning appliances, for instance, implemented
a firewall as a virtual appliance to make it easy to boot with the
firewall parameters in place.

SWsoft’s Virtuozzo system
achieves virtualization through a light-weight intervention into the
operating system similar to Solaris containers. Whereas most
virtualization systems create hypervisors that underlie a set of
operating systems and pull strings to control what resources each one
gets, the SWsoft Virtuozzo product runs with a single operating system
but takes over key functions (scheduling, memory management, network
managements, disk I/O, etc.) to apportion resources. You can tell it
that a certain percentage of memory is always dedicated to a
particular service, or that a service is allowed to eat up a certain
percentage of memory if it becomes available and the service needs it.

Using this bold method of pre-empting key operating system functions
(which arguably means you are running only a shell of the original
operating system), Virtuozzo achieves the goals of virtualization
while adding only 1 to 3 percent overhead. It is particularly smart at
allocating memory among greedy services.

SWsoft has been offering its virtualization software Virtuoso for
Linux since 2001, and for Windows since 2005.

SWsoft has discovered its management tool to be particularly valuable,
and is going to broaden it to work with other vendor’s virtualization
products, starting with VMWare.

The experience of going open-source

Every LinuxWorld is a chance to meet dozens of new companies in the
free software space, many of them founded with the expectation of
making their software mostly or completely free. But established
companies come at open-source from other models. The transition often
takes a long time, as I explained in my earlier
blog on this convention.

Netscape provides a historical example that has been repeated by many
other companies. First the company builds a proprietary product using
standard team dynamics. Then it announces that the product is
open-source, but maintains the same basic team management
techniques. Finally, when managers discover that developers outside
the organization are discouraged and dropping away, the company makes
a major shift, often spinning off the project.

Thus, I asked an IBM VP Adam Jollans what the company had learned from
open-sourcing Eclipse and Cloudscape. His answer was that they
discovered the importance of building a community, which in turn meant
releasing control so developers felt they could have an impact on the
software. His description of the Eclipse project seemed to fit the
model I just laid out.

IBM has contributed to Linux kernel development in three major ways:
support for the cell processor, work on virtualization and Xen, and
SELinux as their security choice. Among IBM’s many other open source
projects are the communities building up around its chips at Power.org and Blade.org.

The historic database Ingres went through a similar corporate evolution.
Described as the first open-source database, and the source of many
other databases, of which PostgreSQL is the most prominent in the
open-source area. Ingres was taken closed-source in the 1980s and
eventually developed a feature set as rich as Oracle’s before Computer
Associates decided to open-source it again.

The first attempt to open-source Ingres produced no improvement in
adoption or the development of community, because the license was
idiosyncratic and management did not put special efforts into
capitalizing on the open-source advantages.

In November 2005, Ingres was spun off into a separate company and took
a fresh approach to open-source. It released the code under the GPL
and started a project called Icebreaker that incorporated
operating-system features from Linux. Ingres can now run directly on
computer hardware with no intervening operating system.

More on mobile

I talked a bit about mobile companies in my previous blog. This week
I met with Access, which
purchased PalmSource and its
historic PalmOS operating system. Access has barreled into the mission
of creating a Linux-based software stack appropriate for the advanced
needs the industry anticipates for mobile systems. At the kernel
level, Access has open-sourced a number of enhancements, including a
telephony driver subsystem that controls dialing, signal strength, and
other key tasks. They are working with IBM’s Dynamic Power
Management project.

At a higher level, Access has added theming to GTK+, in recognition
that service providers take the appearance of their displays very
seriously and want to differentiate their products based on,
backgrounds, and so forth. They’ve also created a GTK+ library more
suitable for small devices, replacing some widgets.

The Access security framework allows applications to present
certificates to cell phone users, just as web sites can present
certificates to browsers. Thus, before you run a cool-looking
application, you can get some assurance that it was actually
downloaded from a company you trust.

Application management is a complex area of mobile devices that Access
has done a lot of work in. Their Exchange manager controls requests
issued from one application to another, in recognition that many
applications must work seamlessly to create a positive user experience
(for instance, the mail program must invoke the contact manager).

Unicon makes a thin mobile Linux
device based on chip-on-film technology. It has all the usual ports
and wireless options (802.11, Bluetooth, IrDA). They anticipate it can
last nine hours with WiFi turned off and seven hours with it turned
on.

Data warehousing and report generation

A lot of organizations find it hard to turn their ships around in
today’s demanding environment, because they can’t get their employees
aggregated data in useful form. Report generation systems (which focus
on the view of the data) and data warehousing systems (which focus on
producing and organizing the data) try to provide a fast path between
the rows of the databases and the employee who needs to answer such
questions as, “Are sale trends of this item going up or down?”

Pentaho is a fast-rising company
that offers a collection of open-source components that can be used
for data warehousing, from retrieval to display.

JasperSoft is an open-source
reporting system that features flexibility, accepting many database
formats as input and producing HTML, PDF, Excel spreadsheets, ODF,
etc. in output. Charts can be generated in various formats. Originally
just a Java library, now it includes a GUI so non-programmers can
design reports. However, the presence of the library means enterprise
programmers can integrate report generation with other in-house
applications used by staff, a much more efficient way to use them than
a standalone application.

System Management

In my blog
from the Boston LinuxWorld, I mentioned the company Splunk, which maintained a searchable
database of system problems and accepted solutions from users. In
addition to Splunk, I met this week with several other entrants into
this space

GroundWork stores a
huge amount of log data in a MySQL database and exposes it as an XML
tree so tools can parse it and display it. The display is controlled
by an open-source Ajax framework called Guava, which hides JavaScript
details behind a simple object framework and makes it easier for
people familiar with Java or other object-oriented languages to create
web displays.

FiveRuns offers system
management as a service. It features almost fully automated
installation. So long as your servers are set up to use defaults for
ports and other parameters, FiveRuns can discover your services and
set up monitoring, logging, and reports such as histograms. Small
organizations can just contract out to FiveRuns, but larger ones will
feel more comfortable installing a FiveRuns server within their
firewalls. Like Splunk, FiveRuns collects trouble-shooting information
and feeds it up when users do a search. The trouble-shooting
information can come from public sources on the Web as well as from
users submitting suggestions and fixes they’ve discovered.

Open Country sells a system
administration tool for installing Linux on multiple systems and
updating software on a routine basis. Open Country works like many
other vendor services, but is distinguished in two ways: low cost and
breadth of systems supported. At a cost of five to ten thousand
dollars, it’s attractive in emerging economies. Its screens appear in
8 languages and it can handle 20 Linux distributions. In its most
recent release, Open Country has improved discovery and reporting of
system components.

Mail and messaging

Their were far too many candidate Exchange-killers at the show for me
to sample them all, but I did get a kick when a Zimbra manager took me on an
accelerated tour of their email and messaging . Even just touching on
highlights, the tour took half an hour. He described the system as an
open-source alternative to Exchange, Domino, or GroupWise, but it
seems to try for a much richer user experience. It received the best
messaging produce award at LinuxWorld.

Zimbra certainly provides the kinds of flexible delivery features
you’d expect from a mail server, such as support for Outlook/Exchange
features, a web interface to mail, and interchange of contacts and
email with several popular types of cell phones. The cell phone
support runs entirely on the server, unlike other programs that
require the cell phone owner to install a client plug-in. It also
treats Windows systems, Macs, and Linux systems as equal partners,
offering the same features to all.

For demo purposes, its coolest features are the mash-ups offered when
you read your mail in a browser. Basically, the mail message comes
alive. However the mouse over a date, and your calendar for that date
pops up. However over a name, and contact info (if any) pops
up. However over an address, and a map pops up.

The default format is a dedicated LDAP directory, but data can be
taken from other sources in the organization instead.

Zimbra’s zippy interface is based on Ajax. A technology they call Ajax
Linking/Embedding allows mail messages to contain dynamic documents
such as spreadsheets, as Microsoft’s OLE does.

While it’s an enterprise product, it’s also popular among hosted
services so the features can be offered to individuals. Because it’s
open-source, it’s been discovered by a lot of people around the world
and has a much broader base than a proprietary product has over a few
years. Zimlets allow extensions.

I asked about instant messaging and was told it may be added to a
future release. Also, trust relationships between Zimbra servers may
be instituted so that information in one are recognized by another.

Other interesting meetings

Red Hat was the first Linux distribution to break out of the
early-adopter circle of select Linux aficianados. Red Hat’s
well-deserved fame in marketing Linux to a broad public shouldn’t
blind us to the shift they’ve undergone in the past few years; like
Novell and other big companies such as IBM and Sun, Red Hat wants to
be more than a vendor of operating systems and server software. All
these companies have gotten involved in Web Services and SOA.

Red Hat boasts about the affordability of the JBoss software it
acquired. (At IBM, I found, they stress their understanding of the business
end of SOA and distinguish that from the programming end.)

At the same time, Red Hat is interested in finding new markets outside
of enterprises, and looking at what kind of computer system is best
suited to different kinds of people. This interest drives their
support of One Laptop Per Child.

I heard Linspire CEO Kevin
Carmony and some of his staff promote the well-publicized Freespire. I pointed out that, in
calling for a developer community around the new Linux distribution,
they were competing for developers with many other distributions. It’s
too soon to tell what response will be, but Carmony said they were
less concerned with attracting the current crop of Linux developers
than with interesting Windows developers. The Linspire vision has
always been to make Linux a mass phenomenon by providing everything
home users want, including a rich multimedia experience. If they hold
out the hope of doing this, they may get Windows developers to develop
applications for Linux and be the first to get something enticing for
home users on this operating system.

CodeWeavers, the product
based on the Wine project (which gets a lot of funding from
CodeWeavers) has just released a Mac version of its software for
running Windows programs. Because the hallmark of Wine is to implement
Windows programs with native calls, the desktop experience is much
more elegant than with a Mac emulator. For instance, you can cut and
paste between native programs and Windows programs running in Wine or
CodeWeavers. (In practice, I found this takes a few extra mouse clicks
on Linux.) CodeWeavers also keeps adding to their list of supported
Windows programs.

Coraid now offers ATA over
Ethernet, so you can back up your data remotely on an array of ATA/IDE
disks.

I wrote about Jitterbit
a few months ago; they represent an application of the idea of
mash-ups for enterprise applications. For instance, if you want
SugarCRM to manage one part of your business and AppExchange to manage
another, Jitterbit can provide a bridge between them so you don’t have
to tediously extract data from each and combine them. LDAP is now also
supported. Managers reported to me that downloads and forum
participation are rising, as are the number of user-created solutions
(in the form of Jitterpaks) shared on the forums.

Earlier blog on this convention:

LinuxWorld 2006 opens as companies adjust to pervasive open source