No
Instead, urge developers to do a better job
of debugging, and set up better channels of
communication so the administrators will know
what they really need to install.

Haven't we been doing that for years? And years? And what have we got to show for it? The same story over and over and over again. Why can't we just wake up and stop using bad quality products?

I don't blame the sys admins. I blame the CIOs and IT managers who continue to make bad, costly decisions and they're not held accountable. Someone at O'Reilly decided to deploy MS SQL Server to run your operations? Fire that guy (or gal) and fix it (replace it)! Enough already! When will this epidemic end?!

Patching Database Servers
I think also that this introduces a new realm of awareness for exactly what needs to be patched. Many sysadmins have gotten into the habit of religiously applying windows and iis patches (after internal verification), but no one I know is as eager to apply patches to database servers. First of all, this usually means downtime, and second, no one wants to think about what would happen if the patch caused a problem.

SQL server is vulnerable
Whenever I examine my logs, I've found that roughly half of the port scan attacks I get are targeted at SQL server and virtually every HTTP 404 response code is for an IIS/FrontPage vulnerability. Anybody that looks at my site for more than 5 seconds knows that this would be a waste of time. While it doesn't cost much to probe, there is a reason so many malicious agents spend their time running these tests on every single server they can find on the internet: these attacks pay off.

My sympathy is with O'Reilly. Based on pure database functionality, SQL Server has represented a good value in the market. However, when you mix the application layer with the OS layer as Microsoft has repeatedly done, you open yourself to vulnerabilities on the highest order. Multiply that by a factor of 100 for server-side applications. There is no reason a browser should allow arbitrary code to run on a system, but Internet Explorer had just such a bug last year. Why? Because, as Microsoft maintains in court, it is inseparable from Windows: they coded it that way.

Bill Gates, if you honestly want a reputation for trustworthy computing, you *must* define the split and publish *all* the APIs to the system layer. Of course, if you do, realize that you will then actually have to fairly compete for Windows application market share.

Slight problem overlooked
The real question is why were these SQL servers exposed to the internet? They should have been firewalled properly to only communicate with the web servers that they provided content to.

As a sysadmin, I know that patching database servers is not always fun, as MDAC has a tendancy to break things when upgraded. However, if the server is firewalled and only the webserver can access it, this worm would never have been commented upon.

Lack of process is the problem
I think many, many shops simply lack any regular, rational process for evaluating and deploying patches. All too often the theoretical risks of applying patches in general are invoked as an excuse for not bothering to patch at all until there is an emergency. Then the patches go flying on. Sure, developers should try to avoid security bugs. But admins should try to close critical holes without waiting until they are forced to act.

Monocultures are susceptible to epidemics
Microsoft illustrates if there are too few alternatives in the marketplace the adaptive evolution of viruses and worms will produce and continually reproduce widespread epidemics. This is well known in agriculture, e.g. the Irish potato famine. It's well known in human diseases, e.g. the spread in the New World of Old World infections post Columbus.

If Microsoft were to be broken into 2 pieces, the monocultures would still be there. As a matter of national and global security Microsoft should be divided into at least 10 equal parts, and each part should be unrestricted to pursue any product area.

In the long term, if monopolies eventually re-occur by competition, we should again follow the advise of that great Republican trust buster (Teddy Roosevelt) and break up any resulting monopolizing entities.

need security behind firewalls, too
To be honest, when I saw all the SQL server port scan attacks I was getting, I also wondered: Why would anybody host an rdbms server publicly?

However, even if your servers are behind firewalls, if they are open to a network they should still be secured. The most devastating attacks often come from within.

Monocultures are susceptible to epidemics
Of course, this is just a far fetched analogy. And Slapper wreaked a lot of linux havoc not so long ago. Fragmentation leads to higher costs and lower productivity. It plays into the hands of rapacious consultants and charlatans. And it allows obnoxious programmers to terrorize captive in house audiences. Eventually the holes will be plugged and the world will be a better place for everyone. Open Source will flourish as a non-commercial niche adjunct to Microsoft Research.

better vendor support helps, too
I have had bad experiences with Microsoft patches and updates to compare with relatively few good ones. I don't know what it's like to backup and rebuild a SQL Server database, but I'm sure it's a deterrant to making changes. Windows systems are so monolithic and homogenous, it's hard for me to understand why they can't get these things right.

In contrast, using Red Hat Network and RPM I can update disparate systems easily and cleanly. I've even made kernel updates without missing a step. This is particularly impressive considering how different any two Linux systems can be. Red Hat is good about pro-actively notifying their user community of vulnerabilities where Microsoft is often hush hush as a policy. Finally and perhaps most importantly, Red Hat has been exceptional at establishing trust with their users; some Microsoft "updates" load you with more than you want (e.g. Windows NT SP6 = IIS).

Shops need to take patching seriously, but the vendors should never be considered free of blame.

of rapacious consultants, charlatans, and obnoxious people
Slapper did wreak a lot of havoc, and the open source community needs to be prepared for future vulnerabilities because they *will* occur. Some open source developers turn into rapacious consultants, and any individual can play the charlatan. Sometimes obnoxious programmers do hold audiences captive, and Microsoft can improve. However...

These symptoms are not the result of fragmentation. Microsoft itself is a rapacious business. They play the charlatan more than any single company, with "innovations" that are more often directly based on stolen ideas or acquired products than true knowledge or insight. And no audience is more captive than the Windows user community, having to listen to statements like "Linux is a cancer" from obnoxious executives. The holes get plugged, but new holes arrive with each new product and release. Slapper is one failure to Microsoft's many. Fragmentation, and hence competition, brings strength.

lteti, I don't know who you are, but I find myself responding to your statements and attacks too often. I have administered both Microsoft and Unix servers in my career, and I have an opinion. With all those labels, you may be thinking you're talking about me, but I assure you I have a bit of knowledge and experience, do not seek to exploit users, and will terrorize no person. I think we should keep this dialogue off-blog from now on: you can always find me at www.mentata.com. If all you want to do is try to hack me, go ahead: it will be an act of diminishing returns because I rely on open standards, not any single vendor.

I Love Patching Database Servers!
We applied a large set of patches to a multi-terabyte system over the weekend. We touched the RDBMS, the parsing engine, the optimizer, and the UNIX kernel.

Total outage: Under five hours, more than half of which was verification.

I love applying patches to systems run by companies where information runs freely, the patches are well-tested, and the backout plan is clear.

Note: We aren't running MS SQL or Oracle. Obviously.

Sorry
I was not referring to you.

better vendor support helps, too
I have a lot of experience with Microsoft support, and it has generally been the best support I have received from any vendor. Of course, we pay for it.

As for vendor openness: I don't see how anyone who reads the freely available MS security bulletins could accuse them of being hush hush. They strike me as being very proactive.

microsoft support by contrast
Although I have administered systems for almost 5 years, I'll admit I don't have a lot of direct experience with support staffs because I am usually able to fix problems on my own. As a user and administrator, I personally have had about a half dozen encounters with Microsoft support. Of those, twice they were unable to help at all, although they still tried to charge an "incident" from a 10-pack purchased by my employing organization.

RH Network is not free (about $80/year), although you get a free trial if you purchase the software. It is also not a phone number: it's self-service. You are notified by email immediately of patches that affect your system as configured, and can run a single command to select and install updates. You have an option to test the update first, and can find out what is being modified down to the *file*. You usually don't even need to reboot the machine to apply the changes. I've installed a lot of upgrades, and I've yet to have a problem. I know it sounds like I'm advertising, but believe me: you have to see it for yourself.

The Microsoft bulletins are getting better all the time, due in part to extreme community pressure. They have a policy against third-party bug disclosure but are known to drag their feet if bugs are shared with them privately. It's my understanding that some stuff just doesn't make the bulletins. If you're reactive, you're not also pro-active.

Linux is not invulnerable
Andy Oram is right to call me on the bashing. Microsoft can change, and hopefully they're listening. I'm really just interested in sharing ideas and living in a world of better software....

Hey Bill, give me a call. Nobody can buy me, but I'll partner with anyone. You've got a directory server, right?

Let me set the balance. First, my sympathies to the SQL Server users who were damaged by this threat. The web is a rough neighborhood, where nobody can be complacent about security. Linux will experience vulnerabilities again, too. A Linux installation can require constant patching, and may even suffer if patches fail. Software is complex.

Fred Langa has an interesting article on vulnerabilities comparing Microsoft and Red Hat. He demonstrates how, by errata, Red Hat 7.2 has had more vulnerabilities than Windows XP. I believe it, although I'd be tickled if I saw any of my many Windows XP using relatives in a religious fervor over patches. A Red Hat distribution has a lot of software from a lot of different people, and Fred is quick to use that word again: fragmentation. It's no joke.

I do have to take issue with his experiment protocol, though. I use Linux v7.x. I don't apply every patch. Why? Well I am lazy. Larry taught me: laziness a virtue for a programmer. But more importantly, I don't need to....

I'm an administrator, not a desktop user. I don't need or want the whole enchilada, just what I use. With the kernel as it is, that can be a toothpick sized taquito. If I use GNOME or KDE, I don't need to apply the patches for the other. I could even elect to use neither: a server doesn't necessarily need a windowing interface after all. Try comparing the number of security patches for the slimmest Windows 2000 system you can make to those of the Linux kernel.

I believe Fred's point is still valid: everybody will have their dirty laundry, and Linux installations will be no different as they become more common and featureful. My point in this blog was that Red Hat pro-actively supports your right to exercise choice *and* stay secure. Where security is concerned, fragmentation is a strength of Linux.

Perhaps Linux isn't ready for the desktop. Maybe Microsoft will always write better or more secure clients. But we were talking about servers here :)