Since fellow O’Reilly blogger Luke Kanies mentioned me in his excellent piece Why Isn’t System Administration Evolving? I thought I should clear up some misstatements.

First, I want to say that I’m enjoying reading his series of articles. There are a lot of big issues our there that aren’t being talked about, and Luke is doing a good job of bringing them out so that we can discuss them.

My issue comes from the second half of a paragraph, quoted here:

> Heck, Tom did a short presentation at the Configuration Management Workshop
> in 2005 explaining how he has successfully avoided any real automation
> throughout his career. He’s got the best sysadmin book out there today, but
> he doesn’t use any automation. I like the guy a lot, but that pretty much
> disqualifies him as a key person in what I think of as my profession, and I doubt
> that most sysadmins would say that his book has the solutions to the real
> problems they face today.

First, I’d like to point out that I co-authored the book that he described as “the best sysadmin book out there today”. Christine Hogan deserves credit too. In fact, I recently re-read the book in preparation for a project and I have to say that it was quite clear to me which chapter she wrote because they are the excellent ones. (Mine were the good chapters. The mediocre chapters were written by an elf named Marvin that we keep locked in a closet.)

Secondly, I’m confused as to Luke’s point that the book doesn’t have solutions to the “real problems” people face today. The book is about the techniques and policies related to 31+ topic areas in system administration. The goal, and I believe we were successful, was to make the book as timeless as possible by not mentioning specific products, but by educating people so they know the issues and principles so they can make informed decisions as times change and technologies shift. Therefore, anyone that writes a CM system should heed the advice in chapters 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 23, 24, 26 and 31 (plus the advice on negotiating salary on page 592-593). Ignoring those chapters would doom your CM system to creating something that does not take into account the real world of system administration.

I think what you meant to say is that the book doesn’t have a chapter on CM or a focus on it. That’s true. When the book was written (mostly in 1998 and 1999) the term “CM” wasn’t a term many people used yet. We instead chose to advocate for automation on nearly every page, from the preface to the very last chapter.

However, none of that is why I initially sat down to write this reply.

I nearly fell out of my chair after reading the statement that my presentation at the Configuration Management Workshop in 2005 claimed that I don’t “use any automation.” Disliking the current batch of CM tools doesn’t mean that I don’t use automation. It simply means that I don’t like the current batch of CM tools. I automate the hell out of everything I do. The point of the talk was to convince the CM folks that “the competition” isn’t “doing things manually”. The competition is “doing things using vendor-supplied point tools” which, in a general sense, include things like DHCP clients, LDAP, Kerberos, and software depots. I also pointed out that when trying to sell something new, one should focus on “barrier to entry” issues and current CM tools do a terrible job of that. (This was in 2005… I hope things are better.)

Automobiles aren’t great because they get you somewhere. People could “get somewhere” before there were automobiles. Automobiles are good because they get you there fast (unlike walking), and door-to-door (unlike trains). Likewise, I feel that people are advocating the use of CM systems because “they let you configure hundreds of computers”. Well, people configure hundreds of computers today already. They clone disks (the technology in Norton Ghost SuperDeluxTurboOMWTFBBQ Edition impresses me more than the algorithms I’ve seen in some CM systems). They use JumpStart/Kickstart/RIS and reload machines a lot (A LOT). I’d also say that a lot of sites, if not most sites, simply use the powerful technique of LeaveTheOSAsItArrivedFromTheVendorForever. In fact, LTOAIAFTVF has a pain-point that is a lot better than most CM systems because (1) it’s easier to hire people that know it, (2) by the time it really causes a problem I’ll have moved to another job. I hate to admit that (2) is common, but it is. Everyone hates to admit that (1) is true because that’s “a management issue”.

I have never advocated for doing things manually. I’m the person that’s quoted for saying things like, “If you don’t use procmail, you’re working too hard” and “I’m so lazy I’ll spend a day automating something that will save me 5 minutes.”

The point of my presentation wasn’t that doing things manually is good. “Doing it manual” is bad. My point is that CM folks need to get out of their echo chambers and glass towers and realize that they aren’t competing with “doing it manually”, they’re competing with Norton Ghost, SMS, and policies enforced by ActiveDirectory.

Configuration management has always been important to folks like us, but now it is being recognized as important at higher levels of but now its being recognized as important by higher and higher levels of management. Vendors are feeling pressure to solve the problem, and I think they are doing it at the lower levels (DHCP for network configs, patch distribution, and so on). The challenge now seems to be solving problems at the application level.

It is not just important that CM developers face the usability issues that make CM easy to deploy, it is dishonest us as a community to produce CM systems that are difficult to adopt and then whine that people don’t use them. Every CM project should be spending 40 percent of their budget on usability testing (yes, with the one-way mirror and all).

Walk though a hospital and watch how every aspect of healthcare is directly computer controlled (not just tracked and billed, like it used to be). The added cost of “doing things badly” takes away dollars that our society could be spending in other areas (education, feeding the hungry, clothing the poor, healing the sick). Giving IT groups any excuse to not have a well-run, automated, IT system is not just “bad” or “stupid” like 5 years ago. It is unethical.

Tom