There’s a lot of good, interesting discussion happening on this blog as of late. The state of system administration tools, in general, is pretty poor. “Taking care of business” is still very dependent on the ingenuity and creativity of the individual administrator. Maybe being able to be creative is one of the things you like about administration, but wouldn’t it be nice to solve a problem more interesting than, say, installing emacs on 30 machines without working up a sweat?
And sure, more sysadmins could probably participate (or initiate) more projects, or provide more feedback. And more sysadmins could probably publish more of the bits of code they write than they do. And more of us could probably be more productive if we could find a way to interact with more admins, thereby realizing that we’re really *not* the only people who could ever make use of that 20-line piece of perl mastery. I agree on all of these points. However, I think much of this dances around a few large,
multi-colored elephants in the room. Flame if you must, but allow me to point them out for you….
First, the state of the sysadmin ‘community’ is in a somewhat unstable state for the moment. There’s a lot of hubbub lately around the “LOPSA or SAGE” question. Which one do I join? What’s the difference? Where do I fit? These questions need to be answered satisfactorily by someone who doesn’t really
care which organization you join. I’ll do my best to be a neutral third party and ask others to comment below:
I’m a member of USENIX, SAGE, IEEE, a local LUG, and LOPSA. I’m a co-founder of LOPSA-NJ, which is a New Jersey chapter of LOPSA. We have meetings, speakers, discussions, and the like. The group took about 20 minutes to form. We started a mailing list, put word out to a few of the other LUGs in the area, spoke to everyone we knew, and the first meeting was a smashing success. However, there was discussion at the meeting about this LOPSA vs. SAGE bit, to which I replied that the LOPSA vs. SAGE nonsense should be left to be fought over by those who have some reason to care about it, or who have nothing better to do.
Systems administrators should be more interested in getting things done than about who wins some political battle. I belong to SAGE because I derive some benefit from that. I belong to LOPSA for the SOLE REASON that they reached out and said they’d support us if we started a local chapter to gather systems administrators and form a community. Having a local presence, to me, and to many others I’ve spoken to, is the only difference between LOPSA and SAGE that matters to the average administrator.
Being able to get together with other admins and talk not about photo management in ubuntu, but about monitoring a globally dispersed collection of servers, regardless of what OS they’re running, is valuable to me. I also think that the lack of standards and best practices is better addressed through efforts that start with the community than those that start with a committee.
LISA is the only conference I’ve ever seen that attempts to be devoted to system administration. Sure, there are service-specific conferences like ApacheCon and security-specific events like SANS, and they’re great, but overall, LISA is the only one that is truly and unapologetically admin-centric, and attempts to cover a broad range of admin-specific topics (ie, not just apache, and not just security).
I think this is a sad state of affairs, really. I went to OSCON in ‘06 as well as LISA, and I think OSCON could do much to exploit the overlap between development and administration. Even though I do write a little code, I was thoroughly disappointed in OSCON because, even in places where the administration end of things was germane to the discussion at hand, it was completely ignored; avoided even.
What’s sad about that is that I was really pretty unimpressed with LISA, and would happily take my business elsewhere, if only there were a place to take it. I view LISA as proof that there’s a demand for admin-centric conference content. I view LISA as free market research for O’Reilly or some other willing organization who can take the ‘AdminCon’ ball and run with it.
By the way, I’m happy to attend/speak at/help with the planning of an admin’s conference, or the admin component of an existing conference. I’ve done conference planning before: I helped php|architect magazine organize their first php|cruise when I was still Editor in Chief of that publication. I was unable to attend, but it was a packed house, and subsequent conferences have also done well, which, of course, speaks somewhat to the usefulness of the first. In fact, I’m so interested in seeing this happen that I’ll *volunteer* my time to help with organizing, planning, coordinating, etc., for the first year that someone attempts to address this outside of LISA. Email me (jonesy - linuxlaboratory - org)
System administration isn’t evolving? Are you for real?
Let’s face facts, system administration isn’t what it was 20 years ago. Whether you want to believe it or not, administration of very large sites has gotten easier, not harder. Just ask anyone who has been doing administration for 20 years. They do not long for those days on the by and large.
One thing that inevitably happens as things get easier is it lowers the barrier to entry to the profession. Sure, there’s a TON of technologies I need to know about in order to do my job, but overall, maintaining
infrastructure services is a far more streamlined operation than it was in the days before luxuries like NTP and DHCP. Those two technologies alone save us all many hours of grief every week. In a nutshell, it’s probably safe to say that it’s not as ludicrous these days for someone without an engineering background to think about becoming a systems administrator.
System administration IS evolving. There used to be *no* configuration management tools. It used to be *completely* on the admin, and every single solution was specific to the site where it was deployed. There used to be no such thing as kickstart and jumpstart and FAI, and… the list goes on.
There used to be virtually no non-commercial solutions for monitoring that were not admin and site specific. RRDTool, MRTG, Nagios, Zabbix, Ganglia, Cacti, these people, and many others I don’t have room to name, are doing good work.
Splunk is also trying very hard to catalyze change in the way we all think about the most mundane part of administration: log analysis. Well, log analysis isn’t mundane (at least not to me), but writing code to do it is.
Google has also made many tools available that are causing administrators to think twice about how they approach deploying services, and with things like Google Apps for your domain, maybe some are thinking twice about whether they maintain particular services in-house at all.
Red Hat has open sourced Fedora Directory Server. The OpenID project is gaining traction and changing how people think about digital identity. And how could any administrator who manages large numbers of machines live without ClusterSSH? And what would beowulf administrators do without ROCKS or OSCAR?
I guess my point is made without going further (though that wouldn’t be hard to do).
The hard part about developing new tools specifically for administrators is that administrators are, many times, not programmers. I would classify myself more of a scripter than a programmer. I can whip out some quick perl or shell to do what I need. If something needs fixing (in whatever language I’m likely to run across on a day-to-day basis) I’m probably competent enough to figure out what’s going on, but that doesn’t make me a programmer. I’m not well versed in how to organize a large complex project, how to implement a proper MVC design from scratch, how to design complex and efficient algorithms to handle complex tasks. I *know* for a *fact* I’m not good at this because I once tried to start a project myself that eventually went by the wayside as another project came into existence which addressed most of the issues I was addressing, though in a much different fashion. The key there was that my project was being coded by an admin (me), and the other was being coded by a coder with direction of an admin.
As for me…
I give away all the code and information I can on my website. If you do, too, I invite you to link us to your stuff below in the comments. I’m also happy to post other information there that I didn’t write. If you have any, send it along (jonesy - linuxlaboratory - org).
I’ve contributed code to a few open source projects, and have reported bugs to even more. I try my best to pay attention to the questions I’m asked not only so I can answer the person asking, but so I can publish the questions and answers later for others to benefit from. Some of those answers actually
wound up in Linux Server Hacks, Volume Two which I was lucky enough to be invited to co-author.
My philosophy is that, if I know something, and someone else asks, I spill the beans. I tell them absolutely everything I possibly can that might be useful. My hope is that this will make the administration world a better place. Maybe something I gripe about will inspire a much more talented code slinger than myself to take action. Maybe I’ll be enlightened to another way of thinking by a more experienced administrator, and I’ll then be able to pass that information on to others. Maybe another administrator will follow my lead and from that day forward share everything he or she knows whenever possible.
In the end, I try to take the action that will make administration on the whole a better profession, and I like to believe that taking that attitude will make a difference.