I’ve seen more than a couple of sites in the past where there are teams of administrators who work together to maintain the system and/or network infrastructure, or the data management infrastructure, or whatever. On these teams there’s often a lot of task overlap even when the team is made up of specialists. For example, it might be the case that no single person on the team is “in charge” of DNS. If there’s a DNS-related issue, whoever sees it first, or whoever is on call, is the person who deals with it.
Inevitably, whoever is dealing with the issue today seems to have a script they’re not very proud of that does something useful that helps them solve some problem that comes up, well, often enough that there’s a script to help deal with it.
The key words these admins tend to use to describe these scripts are “hack”, “brute force”, “spaghetti code”, “inelegant”, “5-liner”, and “quick-n-dirty”. This is by no means a complete list - other slang and colloquialisms abound. They often try to rationalize the hack’s creation by following up with something that makes them seem witty. A comment like “I’m lazy, so I just hacked it together one day”. Putting aside the condescending tone of the comment (it implies that you’re not nearly as “lazy” or witty as they are), the fact of the matter is that the truly lazy among us will find a way to *not* write *any* code if they don’t have to, and minimize the code we ever have to write. Ever.
How? Well, one way is to take advantage of two common admin traits: a natural tendency toward sloth, and their ability to write and improve code to do useful things. See, in a lot of groups, where there’s task overlap, there’s also code redundancy. In other words, everyone has their own hack they coded in a hurry to do basically the same exact task…. because they’re lazy.
If you’re in a group with lots of disparate and probably redundant code, you might find an unlikely friend in CVS. You can create a repository called “adminstuff”, and under it, import modules representing a problem domain, a service, or something else that makes sense for your needs. While you’re at it, go ahead and create modules for the various directories around your environment that are full of code that’s either managed using something local to the machine like RCS, or not managed at all. This has several benefits:
First, stuff that’s not managed, or is managed using a facility on the local machine is now in a central location. This is nice because, if the machine croaks on you one day, you don’t have to go to back ups - you can instead just do a checkout of the module to another box and get back to work.
Second, everyone winds up writing less code, because instead of having a script per person per task, there’s just a script per task that anyone/everyone in the group can work to improve. This increases the chances that the code will be *less* hackish than it once was.
Third, if you create a read-only user to check out the code, and read-write accounts for the developers in the group, then you get some level of accountability for free, because each person will have to use their own credentials to commit changes to the code.
Fourth, I forgot to mention the ability to rollback to earlier versions of the code if something breaks!
Fifth, and this is one of my favorites, it means you don’t have to ssh to a machine, su to root (or remember to use sudo), and edit the code as root, which just feels dirty to me. Now you can just check the module out to your workstation, work in your own development environment, and check it into CVS when you’re done. This means I don’t have to think about whether I’m using the Solaris vi or the Linux vim install, and I can even use my shiny new IDE I found if I want to. This is a little more convenient than scp’ing code around, or using a root account to copy it to a user directory and chown’ing it or some of the other hacks I’ve seen (and even used once or twice) to get around limitations of not having some code management mechanism in place.
Finally, moving and organizing admin scripting tasks in this way may naturally lead to the creation of an administrative API for your environment, which means everyone writes much, much less code. For example, I grabbed a “hack” someone told me about the other day, and after editing it to take advantage of our API, I was able to cut the line count of that “hack” in half, while simultaneously adding more checks to the operation, making it more robust.
There are other benefits, of course, like the ability to write the enforcement of certain data handling or task-related policies into the single, unified code base instead of hoping everyone is following the policy in whatever hacks they’re using. I’ve also found that I work on more code, because it’s more convenient to work on. If that feeling catches on in your environment, how can in *not* improve how things get done?
If you’re an admin who is new to CVS, I’ve taken some time to write up a CVS cheat sheet that covers some things you might find useful. Enjoy!