Some of the hosts I administer do nothing but batch jobs. On any one of those hosts, I will find cron entries like the following:
* * * * * /opt/application/bin/batch1.pl * * * * * /opt/application/bin/process.pl /tmp/foo * * * * * /opt/application/bin/process.pl /tmp/bar * * * * * /opt/application/bin/submit.pl
In case you don’t read crontab, every program is instructed to run once a minute, at the top of the minute.
Cron is an okay scheduler, but it isn’t a perfect scheduler. It’s limited in its scheduling options, it doesn’t care if dual copies of the same program are running in conflict, and there is no way to assign priorities
I talked to one of my developers and suggested ways in which to improve his program.
The biggest problem is overhead. When each job starts, the perl interpreter starts up, it reads the script file, compiles the code, and then executes it. Depending on the job, that could be a delay of one or two real world seconds, and it will be a highly CPU intensive process.
It’s a given that the program needs to run every minute, the simplest thing to do is have the script execute, and then sleep for 60 seconds.
# main execution block
while (1) {
runTask()
sleep 60;
}
Or, if you really need to start on the zero second….
# main execution block
while (1) {
runTask()
sleep 1 # we need this to prevent the task from running more than once during the 0 second.
while (((localtime(time))[0] != 0) {
sleep 1;
}
}
To execute the script, it would be started from an RC script on host boot-up. It would then run in the background, looping through its routine once a minute.
The second and third cron jobs in the crontab are the same program, but they are performing the same operation on different files. If the code was re-written to handle the files in sequence, it would eliminate one process. Or, if the processes needed to be handled in parallel, two background processes could be initiated from the RC script.
For a test, we checked out the execution times of the processing routine. If it took longer than 60 seconds to run, then cron could have started up multiple copies of the same script, leading to potential conflicts. If it was under 60 seconds, we had available resources that were never being used. In one case, we changed the sleep delay from 60 seconds to 30 seconds giving us more processing iterations per hour.
Another advantage I learned from this experiment was that it forced the developer to become a better programmer. In the newer version of the script, filehandles were properly closed, and arrays that were ignored at exit, are now initialized at the begining of each loop iteration.
The program also has to do better error handling, because if it crashes during an iteration, there is no scheduler to restart it. Actually, that isn’t 100% true, because if I had the process run an an inittab entry, or if the OS had a process management environment, I could have the job restart during a failure. But we felt it was better that the program should die with some form of external warning, so developers can better monitor the situation.
I use supervise to do this
sort of work. Since we run qmail, we are used to these tools, and modifying
the setup to be a cron replacement works remarkably well.
When I have a job running under supervise, it never dies and there is only
ever one version of it running.
It's not ideal for all jobs, but it is worth looking at before you go too far down the re-implementation road.
how to automate backup scheduler under windows. i can able to create task for backup,but i want the backup wizard to run automatically through perl code. help me?
christmas world