I run a large data warehouse plant where we have a lot of nightly jobs running concerruently however many have dependencies on a extract or data load process before they start. Currently we use an expensive scheduling system to scehdule these at the moment.
Is there any way you can setup job dependencies using an open source or free unix/linux tool such as cron?
Moving to an open soruce solution would be great and save us lots!
Regards Matt
I asked a similar question last year (maybe Serverfault would be a better place these days?). There doesn t seem to be a simple, install-and-go solution unfortunately.
Cfengine can be made to do something like this. You can set it up as a cron replacement, running arbitrary commands at scheduled times, and you can set up "classes" so that certain actions are performed only if certain classes are enabled. Classes can be anything from "this is a Linux system" to "it s currently between 5 and 10 minutes after the hour" to "system load is above value x" to "this arbitrary shell command that I just specified returned true", so you could set up your classes to indicate your job dependencies.
I doubt that this would be as powerful as a scheduling system (dependencies would have to be set up manually by configuring classes, scheduling concurrently would requires extra scripting or configuration work), but it is free and open source.
Version 2 of Cfengine was not particularly pleasant to work with (in the words of Seth Vidal, "it s [sic] syntax kills kittens"). I haven t used Cfengine 3. Puppet has similar design goals as Cfengine and may be easier to work with.
Cron doesn t handle this natively. Can the process that loads the data write out a status file upon completion? This would allow subsequent jobs to check the status file before doing their real work. Obviously, this isn t an ideal solution (too many points of failure, I suspect), but perhaps it s good enough for what you re trying to accomplish.
Schedulix is an open source workload automation solution for Linux: http://www.schedulix.org
For security reasons, it is desirable to check the integrity of code before execution, avoiding tampered software by an attacker. So, my question is How to sign executable code and run only trusted ...
How can I check the file encoding in a shell script? I need to know if a file is encoded in utf-8 or iso-8859-1. Thanks
i would like to know if there are any solution to do this. Does anyone? The big picture: I want to access data over the web, using my delphi thin clients. But i´would like to keep my server/service ...
Since cruise control is full of bugs that have wasted my entire week, I have decided the existing shell scripts I have are simpler and thus better. Here is what I have so far svn update /var/www/...
I ve just installed Zend Studio 7.0.2 on my Linux-Ubuntu 9.10 system. There were no problems during the installation but when I try to create a new project, the New Project form hangs when I click ...
I am running valgrind as follows:- /usr/local/bin/valgrind "process_name" After excecution its giving me following error ==21731== ==21731== Warning: Can t execute setuid/setgid executable: ==...
I was wondering if there were any good free graphics libraries for C that are easy to use? It s for plotting 2d and 3d graphs and then saving to a file. It s on a Linux system and there s no gnuplot ...
Is there anything other than DDD that will draw diagrams of my data structures like DDD does that runs on Linux? ddd is okay and runs, just kind of has an old klunky feeling to it, just wanted to ...
