NAME rq v0.1.5 SYNOPSIS rq [queue] mode [mode_args]* [options]* DESCRIPTION rq is an __experimental__ tool used to manage nfs mounted work queues. multiple instances of rq running from multiples hosts can work from these queues to distribute processing load to 'n' nodes - bringing many dozens of otherwise powerful cpus to their knees with a single blow. clearly this software should be kept out of the hands of radicals, SETI enthusiasts, and one mr. jeff safran. rq operates in one of the modes create, submit, feed, list, delete, query, snapshot, or help. depending on the mode of operation and the options used the meaning of mode_args may change, sometime wildly and unpredictably (i jest, of course). MODES abbreviations for many modes exist c => create s => submit f => feed l => list d => delete q => query p => snapshot h => help create, c : creates a queue. the queue MUST be located on an nfs mounted file system visible from all nodes intended to run jobs from it. examples : 0) to create a queue ~ > rq q create or simply ~ > rq q c list, l : show combinations of pending, running, dead, or finished jobs. for this command mode_args must be one of pending, running, dead, finished, or all. the default is all. mode_args may be abbreviated to uniqueness, therefore the following shortcuts apply : p => pending r => running f => finished d => dead a => all examples : 0) show everything in q ~ > rq q list all or ~ > rq q l all or ~ > export RQ_Q=q ~ > rq l 0) show q's pending jobs ~ > rq q list pending 1) show q's running jobs ~ > rq q list running 2) show q's finished jobs ~ > rq q list finshed submit, s : submit jobs to a queue to be proccesed by any feeding node. any mode_args are taken as the command to run. note that mode_args are subject to shell expansion - if you don't understand what this means do not use this feature. when running in submit mode a file may by specified as a list of commands to run using the '--infile, -i' option. this file is taken to be a newline separated list of commands to submit, blank lines and comments (#) are allowed. if submitting a large number of jobs the input file method is MUCH more efficient. if no commands are specified on the command line rq automatically reads them from STDIN. yaml formatted files are also allowed as input (http://www.yaml.org/) - note that the output of nearly all rq commands is valid yaml and may, therefore, be piped as input into the submit command. the '--priority, -p' option can be used here to determine the priority of jobs. priorities may be any number (0, 10]; therefore 9 is the maximum priority. submitting a high priority job will NOT supplant currently running low priority jobs, but higher priority jobs WILL always migrate above lower priority jobs in the queue in order that they be run as soon as possible. note that constant submission of high priority jobs may create a starvation situation whereby low priority jobs are never allowed to run. avoiding this situation is the responsibility of the user. examples : 0) submit the job ls to run on some feeding host ~ > rq q s ls 1) submit the job ls to run on some feeding host, at priority 9 ~ > rq -p9 q s ls 2) submit 42000 jobs (quietly) to run from a command file. ~ > wc -l cmdfile 42000 ~ > rq q s -q < cmdfile 3) submit 42 jobs to run at priority 9 from a command file. ~ > wc -l cmdfile 42 ~ > rq -p9 q s < cmdfile 4) re-submit all finished jobs ~ > rq q l f | rq q s feed, f : take jobs from the queue and run them on behalf of the submitter. jobs are taken from the queue in an 'oldest highest priority' order. feeders can be run from any number of nodes allowing you to harness the CPU power of many nodes simoultaneously in order to more effectively clobber your network, anoy your sysads, and send output raids up in smoke. the most useful method of feeding from a queue is to do so in daemon mode so that if the process loses it's controling terminal and will not exit when you exit your terminal session. use the '--daemon, -d' option