Phenyx perl scripts - batchsystem 2.5
From GBWiki
The SBS Batch queuing system
Contents |
Scheduler interaction
explore status
to print the status
$(phenyxvar perlscripts)/sbs-scheduler-print.pl --config=$(phenyxvar batch.configfile) | less
check for dead jobsa
$(phenyxvar perlscripts)/sbs-batch-check.pl --config=$(phenyxvar batch.configfile)
remove a job
Remove (and possibly kill if it is active)
$(phenyxvar perlscripts)/sbs-batch-remove.pl --config=$(phenyxvar batch.configfile) id1 [id2[...]]
Additional options
-
--isfinished<code> to avoid removing job if they are not finished
Resubmiting a list of failed phenyx job
Consider we have in $l a list of failed job id, which all failed during the preocessing step (mMPD was down for example), and we want to resubmit them all through the batch system
Of course, if we have only one, the scheduler is not important and we can simple use the job-execute.pl command
An example of building a list of failed job
I want to have the list of jobs where the final error contains "mpd", among the 100 last submitted jobs
cdjob l=$(for i in $(ls -1rtd [0-9]* | tail -200 ) ; do if [ "$(tail -2 $i/runstatus.txt | grep mpd)" ] ;then echo $i; fi; done )
remove error flag and set a special flag
So that rescue.xml is moved and the status flag is set to warn the user
for i in $l; do $(phenyxvar perlscripts)/manage/job-status.pl --id=$i --set="pending" --comment "being reprocessed" --cancelerror done
resubmission
As they failed during processing, we must relaunch processing and postprocessing. We can use prioritary queues for that.
for i in $l; do echo $i gunzip $(phenyxvar results)/$i/idj.xml.gz bid=$( $(phenyxvar perlscripts)/sbs-batch-submit.pl --config=$(phenyxvar batch.configfile) --command=$(phenyxvar results)/$i/batch/process-$i-0.pl --queue=cluster_dev_default_high) ($(phenyxvar perlscripts)/sbs-batch-submit.pl --config=$(phenyxvar batch.configfile) --command=$(phenyxvar results)/$i/batch/postprocess-$i-0.pl --queue=phenyxmaster_default --onfinished=$bid &) done
