...
...
...
...
...
...
...
...
|
This page shows you how to run a regular bash script as a pipeline. The runAsPipeline script, accessible through the rcbio/1.0 module, converts an input bash script to a pipeline that easily submits jobs to the Slurm scheduler for you.
...
With useTmp, the pipeline runner copy related data to /tmp and all file paths will be automatically updated to reflect a file's location in /tmp when using the useTmp option.
Sample output from the test run
...
In case you wonder how it works, here is a simple example to explain:
For each step per loop, the pipeline runner creates a file looks like this (here it is named flag.sh):
| Code Block | ||
|---|---|---|
| ||
#!/bin/bash
srun -n 1 bash -c "{ echo I am running...; hostname; otherCommands; } && touch flag.success"
sleep 5
export SLURM_TIME_FORMAT=relative
echo Job done. Summary:
sacct --format=JobID,Submit,Start,End,State,Partition,ReqTRES%30,CPUTime,MaxRSS,NodeList%30 --units=M -j $SLURM_JOBID
sendJobFinishEmail.sh flag
[ -f flag.success ] && exit 0 || exit 1 |
Then submit with:
| Code Block | ||
|---|---|---|
| ||
sbatch -p short -t 10:0 -o flag.out -e flag.out flag.sh |
...