Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

...

With useTmp, the pipeline runner copy related data to /tmp and all file paths will be automatically updated to reflect a file's location in /tmp when using the useTmp option. 

Sample output from the test run

...

Code Block
linenumberstrue
cd proper/directory
module load rcbio/1.1 and all/related/modules


#submit job with proper partition, time, number of cores and memory
sbatch --requeue --mail-type=ALL -p short -t 2:0:0 -c 2 --mem 2G /working/directory/flag/stepID.loopID.stepName.sh


Or you can also submit like: 
https://wiki.rc.hms.harvard.edu/display/O2/Get+more+informative+slurm+email+notification+and+logs

To run your own script as Slurm pipeline

...

In case you wonder how it works, here is a simple example to explain:

For each step per loop, the pipeline runner creates a file looks like this (here it is named flag.sh): 

Code Block
linenumberstrue
#!/bin/bash 
srun -n 1 bash -c "{ echo I am running...; hostname; otherCommands; } && touch flag.success" 
sleep 5 
export SLURM_TIME_FORMAT=relative 
echo Job done. Summary: 
sacct --format=JobID,Submit,Start,End,State,Partition,ReqTRES%30,CPUTime,MaxRSS,NodeList%30 --units=M -j $SLURM_JOBID 
sendJobFinishEmail.sh flag 
[ -f flag.success ] && exit 0 || exit 1 

Then submit with: 

Code Block
linenumberstrue
sbatch -p short -t 10:0 -o flag.out -e flag.out flag.sh

...