Batch small jobs together as a big job

 

Many bioinformatics workflows run the same command on multiple files. When the file sizes are small, the command could only take a few seconds to finish.If you submit the processing of each file as a job often causes the job scheduler to complain of short-running jobs.

Here we will show you how to batch multiple small jobs into a large job with an example.

You can cut and paste the commands below onto the O2 command line to get the idea how it works.

Note: When copying/pasting commands, you can include any text starting with #. They will be ignored by Linux.

Log on to O2

If you need help connecting to O2, please review the