Batch small jobs together as a big job
Many bioinformatics workflows run the same command on multiple files. When the file sizes are small, the command could only take a few seconds to finish.If you submit the processing of each file as a job often causes the job scheduler to complain of short-running jobs.
Here we will show you how to batch multiple small jobs into a large job with an example.
You can cut and paste the commands below onto the O2 command line to get the idea how it works.
Note: When copying/pasting commands, you can include any text starting with #. They will be ignored by Linux.