Submitting an array job

For some applications, you want to submit the same job script repeatedly. In these situations, you can submit an array job. For example,

sbatch -a 1-100 my_job.sh

submits the job script my_job.sh a hundred times. In principle, these 100 jobs are identical, however, inside the job script my_job.sh, you can use the environment variable $SLURM_ARRAY_TASK_ID to customize what each of these 100 jobs does. E.g. you could start your program each time with different input files (input_1, input_2, etc) and write to different output files (output_1, output_2, etc) using a line like

$HOME/my_program input_$SLURM_ARRAY_TASK_ID output_$SLURM_ARRAY_TASK_ID

in your job script to run your program.

Note that the maximum size of one array job is 1000.

Deleting (elements of) an array job

To remove all jobs from an array, use

scancel <jobid>

where jobid is the job ID of the array job.

To remove elements from an array job 1234, e.g. elements 4, 6 and 8, use

scancel 1234_4 1234_6 1234_8
  • No labels