Submitting SLURM Jobs

Batch Job

A batch job can be launched using sbatch command and with a script. The most common options for a batch job are:

-N <#>/ --nodes=<#> -- request nodes

-n <#> / --tasks=<#> -- request tasks (e.g. cores)

-c <#> / --cpus-per-task=<#> -- request # cpus per task

--ntasks-per-node=<#> -- request # tasks/cores per node

A few example scripts are given below.

1. 32 core OpenMP job:

#!/bin/tcsh
#SBATCH --job-name=parallel
#SBATCH --nodes=1 --cpus-per-task=32
#SBATCH -t 30:00
setenv OMP_NUM_THREADS 32
srun ./a.out

2. 2 node hybrid parallel job (32 cores on node, 2 mpi nodes):

#!/bin/tcsh
#SBATCH --job-name=hybrid
#SBATCH --nodes=2 --ntasks=2 --cpus-per-task=32
#SBATCH -t 30:00

setenv OMP_NUM_THREADS 32

srun ./a.out

3. 1 node packed with multiple serial jobs

#!/bin/tcsh

#SBATCH --job-name=array1
#SBATCH --output=job_%A_%a.out
#SBATCH --error=job_%A_%a.err
#SBATCH --time=01:00:00
#SBATCH -N 1
#SBATCH --ntasks-per-node=32

srun -n 1 --exclusive --output=job_%A_%a_1.out ./a1.out &
srun -n 1 --exclusive --output=job_%A_%a_2.out ./a2.out &
srun -n 1 --exclusive --output=job_%A_%a_3.out ./a3.out &
srun -n 1 --exclusive --output=job_%A_%a_4.out ./a4.out &
wait

4. Array job with unique inputs - Please note that argument to --input is used instead of standard input
g
#!/bin/tcsh
#SBATCH --job-name=arrayj
#SBATCH --output=array_%A_%a.out
#SBATCH --error=array_%A_%a.err
#SBATCH --time=1-1:00
#SBATCH --array=1-4
#SBATCH -N 1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=32

srun -N1 -c32 --input=input%a --exclusive ./ppl.py

Interactive Job

>> salloc -N 1 -n 32

>> salloc -N 1 -n 32 --x11

Input/Output Formatting

Given below are some useful specifiers that can be used to distinguish outputs

String	Formatting
//	Cancel replacement
%A	Job id
%a	Array index
%x	Name