Checking the Job status and node status
Job Status
Job-status can be checked using the squeue command, adding -u $USER to display specific jobs. Array jobs can be displayed using --array flag. Some frequent states are abbreviated as follows:
Job State Code | Description | |
PD | Pending | Job is awaiting resource allocation |
R | Running | Currently has allocation |
CG | Completing | Job is in the process of completing |
F | Failed | Job terminated with non-zero exit code |
OOM | Out_of_memory | Job ran out of memory |
RQ | Requeue | Job is being requeued |
For more detailed information on any queued job the user can use the scontrol show job <job_id> command to get further details on the expected start time, priority and other details. Array jobs can also be queried using the same command as <job_id>_<array_id>.
Sattach
Standard input and output of the job can be redirected to the user's terminal using sattach as follows:
> sattach jobid.jobstepid
Controlling Job Execution
Jobs can be canceled using the scancel <job_id> command. The attributes of a queued job can be changed using the scontrol update job <job_id> <attribute>=<value>. Few examples are given below:
> scontrol update job 1650 JobName=NewName
> scontrol update job 1650 TimeLimit=1-3:00:00
> scancel 1650
scontrol can also be used to hold, release and requeue jobs.