Online class: Introduction to Biowulf

Hands-On: Job Dependencies

In the previous Data Storage hands-on section, you should have copied the class scripts to your /data area. If you skipped or missed that section, type

hpc-classes biowulf

now. This command will copy the scripts and input files used in this online class to your /data area, and will take about 5 minutes.


cd /data/$USER/hpc-classes/biowulf/dependencies

# submit the first job
sbatch Job1.bat

# select or note down the job ID for this job
# then submit a second job
sbatch --depend=afterany:Job1num Job2.bat

# sjobs will show you the dependencies
sjobs

# note down or select the number of the second job
# submit a 3rd job dependent on Job1 as well as Job2 completing
sbatch --depend=afterany:Job1num:Job2num Job3.bat

# sjobs will show you the dependencies
sjobs

# note down or select the number of the third job
# submit a swarm dependent on Job3 
swarm -g 2 --depend=afterany:Job3num -f swarm.cmd

Quiz

Noting down job IDs is annoying, and subject to error. Can this be scripted?: Answer

Yes. See examples in bash, perl and python at Bulding pipelines using slurm