Cactus is a reference-free whole-genome multiple alignment program.
Allocate an interactive session and run the program. Sample session:
[user@biowulf]$ sinteractive --cpus-per-task=4 --gres=lscratch:8
salloc.exe: Pending job allocation 46116226
salloc.exe: job 46116226 queued and waiting for resources
salloc.exe: job 46116226 has been allocated resources
salloc.exe: Granted job allocation 46116226
salloc.exe: Waiting for resource configuration
salloc.exe: Nodes cn4224 are ready for job
[user@cn4224 ~]$ module load cactus
[+] Loading cactus  1.2.3  on cn4224 
[+] Loading singularity  3.7.0  on cn4224
[user@cn4224 ~]$ cd /lscratch/${SLURM_JOB_ID}
[user@cn4224 ~]$ export SINGULARITYENV_TMPDIR=/lscratch/${SLURM_JOB_ID}
[user@cn4224 ~]$ wget https://raw.githubusercontent.com/ComparativeGenomicsToolkit/cactus/master/examples/evolverMammals.txt
[user@cn4224 ~]$ cactus /lscratch/${SLURM_JOB_ID}/jobStore \
       /lscratch/${SLURM_JOB_ID}/evolverMammals.txt \
       /data/${USER}/evolverMammals.hal \
       --root mr \
       --binariesMode local
[...]
Workflow Progress 100%||||||||||||||||||||||| 379/379 (0 failures) [19:08<00:00, 0.33 jobs/s]
[2020-12-23T13:47:39-0500] [MainThread] [I] [toil.common] Successfully deleted the job store: FileJobStore(/lscratch/46116226/jobStore)
[2020-12-23T13:47:39-0500] [MainThread] [I] [cactus.progressive.cactus_progressive] Cactus has finished after 1156.1930517529836 seconds
[user@cn4224 ~]$ exit
salloc.exe: Relinquishing job allocation 46116226
[user@biowulf ~]$
Create a batch input file (e.g. cactus.sh) similar to the following.
#! /bin/bash
set -e
module load cactus
cd /lscratch/${SLURM_JOB_ID}
export SINGULARITYENV_TMPDIR=/lscratch/${SLURM_JOB_ID}
cactus /lscratch/${SLURM_JOB_ID}/jobStore \
       /lscratch/${SLURM_JOB_ID}/evolverMammals.txt \
       /data/${USER}/evolverMammals.hal \
       --root mr \
       --binariesMode local
Submit these jobs using the Slurm sbatch command.
Create a swarmfile for the first step of the pipeline (e.g. cactus.swarm). For example:
cd /lscratch/${SLURM_JOB_ID}; \
export SINGULARITYENV_TMPDIR=/lscratch/${SLURM_JOB_ID}; \
cactus /lscratch/${SLURM_JOB_ID}/jobStore \
       /lscratch/${SLURM_JOB_ID}/file1.txt \
       /data/${USER}/file1.hal \
       --root mr \
       --binariesMode local
cd /lscratch/${SLURM_JOB_ID}; \
export SINGULARITYENV_TMPDIR=/lscratch/${SLURM_JOB_ID}; \
cactus /lscratch/${SLURM_JOB_ID}/jobStore \
       /lscratch/${SLURM_JOB_ID}/file2.txt \
       /data/${USER}/file2.hal \
       --root mr \
       --binariesMode local
Submit this job using the swarm command.
swarm -f cactus.swarm [-g #] --module cactuswhere
| -g # | Number of Gigabytes of memory required for each process (1 line in the swarm command file) | 
| --module cactus | Loads the cactus module for each subjob in the swarm |