High-Performance Computing at the NIH
GitHub YouTube @nih_hpc RSS Feed
MIXCR on Biowulf and Helix

MiXCR is a universal software for fast and accurate analysis of T- and B- cell receptor repertoire sequencing data.

Running on Helix

Sample session:

helix$ module load mixcr
helix$ cd /data/$USER/dir
helix$ mixcr align -r log.txt -t 4 \
		/data/maoj/seq/ngs/2_1_62B43AAXX.192_BUSTARD-2010-11-01.fastq.gz \
		/data/maoj/seq/ngs/2_2_62B43AAXX.192_BUSTARD-2010-11-01.fastq.gz \
		aln.vdjca

Note the '-t' flag is required otherwise the program will occupy all available cpus on helix and crash the system.

Submitting a single batch job

1. Create a script file. The file will contain the lines similar to the lines below.

#! /bin/bash 
#SBATCH --cpus-per-task=20 
#SBATCH --mail-type=BEGIN,END,FAIL 

module load mixcr 
cd /data/maoj/mixcr 
mixcr align -r log.txt -t $SLURM_CPUS_PER_TASK \
/data/maoj/seq/ngs/2_1_62B43AAXX.192_BUSTARD-2010-11-01.fastq.gz \
/data/maoj/seq/ngs/2_2_62B43AAXX.192_BUSTARD-2010-11-01.fastq.gz \
aln.vdjca
....
....

2. Submit the script on Biowulf.

$ sbatch --cpus-per-task=20 myscript

--cpus-per-task is a must for this program otherwise it will auto thread to all available cpus on a node

This job took 40 minutes when allocated 20 threads, 7 hours when allocated 2 threads. The input files are 4gb in size each.

see biowulf user guide for more options such as allocate more memory and longer walltime if needed.

Submit a swarm of jobs

Using the 'swarm' utility, one can submit many jobs to the cluster to run concurrently.

Set up a swarm command file (eg /data/$USER/cmdfile). Here is a sample file:

cd /data/user/run1/; mixcr align -r log.txt -t $SLURM_CPUS_PER_TASK 1.fastq.gz 2.fastq.gz aln.vdjca
cd /data/user/run2/; mixcr align -r log.txt -t $SLURM_CPUS_PER_TASK 1.fastq.gz 2.fastq.gz aln.vdjca
cd /data/user/run3/; mixcr align -r log.txt -t $SLURM_CPUS_PER_TASK 1.fastq.gz 2.fastq.gz aln.vdjca
........

The -f flag is required to specify swarm file name.

Submit the swarm job:

$ swarm -f swarmfile -t 20 --module mixcr

-t flag is a must for this program otherwise it will auto thread to all available cpus on a node.

- Use -t flag for more threads requirement (default 2 threads per line in swarmfile)

- Use --time flag for longer walltime

For more information regarding running swarm, see swarm.html

 

Running an interactive job

User may need to run jobs interactively sometimes. Such jobs should not be run on the Biowulf login node. Instead allocate an interactive node as described below, and run the interactive job there.

Note, --cpus-per-task is a must for the program otherwise it will auto thread to all available cpus on the node.

[user@biowulf]$ sinteractive --cpus-per-task=20

[user@pXXXX]$ cd /data/$USER/myruns

[user@pXXXX]$ module load mixcr

[user@pXXXX]$ mixcr align -r log.txt -t $SLURM_CPUS_PER_TASK 1.fastq.gz 2.fastq.gz aln.vdjca
[user@pXXXX]$ exit
slurm stepepilog here!
                   
[user@biowulf]$ 

Documentation

https://mixcr.readthedocs.org/en/latest/