High-Performance Computing at the NIH
GitHub YouTube @nih_hpc RSS Feed
Hisat on Biowulf & Helix

HISAT is a fast and sensitive spliced alignment program. As part of HISAT, we have developed a new indexing scheme based on the Burrows-Wheeler transform (BWT) and the FM index, called hierarchical indexing, that employs two types of indexes: (1) one global FM index representing the whole genome, and (2) many separate local FM indexes for small regions collectively covering the genome. ned for mapping different types of RNA-seq reads. All these together, HISAT enables extremely fast and sensitive alignment of reads, in particular those spanning two exons or more. HISAT is distributed under theGPLv3 license.

Beside the index files under /fdb/hisat, HISAT on Biowulf has been built with sra support. A local copy of the SRA Refseq reference data is maintained in /fdb/sra/refseq, and the NCBI SRA toolkit has been configured on Biowulf to :

  1. first check if the required reference data is available in the local copy
  2. if not, download the required data using the Biowulf web proxy
Thus, running hisat on the Biowulf compute nodes with SRA reference data should be seamless.

Running on Helix

helix$ module load hisat
helix$ cd /data/$USER/dir
helix$ hisat-build $HISAT_HOME/example/reference/22_20-21M.fa 22_20-21M_hisat
helix$ hisat -x 22_20-21M_hisat -U $HISAT_HOME/example/reads/reads_1.fq -S eg1.sam
helix$ hisat -x 22_20-21M_hisat -1 $HISAT_HOME/example/reads/reads_1.fq -2 $HISAT_HOME/example/reads/reads_2.fq -S eg2.sam

Running a single batch job on Biowulf

Set up a batch script along the following lines.

#!/bin/bash 

cd /data/$USER/mydir
module load hisat

hisat-build -p $SLURM_CPUS_PER_TASK $HISAT_HOME/example/reference/22_20-21M.fa 22_20-21M_hisat
hisat -p $SLURM_CPUS_PER_TASK -x 22_20-21M_hisat -U $HISAT_HOME/example/reads/reads_1.fq -S eg1.sam
hisat -p $SLURM_CPUS_PER_TASK -x 22_20-21M_hisat -1 $HISAT_HOME/example/reads/reads_1.fq -2 $HISAT_HOME/example/reads/reads_2.fq -S eg2.sam
......
......

Submit to the batch system with:

sbatch --cpus-per-task=4 --mem=10g  myscript

--cpus-per-task : threads used. This number will be assigned to $SLURM_CPUS_PER_TASK in the script automatically.
--mem : optional, use when more memory is required

Running a swarm of batch jobs on Biowulf

Set up a swarm command file (eg /data/$USER/cmdfile). Here is a sample file:

cd /data/user/mydir1; hisat-build -p $SLURM_CPUS_PER_TASK $HISAT_HOME/example/reference/22_20-21M.fa 22_20-21M_hisat; \
	hisat -p $SLURM_CPUS_PER_TASK -x 22_20-21M_hisat -U $HISAT_HOME/example/reads/reads_1.fq -S eg1.sam
cd /data/user/mydir2; hisat-build -p $SLURM_CPUS_PER_TASK $HISAT_HOME/example/reference/22_20-21M.fa 22_20-21M_hisat; \
	hisat -p $SLURM_CPUS_PER_TASK -x 22_20-21M_hisat -U $HISAT_HOME/example/reads/reads_1.fq -S eg1.sam
cd /data/user/mydir3; hisat-build -p $SLURM_CPUS_PER_TASK $HISAT_HOME/example/reference/22_20-21M.fa 22_20-21M_hisat; \
	hisat -p $SLURM_CPUS_PER_TASK -x 22_20-21M_hisat -U $HISAT_HOME/example/reads/reads_1.fq -S eg1.sam
[...]   

Submit this job with

swarm -t 4 -f cmdfile --module hisat

-t: threads used. This number will be assigned to $SLURM_CPUS_PER_TASK in the script automatically.
--module: setup required environmental variables for the job

Running an interactive job on Biowulf

Users may need to run jobs interactively sometimes. Such jobs should not be run on the Biowulf login node. Instead allocate an interactive node as described below, and run the interactive job there.

[user@biowulf]$ sinteractive --cpus-per-task=4 
      salloc.exe: Granted job allocation 1528

[user@pxxx]$ module load hisat

[user@pxxx]$ cd /data/$USER/run1

[user@pxxx]$ hisat-build -p $SLURM_CPUS_PER_TASK $HISAT_HOME/example/reference/22_20-21M.fa 22_20-21M_hisat; \
	hisat -p $SLURM_CPUS_PER_TASK -x 22_20-21M_hisat -U $HISAT_HOME/example/reads/reads_1.fq -S eg1.sam

[user@p4] exit

[user@biowulf]$ 

Benchmarks

This benchmarks used 3gb paired sequences. 16 cores is most efficient.

 

Core Time(hr:min)
1 1:45:00
2 1:05:00
4 39:50
8 21:51
16 12:46
32 11:39

Documentation

http://www.ccb.jhu.edu/software/hisat/manual.shtml#getting-started-with-hisat