krakenuniq on Biowulf

Krakenuniq is a system for assigning taxonomic labels to short DNA sequences, usually obtained through metagenomic studies. It uses exact matches of kmers against a reference database. It is an upgraded form of Kraken version 1.

References:

Do not run Kraken with the database in the default /fdb location or a shared /data directory. It must be run with the database in /lscratch or /dev/shm.

Documentation
Important Notes

Interactive job
Interactive jobs should be used for debugging, graphics, or applications that cannot be run as batch jobs.

Allocate an interactive session and run the program. In the following sample session we run a minimal classification. See the documentation for Kraken (version 1) for more detailed guidance on custom databases.

[user@biowulf]$ sinteractive --mem=24g --cpus-per-task=4 --gres=lscratch:100
salloc.exe: Pending job allocation 46116226
salloc.exe: job 46116226 queued and waiting for resources
salloc.exe: job 46116226 has been allocated resources
salloc.exe: Granted job allocation 46116226
salloc.exe: Waiting for resource configuration
salloc.exe: Nodes cn3144 are ready for job

[user@cn3144 ~]$ module load krakenuniq/1.0.4

###
### Copy database into lscratch
###
[user@cn3144 ~]$ cd /lscratch/$SLURM_JOB_ID

[user@cn3144 ~]$ cp -r $KRAKEN_DB/20200428_bacteria_32GB .

###
### Classification
###
[user@cn3144 ~]$ cp -r $KRAKEN_TEST_DATA/accuracy .
[user@cn3144 ~]$ cd accuracy
[user@cn3144 ~]$ ls -lh
total 7.5M
-rw-rw-r-- 1 user group 2.2K Aug 14 09:55 ACCURACY.README
-rw-rw-r-- 1 user group 1.2M Aug 14 09:55 HiSeq_accuracy.fa
-rw-rw-r-- 1 user group 1.8M Aug 14 09:55 MiSeq_accuracy.fa
-rw-rw-r-- 1 user group 4.6M Aug 14 09:55 simBA5_accuracy.fa
-rw-rw-r-- 1 user group  396 Aug 14 09:55 taxonomyDocSum.xslt
[user@cn3144 ~]$ kraken --db ../20200428_bacteria_32GB \
                        --output HiSeq.kraken \
                        HiSeq_accuracy.fa
Taxonomy database not at ./20200428_bacteria_32GB/taxDB - creating it .../usr/local/apps/krakenuniq/1.0.4/bin/build_taxdb ./20200428_bacteria_32GB/taxonomy/names.dmp ./20200428_bacteria_32GB/taxonomy/nodes.dmp > ./20200428_bacteria_32GB/taxDB
Building taxonomy index from ./20200428_bacteria_32GB/taxonomy/nodes.dmp and ./20200428_bacteria_32GB/taxonomy/names.dmp. Done, got 2242898 taxa
/usr/local/apps/krakenuniq/1.0.4/bin/classify -d ./20200428_bacteria_32GB/database.kdb -i ./20200428_bacteria_32GB/database.idx -o HiSeq.krakenuniq -a ./20200428_bacteria_32GB/taxDB -p 12
 Database ./20200428_bacteria_32GB/database.kdb
Loaded database with 2505397588 keys with k of 31 [val_len 4, key_len 8].
Reading taxonomy index from ./20200428_bacteria_32GB/taxDB. Done.
Writing Kraken output to HiSeq.krakenuniq
10000 sequences (0.92 Mbp) processed in 0.677s (886.5 Kseq/m, 81.78 Mbp/m).
  8586 sequences classified (85.86%)
  1414 sequences unclassified (14.14%)
Finishing up ...

[user@cn3144 ~]$ head -5 HiSeq.krakenuniq
C       A_hydrophila_HiSeq.922  644     101     0:6 1224:1 0:3 1224:1 0:16 1224:1 0:2 1224:1 0:4 1224:1 0:6 642:1 0:8 644:1 0:19
C       A_hydrophila_HiSeq.1263 1236    101     0:2 2:1 0:12 2:1 0:9 1236:1 0:6 1236:2 0:37
C       A_hydrophila_HiSeq.2905 1448139 101     0:8 1448139:1 0:5 1448139:1 0:16 1448139:1 0:24 642:1 0:14
C       A_hydrophila_HiSeq.4866 642     101     0:4 642:2 0:8 642:1 0:6 642:1 0:2 642:2 0:5 642:1 0:5 642:1 0:5 642:1 0:14 642:1 0:4 642:1 0:2 642:1 0:4
C       A_hydrophila_HiSeq.7009 642     101     0:49 571:1 0:14 1224:1 0:4 642:2

### CLEANUP
[user@cn3144 ~]$ rm -rf /dev/shm/custom_db

The same approach of copying to /dev/shm can be taken with the databases from /fdb/kraken as long as they fit. Note that /dev/shm as a whole is limited to 50% of memory and per job by the memory allocation. For example, using a reduced size 'minikraken' database obtained from the kraken maintainers:

[user@cn3144 ~]$ cp -r /fdb/kraken/20180220_standard_8GB /dev/shm
[user@cn3144 ~]$ kraken --db /dev/shm/20180220_standard_8GB --fasta-input \
                        --output HiSeq.kraken.2 HiSeq_accuracy.fa
10000 sequences (0.92 Mbp) processed in 0.602s (997.0 Kseq/m, 91.96 Mbp/m).
  7523 sequences classified (75.23%)
  2477 sequences unclassified (24.77%)

Batch job
Most jobs should be run as batch jobs.

Create a batch input file (e.g. krakenuniq.sh) which copies a reduced size standard database to /lscratch or /dev/shm and runs several fastq samples against a local copy of the database.

#! /bin/bash
# USAGE: krakenuniq.sh dbname infile [infile ...]

module load krakenuniq/1.0.4
db=$1
dbname=$(basename $1)
shift
[[ ${1:-none} != "none" ]] || exit 1

# make sure the database in /dev/shm is cleaned up
trap 'rm -rf /dev/shm/${dbname}' EXIT

[[ -d ${db} ]] && cp -r $db /dev/shm || exit 1

for infile in "$@"; do
    [[ -f ${infile} ]] \
    && krakenuniq --db /dev/shm/${dbname} --threads ${SLURM_CPUS_PER_TASK} \
        --output ${infile}.kraken.gz
done

Submit this job using the Slurm sbatch command.

sbatch --cpus-per-task=16 --mem=38g krakenuniq.sh \
    /fdb/kraken/20171019_standard_32GB sample0[0-9].fastq.gz
Swarm of Jobs
A swarm of jobs is an easy way to submit a set of independent commands requiring identical resources.

Create a swarmfile (e.g. krakenuniq.swarm) that uses the script above to process several fastq files in each job

krakenuniq.sh /fdb/kraken/20200429_standard_16GB sample0[0-9].fastq.gz
krakenuniq.sh /fdb/kraken/20200429_standard_16GB sample1[0-9].fastq.gz
krakenuniq.sh /fdb/kraken/20200429_standard_16GB sample2[0-9].fastq.gz

Submit this job using the swarm command.

swarm -f krakenuniq.swarm -g 72 -t 16 --module krakenuniq/1.0.4
where
-g # Number of Gigabytes of memory required for each process (1 line in the swarm command file)
-t # Number of threads/CPUs required for each process (1 line in the swarm command file).
--module krakenuniq Loads the kraken module for each subjob in the swarm