Biowulf High Performance Computing at the NIH
Neusomatic on Biowulf

NeuSomatic is based on deep convolutional neural networks for accurate somatic mutation detection. With properly trained models, it can robustly perform across sequencing platforms, strategies, and conditions. NeuSomatic summarizes and augments sequence alignments in a novel way and incorporates multi-dimensional features to capture variant signals effectively. It is not only a universal but also accurate somatic mutation detection method.

Documentation
Important Notes

Interactive job
Interactive jobs should be used for debugging, graphics, or applications that cannot be run as batch jobs.

Allocate an interactive session and run the program. Sample session:

[user@biowulf]$ sinteractive --gres=gpu:k80:1,lscratch:10 --mem=20g -c14
salloc.exe: Pending job allocation 46116226
salloc.exe: job 46116226 queued and waiting for resources
salloc.exe: job 46116226 has been allocated resources
salloc.exe: Granted job allocation 46116226
salloc.exe: Waiting for resource configuration
salloc.exe: Nodes cn3144 are ready for job

[user@cn3144 ~]$ cp -r /usr/local/apps/neusomatic/0.1.2/test .

[user@cn3144 ~]$ cd test

[user@cn3144 ~]$ module load neusomatic
[+] Loading neusomatic  0.1.2  on cn4174 

[user@cn3144 ~]$ ./run_test.sh
--2018-10-15 16:00:17--  ftp://ftp.ensembl.org/pub/release-75//fasta/homo_sapiens/dna/Homo_sapiens.GRCh37.75.dna.chromosome.22.fa.gz
Resolving dtn05-e0 (dtn05-e0)... 10.1.200.241
Connecting to dtn05-e0 (dtn05-e0)|10.1.200.241|:3128... connected.
Proxy request sent, awaiting response... 200 Gatewaying
Length: 10529959 (10M) [text/plain]
Saving to: ‘Homo_sapiens.GRCh37.75.dna.chromosome.22.fa.gz’

100%[======================================================================================================================================>] 10,529,959  3.38MB/s   in 3.0s   

2018-10-15 16:00:22 (3.38 MB/s) - ‘Homo_sapiens.GRCh37.75.dna.chromosome.22.fa.gz’ saved [10529959/10529959]

INFO 2018-10-15 16:00:25,734 __main__             Namespace(dbsnp_to_filter=None, del_merge_min_af=0, del_min_af=0.05, ensemble_tsv=None, good_ao=10, ins_merge_min_af=0, ins_min_af=0.05, long_read=False, matrix_base_pad=7, matrix_width=32, merge_r=0.5, min_ao=1, min_dp=5, min_ev_frac_per_col=0.06, min_mapq=10, mode='call', normal_bam='../normal.bam', num_threads=1, reference='Homo_sapiens.GRCh37.75.dna.chromosome.22.fa', region_bed='../region.bed', restart=False, scan_alignments_binary='/usr/local/apps/neusomatic/0.1.2/neusomatic/bin/scan_alignments', scan_maf=0.05, scan_window_size=2000, skip_without_qual=False, snp_min_af=0.05, snp_min_ao=10.0, snp_min_bq=20.0, truth_vcf=None, tsv_batch_size=50000, tumor_bam='../tumor.bam', work='work_standalone')
INFO 2018-10-15 16:00:25,734 preprocess           ----------------------Preprocessing------------------------
INFO 2018-10-15 16:00:25,735 preprocess           Scan tumor bam (first without quality scores).
INFO 2018-10-15 16:00:25,736 process_split_region Scan bam.
INFO 2018-10-15 16:00:25,736 scan_alignments      -------------------Scan Alignment BAM----------------------
INFO 2018-10-15 16:00:25,771 split_region         ------------------------Split region-----------------------
INFO 2018-10-15 16:00:25,800 split_region         Total length: 40516
INFO 2018-10-15 16:00:25,808 split_region         Split 0: 40516
INFO 2018-10-15 16:00:25,808 split_region         Total splitted length: 40516
[...]
INFO 2018-10-15 16:00:41,694 extract_postprocess_targets --------------Extract Postprocessing Targets---------------
INFO 2018-10-15 16:00:41,700 postprocess          Resolve targets
INFO 2018-10-15 16:00:41,700 resolve_variants     -------Resolve variants (e.g. exact INDEL sequences)-------
INFO 2018-10-15 16:00:41,722 postprocess          Merge vcfs
INFO 2018-10-15 16:00:41,723 merge_post_vcfs      ------------------------Merge vcfs-------------------------
INFO 2018-10-15 16:00:41,728 postprocess          Output NeuSomatic prediction at work_ensemble/NeuSomatic_ensemble.vcf
INFO 2018-10-15 16:00:41,728 postprocess          Done.
### NeuSomatic stand-alone: SUCCESS! ###
### NeuSomatic ensemble: SUCCESS! ###

[user@cn3144 ~]$ exit
salloc.exe: Relinquishing job allocation 46116226
[user@biowulf ~]$

Batch job
Most jobs should be run as batch jobs.

Create a batch input file (e.g. neusomatic.sh). For example:

#!/bin/bash
module load neusomatic
preprocess.py \
        --mode call \
        --reference Homo_sapiens.GRCh37.75.dna.chromosome.22.fa \
        --region_bed region.bed \
        --tumor_bam tumor.bam \
        --normal_bam normal.bam \
        --work work_standalone \
        --scan_maf 0.05 \
        --min_mapq 10 \
        --snp_min_af 0.05 \
        --snp_min_bq 20 \
        --snp_min_ao 10 \
        --ins_min_af 0.05 \
        --del_min_af 0.05 \
        --num_threads 1 \
        --scan_alignments_binary $NEUSOMATIC_BIN/scan_alignments

Submit this job using the Slurm sbatch command.

sbatch --partition=gpu --gres=gpu:k80:1,lscratch:10 --mem=20g -c14 neusomatic.sh
Swarm of Jobs
A swarm of jobs is an easy way to submit a set of independent commands requiring identical resources.

Create a swarmfile (e.g. neusomatic.swarm). For example:

preprocess.py --mode call --region_bed region1.bed --tumor_bam tumor.bam   --normal_bam ../normal.bam [...]
preprocess.py --mode call --region_bed region2.bed --tumor_bam tumor.bam   --normal_bam ../normal.bam [...]
preprocess.py --mode call --region_bed region3.bed --tumor_bam tumor.bam   --normal_bam ../normal.bam [...]

Submit this job using the swarm command.

swarm -f neusomatic.swarm -g 20 -t 14 --partition=gpu --gres=gpu:k80:1,lscratch:10 --module neusomatic
where
-g # Number of Gigabytes of memory required for each process (1 line in the swarm command file)
-t # Number of threads/CPUs required for each process (1 line in the swarm command file).
--module neusomatic Loads the neusomatic module for each subjob in the swarm