QIIME 2 is a powerful, extensible, and decentralized microbiome analysis package with a focus on data and analysis transparency. QIIME 2 enables researchers to start an analysis with raw DNA sequence data and finish with publication-quality figures and statistical results.
mkdir -p $HOME/.config/matplotlib/
echo "backend: Agg" >> $HOME/.config/matplotlib/matplotlibrc
Allocate an interactive session and run the program. Sample session (user input in bold):
[user@biowulf]$ sinteractive salloc.exe: Pending job allocation 46116226 salloc.exe: job 46116226 queued and waiting for resources salloc.exe: job 46116226 has been allocated resources salloc.exe: Granted job allocation 46116226 salloc.exe: Waiting for resource configuration salloc.exe: Nodes cn3144 are ready for job [user@cn3144 ~]$ module load QIIME [user@cn3144 ~]$ cp -R $QIIME_EXAMPLES/qiime2-moving-pictures-tutorial . [user@cn3144 ~]$ cd qiime2-moving-pictures-tutorial [user@cn3144 ~]$ qiime tools import --type EMPSingleEndSequences \ --input-path emp-single-end-sequences \ --output-path emp-single-end-sequences.qza [user@cn3144 ~]$ qiime demux emp-single \ --i-seqs emp-single-end-sequences.qza \ --m-barcodes-file sample-metadata.tsv \ --m-barcodes-column BarcodeSequence \ --output-dir demux-summarize-out \ --o-per-sample-sequences demux.qza [user@cn3144 ~]$ qiime demux summarize \ --i-data demux.qza \ --o-visualization demux.qzv [user@cn3144 ~]$ qiime dada2 denoise-single \ --i-demultiplexed-seqs demux.qza \ --p-trim-left 0 \ --p-trunc-len 120 \ --o-representative-sequences rep-seqs-dada2.qza \ --output-dir denoise-single-out \ --o-table table-dada2.qza [user@cn3144 ~]$ exit salloc.exe: Relinquishing job allocation 46116226 [user@biowulf ~]$
Create a batch input file (e.g. QIIME.sh). For example:
#!/bin/bash module load QIIME cp -R $QIIME_EXAMPLES/qiime2-atacama-tutorial . cd qiime2-atacama-tutorial qiime tools import \ --type EMPPairedEndSequences \ --input-path emp-paired-end-sequences \ --output-path emp-paired-end-sequences.qza qiime demux emp-paired \ --verbose \ --m-barcodes-file sample-metadata.tsv \ --m-barcodes-category BarcodeSequence \ --i-seqs emp-paired-end-sequences.qza \ --o-per-sample-sequences demux \ --p-rev-comp-mapping-barcodes qiime demux summarize \ --verbose \ --i-data demux.qza \ --o-visualization demux.qzv qiime dada2 denoise-paired \ --verbose \ --p-n-threads $SLURM_CPUS_PER_TASK \ --i-demultiplexed-seqs demux.qza \ --o-table table \ --o-representative-sequences rep-seqs \ --p-trim-left-f 13 \ --p-trim-left-r 13 \ --p-trunc-len-f 150 \ --p-trunc-len-r 150 qiime feature-table summarize \ --verbose \ --i-table table.qza \ --o-visualization table.qzv \ --m-sample-metadata-file sample-metadata.tsv qiime feature-table tabulate-seqs \ --verbose \ --i-data rep-seqs.qza \ --o-visualization rep-seqs.qzv
Submit this job using the Slurm sbatch command.
sbatch --cpus-per-task=8 --mem=20g --gres:lscratch:20 --time=4:00:00 QIIME.sh
Create a swarmfile (e.g. QIIME.swarm). For example:
qiime dada2 denoise-single \ --p-n-threads $SLURM_CPUS_PER_TASK \ --p-trim-left 13 \ --p-trunc-len 150 \ --i-demultiplexed-seqs fmt-tutorial-demux-1.qza \ --o-representative-sequences rep-seqs-1.qza \ --o-table table-1.qza qiime dada2 denoise-single \ --p-n-threads $SLURM_CPUS_PER_TASK \ --p-trim-left 13 \ --p-trunc-len 150 \ --i-demultiplexed-seqs fmt-tutorial-demux-2.qza \ --o-representative-sequences rep-seqs-2.qza \ --o-table table-2.qza qiime dada2 denoise-single \ --p-n-threads $SLURM_CPUS_PER_TASK \ --p-trim-left 13 \ --p-trunc-len 150 \ --i-demultiplexed-seqs fmt-tutorial-demux-3.qza \ --o-representative-sequences rep-seqs-3.qza \ --o-table table-3.qza qiime dada2 denoise-single \ --p-n-threads $SLURM_CPUS_PER_TASK \ --p-trim-left 13 \ --p-trunc-len 150 \ --i-demultiplexed-seqs fmt-tutorial-demux-4.qza \ --o-representative-sequences rep-seqs-4.qza \ --o-table table-4.qza
Submit this job using the swarm command.
swarm -f QIIME.swarm -g 10 -t 8 --module QIIMEwhere
| -g # | Number of Gigabytes of memory required for each process (1 line in the swarm command file) |
| -t # | Number of threads/CPUs required for each process (1 line in the swarm command file). |
| --module QIIME | Loads the QIIME module for each subjob in the swarm |