High-Performance Computing at the NIH
GitHub YouTube @nih_hpc RSS Feed
Irreproducible Discovery Rate (IDR)

The IDR (Irreproducible Discovery Rate) framework is a unified approach to measure the reproducibility of findings identified from replicate experiments and provide highly stable thresholds based on reproducibility. Unlike the usual scalar measures of reproducibility, the IDR approach creates a curve, which quantitatively assesses when the findings are no longer consistent across replicates. In layman's terms, the IDR method compares a pair of ranked lists of identifications (such as ChIP-seq peaks). These ranked lists should not be pre-thresholded i.e. they should provide identifications across the entire spectrum of high confidence/enrichment (signal) and low confidence/enrichment (noise). The IDR method then fits the bivariate rank distributions over the replicates in order to separate signal from noise based on a defined confidence of rank consistency and reproducibility of identifications i.e the IDR threshold.

References:

There may be multiple versions of IDR available. An easy way of selecting the version is to use modules. To see the modules available, type

module avail idr

To select a module, type

module load idr/[ver]

where [ver] is the version of choice.

Environment variables set:

On Helix

Sample session:

$ module load idr
[+] Loading idr, version 2.0.3...
$ DATADIR=/usr/local/apps/idr/data
$ idr --samples $DATADIR/peak1 $DATADIR/peak2 -o peak1-2.txt
Initial parameter values: [0.10 1.00 0.20 0.50]
Final parameter values: [1.57 1.26 0.89 0.41]
Number of reported peaks - 50537/50537 (100.0%)

Number of peaks passing IDR cutoff of 0.05 - 12748/50537 (25.2%)
Interactive job on Biowulf

See the Biowulf user guide for interactive jobs.

Batch job on Biowulf

Create a batch input file (e.g. idr.sh). For example:

#!/bin/bash
module load idr
DATADIR=/usr/local/apps/idr/data
idr --samples $DATADIR/peak1 $DATADIR/peak2 -o peak1-2.txt

Submit this job using the Slurm sbatch command.

sbatch --cpus-per-task=1 idr.sh
Swarm of Jobs on Biowulf

Create a swarmfile following the swarm guide using the example commands on this page.

Documentation