High-Performance Computing at the NIH
GitHub YouTube @nih_hpc RSS Feed
loki on Biowulf

Loki is a linkage analysis package, primarily for large and complex pedigrees, which uses Markov chain Monte Carlo (MCMC) techniques to avoid many of the computational problems that prevent exact computational methods being used for large pedigrees. Loki was originally designed for linkage analysis of quantitative traits, though there is some limited support for qualitative trait linkage analysis. Loki is not maintained any more and some of the auxillary scripts were written for a very old version of perl and may be broken.

Documentation
Important Notes

Interactive job
Interactive jobs should be used for debugging, graphics, or applications that cannot be run as batch jobs.

Allocate an interactive session and run the program. Sample session:

[user@biowulf]$ sinteractive
salloc.exe: Pending job allocation 46116226
salloc.exe: job 46116226 queued and waiting for resources
salloc.exe: job 46116226 has been allocated resources
salloc.exe: Granted job allocation 46116226
salloc.exe: Waiting for resource configuration
salloc.exe: Nodes cn3144 are ready for job

[user@cn3144]$ cp -r ${LOKI_TEST_DATA:-none}/* .
[user@cn3144]$ ls -1
data
run

[user@cn3144]$ cd run
[user@cn3144]$ ls -1
control_gaw9
param_gaw9
seedfile

[user@cn3144]$ # set up loki run
[user@cn3144]$ prep control_gaw9
getseed(): Using stored state for mt19937b generator
Reading in data from file '../data/gaw9_d1a1-15'
1497 records of 9 columns read in
Reading in data from file '../data/gaw9_phenotyp'
1497 records of 7 columns read in
Initializing and recoding pedigree
Recoding factors
Recoding marker data
Handling data restrictions
[...snip...]

[user@cn3144]$ loki param_gaw9
Reading from loki.opt
Reading from loki.dat
Reading from loki.gen
Map range for linkage group 'Chromosome 1' set to 0-60cM
Total (genome) map length set to (360,360)cM
Analysis = 0
Created by loki 2.4.7_4: Thu May 17 09:06:10 2018
Output format: 3
Model: Q(1) = QTL + EF + AGE
Input map function: Haldane
Output map function: Haldane
Linkage groups:
  1: Chromosome 1  Map range:  (0cM to 60cM)
    D1G(1) - 0
    D1G(5) - 8
    D1G(10) - 18
    D1G(15) - 28
[...snip...]

[user@cn3144]$ head -n 15 loki.out
Created by loki 2.4.7_4: Thu May 17 09:34:55 2018
Output format: 3
Model: Q(1) = QTL + EF + AGE
Input map function: Haldane
Output map function: Haldane
Linkage groups:
  1: Chromosome 1  Map range:  (0cM to 60cM)
    D1G(1) - 0
    D1G(5) - 8
    D1G(10) - 18
    D1G(15) - 28
Total Map Length: 360cM
Output columns:
  1: Iteration count
  2: Residual variance

[user@cn3144]$ # some postprocessing
[user@cn3144]$ count.pl
----------------------------------------------
Model: Q(1) = QTL + EF + AGE
Iterations: 10000
Switching proportion for QTL no.: 0.09990
Switching proportion for linked QTLs:  0.12300

Linkage group    Count   Prop. linked    BF
----------------------------------------------
Unlinked          9999     0.99990     1.00707
Chromosome 1      4909     0.49090     1.02365      5.58    0.50cM

QTL number       0      1      2      3      4      5
-------------------------------------------------------
Overall       0.0001 0.0018 0.1793 0.3272 0.3178 0.1738
Linked        0.5091 0.3975 0.0809 0.0122 0.0003 0.0000
Unlinked      0.0002 0.0470 0.3268 0.3690 0.2075 0.0495
-------------------------------------------------------
Chromosome 1  0.5091 0.3975 0.0809 0.0122 0.0003 0.0000

[user@cn3144]$ dist.pl -C -o gaw9.ps
[user@cn3144]$ module load ImageMagick
[user@cn3144]$ convert -density 300 -rotate 90 gaw9.ps gaw9.png
[user@cn3144]$ # resulting in the graph shown below
[user@cn3144]$ exit
salloc.exe: Relinquishing job allocation 46116226
[user@biowulf]$

Batch job
Most jobs should be run as batch jobs.

Create a batch input file (e.g. loki.sh), which uses the input file 'loki.in'. For example:

#!/bin/bash
module load loki || exit 1
loki params_file > logfile

Submit this job using the Slurm sbatch command.

sbatch --cpus-per-task=2 --mem=2g loki.sh
Swarm of Jobs
A swarm of jobs is an easy way to submit a set of independent commands requiring identical resources.

Create a swarmfile (e.g. loki.swarm). For example:

cd /data/user/loki/run1 && loki params_file > logfile
cd /data/user/loki/run2 && loki params_file > logfile
cd /data/user/loki/run3 && loki params_file > logfile

Submit this job using the swarm command.

swarm -f loki.swarm -g 2 -t 1 -p 2 --module loki
where
-g #Number of Gigabytes of memory required for each process (1 line in the swarm command file)
-t 1Number of threads/CPUs required for each process (1 line in the swarm command file).
-p 2Pack 2 processes into each subjob
--module lokiLoads the loki module for each subjob in the swarm