High-Performance Computing at the NIH
GitHub YouTube @nih_hpc RSS Feed
TORTOISE on Biowulf & Helix

TORTOISE (Tolerably Obsessive Registration and Tensor Optimization Indolent Software Ensemble) is for processing diffusion MRI data, and it contains two main modules, DIFF_PREP and DIFF_CALC.:

DIFF_CALC is GUI-based. DIFF_PREP has a command-line option and can be used on Biowulf for multiple simultaneous runs. This procedure is described below.

Tortoise was developed at NICHD, NIH. TORTOISE website

Important Notes

Process several subjects using Tortoise/IDL
(Sample scripts and procedure courtesy of Vivek Buch, NINDS)

IDL expects to connect to an X11 session, so the initial connection to Biowulf needs to have X forwarding turned on.

1. For each subject, the raw diffusion-weighted MRI images in DICOM format, DTI_proc directory (created by the "import" process in the DIFF_PREP GUI), and the subject's high resolution structural image (e.g. T1 or T2 image, but recommended image is a T2 with fat sat) need to be transferred to Helix/Biowulf into a unique directory, e.g. for SUBJECT1, the directories could be

Each subject scan must have an "xml" file that directs TORTOISE to the DTI_proc and the high resolution structural image within this subject and scan directory. The xml file can be stored anywhere on biowulf (/home/user/myXMLfiles or /data/user), it does not have to be in the same folder as the subject's data. Information on this xml file is on the Tortoise website. This subject scan-specific xml file will be the input into diffprep.

2. Set up a script along the following lines. This script can be used for all future runs and will not need to be changed.

-------------------------file diffprep.sh--------------------------------
#!/bin/bash

module load TORTOISE
cd ${TORTOISE}/DIFF_PREP/diffprep_main

xvfb-run idl <<EOF
diffprep,$1
EOF

3. Set up a batch script along the following lines. This sample script includes 4 subject scans, but if submitting to a 32-core node, up to 32 subject scans can be included in a single batch file. The input parameter for the diffprep.sh script is the XML file described above.

Note: The spaces, non-spaces, single quotes, and double quotes are required exactly in this format.

----------------------- file tortoise.bat --------------------------------
#!/bin/bash
#name of job -- not important
#SBATCH --job-name Tortoise                   
#send email at beginning and end of job
#SBATCH --mail-type=BEGIN,END                

#
cd /home/user/myscripts                   #location of the diffprep.sh script

./diffprep.sh "'/home/user/myXMLfiles/SUBJECT1-scan1.xml'" &
./diffprep.sh "'/home/user/myXMLfiles/SUBJECT1-scan2.xml'" &
./diffprep.sh "'/home/user/myXMLfiles/SUBJECT2-scan1.xml'" &
./diffprep.sh "'/home/user/myXMLfiles/SUBJECT3-scan1.xml'" &
wait

4. Submit this job with:

sbatch --cpus-per-task=32 --ntasks=1  --lic=idl:6   jobscript

This job will be submitted to a single node with 32 cores, and will use 6 IDL licenses. The job will wait in the queue until 6 IDL licenses are available.

If Tortoise runs successfully, the DTI_proc directories will be updated with new files tagged by "_DMC_" which means distortion motion corrected. If it does not work, the DTI_proc will not have any files with this tag. Users can check for the existence of these files to see if the run was successful.

Documentation

TORTOISE website.