Biowulf High Performance Computing at the NIH
TeraStitcher on Biowulf

TeraStitcher is a free tool that enables the stitching of Teravoxel-sized tiled microscopy images even on workstations with relatively limited resources of memory (<8 GB) and processing power. It exploits the knowledge of approximate tile positions and uses ad-hoc strategies and algorithms designed for such very large datasets. The produced images can be saved into a multiresolution representation to be efficiently visualized (e.g. Vaa3D-TeraFly) and processed.

References:

Documentation
Important Notes

Some steps run multithreaded, with the number of threads set to the number of CPUs on the node. Make sure to allocate at least 16 CPUs per task.

Interactive job
Interactive jobs should be used for debugging, graphics, or applications that cannot be run as batch jobs.

Allocate an interactive session and run the program.
Sample session (user input in bold):

[user@biowulf]$ sinteractive
salloc.exe: Pending job allocation 46116226
salloc.exe: job 46116226 queued and waiting for resources
salloc.exe: job 46116226 has been allocated resources
salloc.exe: Granted job allocation 46116226
salloc.exe: Waiting for resource configuration
salloc.exe: Nodes cn3144 are ready for job

[user@cn3144 ~]$ module load TeraStitcher
[user@cn3144 ~]$ terastitcher --import --volin=my_data --volin_plugin="TiledXY|2Dseries" --imin_plugin="tiff2D" --ref1=y --ref2=-x --ref3=z --vxl1=0.8 --vxl2=0.8 --vxl3=1 --projout=xml_import

[user@cn3144 ~]$ exit
salloc.exe: Relinquishing job allocation 46116226
[user@biowulf ~]$

Batch job
Most jobs should be run as batch jobs.

Create a batch input file (e.g. TeraStitcher.sh). For example:

#!/bin/bash
set -e
module load TeraStitcher
# An example of stitching pipeline
cp -Rp $TERASTITCHER_EXAMPLES/mouse.cerebellum.300511.sub3/tomo300511_subv3 .
terastitcher --test --volin=tomo300511_subv3 --ref1=1 --ref2=-2 --ref3=3 --vxl1=0.8 --vxl2=0.8 --vxl3=1
terastitcher --import --volin=tomo300511_subv3 --ref1=1 --ref2=-2 --ref3=3 --vxl1=0.8 --vxl2=0.8 --vxl3=1 --projout=xml_import
terastitcher --displcompute --projin="tomo300511_subv3/xml_import.xml" --projout=xml_displcomp --subvoldim=100
terastitcher --displproj --projin="tomo300511_subv3/xml_displcomp.xml" --projout=xml_displproj
terastitcher --displthres --projin="tomo300511_subv3/xml_displproj.xml" --projout=xml_displthres --threshold=0.7
terastitcher --placetiles --projin="tomo300511_subv3/xml_displthres.xml" --projout=xml_merging
mkdir output
terastitcher --merge --projin="tomo300511_subv3/xml_merging.xml" --volout=output --resolutions=01234 --volout_plugin="TiledXY|3Dseries" --imout_plugin="tiff3D" --imout_format="tif"

Submit this job using the Slurm sbatch command.

sbatch --cpus-per-task=16 --mem=10g TeraStitcher.sh