Biowulf High Performance Computing at the NIH
Tybalt: training and evaluating a variational autoencoder for pan-cancer gene expression data

Tybalt employs Keras deep learning framework to implement a variational autoencoder (VAE), capable of generating meaningful latent spaces for image and text data. Tybalt has been trained on The Cancer Genome Atlas (TCGA) pan-cancer RNA-seq data and used to identify specific patterns in the VAE encoded features.

References:

Documentation
Important Notes

Interactive job
Interactive jobs should be used for debugging, graphics, or applications that cannot be run as batch jobs.

Allocate an interactive session and run the program. Sample session:

[user@biowulf]$ sinteractive --mem=16g --gres=gpu:v100,lscratch:10
[user@@cn3200 ~]$module load Tybalt  
[+] Loading libarchive  3.3.2 
[+] Loading singularity  on cn4470 
[+] Loading python 3.6  ... 
[+] Loading cuDNN 7.0  libraries... 
[+] Loading CUDA Toolkit  9.0.176  ... 
[+] Loading gcc  7.2.0  ... 
[+] Loading GSL 2.4 for GCC 7.2.0 ... 
[+] Loading openmpi 3.0.0  for GCC 7.2.0 
[+] Loading R 3.5.0_build2 
[+] Loading Tybalt 0.1.3  ... 
Tybalt includes a script vae_pancancer.py which performs training of a variational autoencoder (VAE) on pre-processed data downloaded from the TCGA. To visualize a usege mesaage for this script, type:
[user@biowulf]$ vae_pancancer.py -h
...
usage: vae_pancancer.py [-h] [-l LEARNING_RATE] [-b BATCH_SIZE] [-e EPOCHS]
                        [-k KAPPA] [-d DEPTH] [-c FIRST_LAYER]
                        [-f OUTPUT_FILENAME] [-n NUM_COMPONENTS]

optional arguments:
  -h, --help            show this help message and exit
  -l LEARNING_RATE, --learning_rate LEARNING_RATE
                        learning rate of the optimizer
  -b BATCH_SIZE, --batch_size BATCH_SIZE
                        Number of samples to include in each learning batch
  -e EPOCHS, --epochs EPOCHS
                        How many times to cycle through the full dataset
  -k KAPPA, --kappa KAPPA
                        How fast to linearly ramp up KL loss
  -d DEPTH, --depth DEPTH
                        Number of layers between input and latent layer
  -c FIRST_LAYER, --first_layer FIRST_LAYER
                        Dimensionality of the first hidden layer
  -f OUTPUT_FILENAME, --output_filename OUTPUT_FILENAME
                        The name of the file to store results
  -n NUM_COMPONENTS, --num_components NUM_COMPONENTS
                        The latent space dimensionality to test
Here is an example of training command:
[user@biowulf]$ vae_pancancer.py -l 0.001 -b 100 -e 40 -k 0.05 -d 2 -c 100 -f res
...
Using TensorFlow backend.
...
1071  14:12  Epoch 1/40
 1072  14:12  9413/9413 [==============================] - 3s 355us/step - loss: 2955.5256 - val_loss: 2837.6293
 1073  14:12  Epoch 2/40
 1074  14:12  9413/9413 [==============================] - 2s 204us/step - loss: 2796.2215 - val_loss: 2782.0771
 1075  14:12  Epoch 3/40
 1076  14:12  9413/9413 [==============================] - 2s 204us/step - loss: 2764.0638 - val_loss: 2766.8539
...
Epoch 38/40
9413/9413 [==============================] - 2s 204us/step - loss: 2668.6827 - val_loss: 2676.4443
Epoch 39/40
9413/9413 [==============================] - 2s 206us/step - loss: 2668.2820 - val_loss: 2674.2052
Epoch 40/40
9413/9413 [==============================] - 2s 207us/step - loss: 2667.7264 - val_loss: 2674.6204

Tybalt employs a variational autoencoder by default, but optionally it can also perform Analysis using a Denoising Autoencoder applied to Gene Expression (ADAGE), which can be trained on the same data:

[user@biowulf]$ adage_pancancer.py  -l 0.001 -b 100 -e 40 -f res_adage -n 0.1 -s 0.001
...
Using TensorFlow backend.
...
Epoch 1/40
...
9413/9413 [==============================] - 4s 402us/step - loss: 0.0987 - val_loss: 0.0729
Epoch 2/40
9413/9413 [==============================] - 2s 210us/step - loss: 0.0608 - val_loss: 0.0478
Epoch 3/40
9413/9413 [==============================] - 2s 208us/step - loss: 0.0407 - val_loss: 0.0366
...
Epoch 39/40
9413/9413 [==============================] - 2s 226us/step - loss: 0.0349 - val_loss: 0.0349
Epoch 40/40
9413/9413 [==============================] - 2s 224us/step - loss: 0.0349 - val_loss: 0.0349

End the interactive session:
[user@cn3200 ~]$ exit
salloc.exe: Relinquishing job allocation 46116226
[user@biowulf ~]$