Biowulf High Performance Computing at the NIH
Upcoming Classes & Seminars

Scientific Python for Matlab users

Instructor(s): Antonio Ulloa (NIH HPC Staff)
Location: Bldg 12, Rm B51     Date/Time: Tue Jul 09, 2019 / 10:00 AM - 12:00 PM
  Webcast available -- registration required

[Details]

This course is an introduction to scientific Python for those with some experience programming in Matlab. We will introduce basic concepts of Python and its scientific stack and compare those to Matlab's commands and data structures. Additionally, we will conduct a step-by-step re-implementation of a well-known Matlab scientific computing script in Python. Expected knowledge: Some programming experience in Matlab. Familiarity with the Linux/Unix command line.
Register
(NIH login required)
Introduction to Biowulf
Instructors: NIH HPC Staff

An online, self-paced class, with video tutorials and hands-on exercises. New Biowulf users are encouraged to work through the entire class, and experienced Biowulf users can view specific videos to brush up on a particular section.

Click here to get to the class.

No registration required

Biowulf 20th Anniversary Seminar Series (2019)

  6 June 2019 [Videocast]
  Using whole-genome sequencing to investigate hospital-associated bacterial reservoirs
   Sean Conlan (NHGRI)

  23 May 2019 [Videocast]
  Decoding Time-resolved Neural Representations of Working Memory and Motor Learning
   Romain Quentin and Leonardo Claudino (Leo Cohen lab, NINDS)

  7 May 2019 [Videocast] [Slides]
   Neurogenetics on Biowulf: From GWAS to Machine Learning
   Mike Nalls (Data Tecnica, NIA)

  9 April 2019 [Videocast]
  Integrative analyses of gene regulation via long-range chromatin interactions
  Ryan Dale (Scientific Information Officer, NICHD)

28 Feb 2019: [Videocast]
  Biowulf at 20: Celebrating Two Decades of Supporting Biomedical Computing in the NIH IRP.
  Andy Baxevanis (Director of Computational Biology, NIH IRP)

  State of the Cluster: Past, Present and Future
  Steven Fellini (Biowulf Lead Architect). [Slides - Fellini]

  Telomere-to-telomere assembly of a complete human X chromosome
  Sergen Koren (NHGRI) [Slides - Koren]



Biowulf 2017-2018 Seminar Series

Slides and Handouts from Previous HPC Classes

Apart from the handouts listed below, the NIH HPC staff creates and maintains Training Videos to help users get the most out of our resources.

Deep Learning by Example on Biowulf, class #2 (PDF) (Software manual)
This introductory course teaches major types of deep learning networks (Convolutional, Recurrent, Autoencoders, etc.) through a set of hands-on biological examples implemented in Keras, adopted and installed on Biowulf. Class #2 focuses on Recurrent Neural Networks and their application to the prediction of the function of non-coding DNA directly from its sequence.
Gennady Denisov, 29 May 2019

Data Management Best Practices for Groups (PDF) (PPT)
An overview of file storage, access permissions, file transfer, and sociological behaviors to keep your collaborative group functioning.
David Hoover, 24 April 2019

Deep Learning by Example on Biowulf, class #1 (PDF) (Software manual)
This introductory course teaches major types of deep learning networks (Convolutional, Recurrent, Autoencoders, etc.) through a set of hands-on biological examples implemented in Keras, adopted and installed on Biowulf. Class #1 focuses on biomedical image segmentation using Convolutional Neural Networks.
Gennady Denisov, 23 Apr 2019

Introduction to Revision Control with Git (PDF) (demo forthcoming)
Revision control systems such as Git allow you to manage changes to your files, including their attribution and context. You have the ability to quickly revert specific changes, view your files (and differences between them) at arbitrary points in their history, pursue experimental lines of development without disturbing the main version, and safely collaborate with a team--all without personally managing multiple copies of the same file. While revision control systems are popularly known for managing the development of software, they can be used for other kinds of manually-developed content, such as documents and configurations. This hands-on tutorial will cover the basics of Git, interacting with Git repositories, and workflows for personal use and collaboration.
Afif Elghraoui, 28 Mar 2019

Introduction to Linux (PDF)
Course intended for researchers that are new to Linux/Unix. Covers Unix/Linux operating system concepts, a little history, basic file system navigation, text editing, bash (shell) syntax, file transfer, and a number of useful commands and utilities.
Mark Patkus, 7-8 May 2019

NIH HPC Object Storage System Overview (PDF for Jan, 2018 class) (PDF for Oct, 2018 class)
This course introduces users to the concept of object storage - a new technology being used by many large Internet companies that is becoming increasingly popular for scientific use because of its capability to store large-scale, unstructured data. The class describes the NIH HPC object storage system in detail and includes a practical example of its use in a real scientific workflow.
Tim Miller, 9 Oct 2018

Bash Shell Scripting (PDF) (PPT)
The default shell on many Linux systems is bash. Bash shell scripting provides a method for automating common tasks on Linux systems (such as Helix and Biowulf), including transferring and parsing files, creating sbatch and swarm scripts, pipelining tasks, and monitoring jobs. A step-by-step data-driven lesson is available. There is a summary of Linux commands (PDF) and the GNU Bash manual (PDF) available as well.
David Hoover, 22-23 January 2019

Creating and running software containers with Singularity (slides and tutorial)
This was a three hour hands-on workshop on how to user Singularity to create and run containers. Students learned how to install Singularity on a Linux system, create containers with their choice of Linux distribution and software, and use Singularity to run containerized apps.
Afif Elghraoui, 26 July 2018

Building a reproducible workflow with Snakemake and Singularity (Slides and GitHub repo)
Students attending this class will learn how to build a workflow with Snakemake and how to make it more reproducible with Singularity containers. This class will make use of the Biowulf cluster and requires knowledge of the Linux command line as well as Python.
Wolfgang Resch, 21 February 2018

The NIH Biowulf Cluster: Scientific Supercomputing (PDF)
This two-part class is an introduction to the Biowulf Linux cluster for users who have NIH Biowulf accounts or Helix users planning to get one. Topics covered: cluster concepts, accounts, connection, storage, batch system, how to set up and submit a simple batch job, partitions, interactive jobs, swarm jobs, available scientific applications, job monitoring, resource allocation, licensed software.
Steven Fellini and Susan Chacko, 13 Feb 2018

Singularity Demo( Slides |  Demo Readme)
1h intro and demo of Singularity containers for NIMH Reproducible Neuroscience Workshop

Using the HPC Systems Storage Effectively (PDF)
A course that describes the different storage systems available to NIH HPC users along with policies and best practices. Also explains how to avoid storage bottlenecks when running jobs.
Tim Miller, 16 Feb 2017

Parallel MATLAB jobs on Biowulf (PDF) (PPT) (Videos)
Developing MATLAB code for parallel computing, using the MATLAB compiler to deploy license-free code, automating swarm file generation, spawning and monitoring swarms interactively from within the MATLAB environment, and ordering jobs with dependencies to develop an analysis pipeline.
Dave Godlove, 17 Feb 2016

Swarm on the Biowulf Cluster (PDF) (PPT)
Swarm is a script designed to simplify submitting a group of commands to the Biowulf cluster. With the shift from PBS to Slurm, the functionalities and indiosycrasies of swarm have changed.
David Hoover, 22 Sep 2015

Rosetta Workshop
Tutorials and presentations from the Rosetta Design Group, hosted by Helix Systems. (NIH only)
Xavier Ambroggio and Monica Berrondo, 19-21 May 2009

Gene Synthesis using DNAWorks (PPT)
David Hoover, 15 Nov 2006

Linux Tutorials

Helix and Biowulf users will make most effective use of the systems if they are familiar with GNU/Linux.

Below are links to some tutorials which cover the basics of GNU/Linux commands.

Introduction to Linux at the TACC, Texas.
Introduction to Linux guide located on the the Linux Documentation Project's website.
A Basic Unix/Linux Tutorial, at Oxford University, UK.
Unix Tutorial for Beginners. Eight simple tutorials which cover the basics of Unix, at U. Surrey, UK.
Command-line crash course
The Linux Command Line, by William Shotts
Learn the Command Line, A web-based GUI tutorial by codeacademy

Other Training at NIH

The Technology Training Program at CIT offers courses relating to computing, networks, and information systems.

The NIH training program offers classes in writing, speaking, grant writing and more.

The FAES Graduate School runs short and long courses.