Biowulf High Performance Computing at the NIH
File Backups on the Helix Systems

Storage Area Backup policy Snapshots
Home directories
(/home/username)
Weekly backups, with daily incremental backups. These tapes are kept onsite for four weeks. Every four weeks a special full backup of home directories is done and sent offsite; these tapes are kept for eight weeks 6 hourly, 6 daily and 8 weekly
Data directories and shared data directories
(/data/username or /data/SharedDir)
NOT BACKED UP. Users should back up any irreplaceable data to their own local systems. 2 nightly and 1 weekly. Very large /data directories may contain only one nightly snapshot.
Buy-in storage As agreed in the storage buy-in agreement. As agreed in the storage buy-in agreement.

Snapshots

A 'snapshot' is a read-only copy of the data at a particular point in time.

If you inadvertently deleted a file, you can try to restore it yourself from an existing snapshot, using the instructions below. If the file had been present in your directory when a snapshot was made, you will be able to see the file in the .snapshot and copy it back into place.

Note that .snapshot directories are 'hidden'. They won't show up in an 'ls'. .

Recover files in home directory from snapshot

Log into your helix account, go to your home directory:

 
  % cd 
  

Assume that you are going to restore the files in your home directory, type:

 
% cd .snapshot

% ls
Hourly.2016-05-11_1700  Hourly.2016-05-12_1400   Nightly.2016-05-12_0010  Weekly.2016-04-24_0010
Hourly.2016-05-11_2000  Nightly.2016-05-06_0010  Weekly.2016-03-20_0010   Weekly.2016-05-01_0010
Hourly.2016-05-11_2300  Nightly.2016-05-07_0010  Weekly.2016-03-27_0010   Weekly.2016-05-08_0010
Hourly.2016-05-12_0500  Nightly.2016-05-09_0010  Weekly.2016-04-03_0010
Hourly.2016-05-12_0800  Nightly.2016-05-10_0010  Weekly.2016-04-10_0010
Hourly.2016-05-12_1100  Nightly.2016-05-11_0010  Weekly.2016-04-17_0010
  

Select the time frame when your files still existed. There are 3 time categories: hourly, nightly, and weekly. Each category has several snapshots with timestamps as part of the directory name (e.g. Nightly.2016-05-06_0010 is a snapshot taken on May 6, 2016, at 6 am.)

Go to selected directory:

% cd Nightly.2016-05-06_0010

Then find the file(s) you are interested in and copy them to your current home directory:

% cp lostfile /home/$USER/lostfile

If you have your directory mounted on your desktop Windows machine (see http://helix.nih.gov/Documentation/transfer.html), you can also recover files:

.snapshot directory

Recover files in data directory from snapshot

Snapshots for data directories are done less frequently than for home directories. At a minimum there will be at least one daily and one weekly snapshot available for all data directories.

The names of the snapshot directories in /data vary depending on the storage system on which a data directory is located. The snapshot directory names have one of two formats: either 1) _nightly.0 and _weekly.0 or 2) daily.‹timestamp› and weekly.‹timestamp› where ‹timestamp› is the date/time when the snapshot was created.

To recover data directory files from snapshots, go to the snapshot area of your data directory:

% cd /data/$USER/.snapshot

Select the time frame when your files still existed based on the snapshot filename formats descibed above. Go to selected directory:

% cd weekly.2016-05-10_0751

Find the file(s) you wish to recover and copy them to your data directory:

% cp lostfile /data/$USER/lostfile

Note that data directory snapshots are updated daily between 1 and 5 AM Eastern Time and may be temporarily unavailable during that time period.