Globus on NIH HPC (Biowulf)

Quick Links

Globus is a service that makes it easy to move, sync, and share large amounts of data. Globus will manage file transfers, monitor performance, retry failures, recover from faults automatically when possible, and report the status of your data transfer. Globus uses GridFTP for more reliable and high-performance file transfer, and will queue file transfers to be performed asynchronously in the background.

Globus was developed and is maintained at the University of Chicago and is used extensively at supercomputer centers and major research facilities. [Globus website]

No matter how you transfer data in and out of our systems, be aware that PII and PHI data cannot be stored or transferred into the NIH HPC systems.

See the links in the Quick Links menu at left for details.

NIH HPC (Helix/Biowulf) Endpoints

Globus Collection Name
Globus UUID Purpose
NIH HPC Data Transfer (Biowulf)
e2620047-6d04-11e5-ba46-22000b92c6ec Main Biowulf collection. This collection will let you transfer data to or from /home/$USER, /data/$USER and any shared data directories you have access to. The endpoint is implemented using ten "Data Transfer Nodes" which can operate in parallel to provide 100 Gb/s of aggregate bandwidth.
NIH HPC Internet2 - AWS S3
c24547a8-ef53-4b86-bcf6-d050c55d00f4 Transfer to or from AWS S3 buckets to any other Globus collections, including the NIH HPC (Biowulf) collections. Please note that HPC/Biowulf does not provide or fund AWS accounts for users. (see the NIH Strides initiative for cloud environments)
NIH HPC Google Cloud Collection
46312f97-8565-456d-a1ea-cb3e28e49caa Transfers to/from Google Cloud Storage. Please note that HPC/Biowulf does not provide or fund Google Cloud accounts for users. (see the NIH Strides initiative for cloudenvironments)
NIH HPC Google Drive Collection
22629017-758c-469c-8f75-51eaebcf0417 Transfers to/from Google Drive. Please note that HPC/Biowulf does not provide Google Drive accounts.
NIH HPC OneDrive Collection
ba595c6f-8822-4905-8ce9-6e072bb49ce4 Transfers to/from NIH OneDrive (the Microsoft cloud service which is part of the NIH Microsoft 365 subscription)
NIH HPC Internet2 - Biowulf /home/data
55bad7bd-4b2b-466a-8019-3666483681c2 This collection is on Internet2 and can access Biowulf /home or /data. Transfers within NIH (e.g. laptop or another NIH collection to/from Biowulf) should NOT use this endpoint. Instead, use the main 'NIH HPC Data Transfer (Biowulf)' collection.
This Internet2 endpoint may provide faster transfers between Biowulf and other Internet2 sites, but our tests have been inconclusive

Globus Collection Name Globus UUID	Purpose
NIH HPC Data Transfer (Biowulf) e2620047-6d04-11e5-ba46-22000b92c6ec	Main Biowulf collection. This collection will let you transfer data to or from /home/$USER, /data/$USER and any shared data directories you have access to. The endpoint is implemented using ten "Data Transfer Nodes" which can operate in parallel to provide 100 Gb/s of aggregate bandwidth.
NIH HPC Internet2 - AWS S3 c24547a8-ef53-4b86-bcf6-d050c55d00f4	Transfer to or from AWS S3 buckets to any other Globus collections, including the NIH HPC (Biowulf) collections. Please note that HPC/Biowulf does not provide or fund AWS accounts for users. (see the NIH Strides initiative for cloud environments)
NIH HPC Google Cloud Collection 46312f97-8565-456d-a1ea-cb3e28e49caa	Transfers to/from Google Cloud Storage. Please note that HPC/Biowulf does not provide or fund Google Cloud accounts for users. (see the NIH Strides initiative for cloudenvironments)
NIH HPC Google Drive Collection 22629017-758c-469c-8f75-51eaebcf0417	Transfers to/from Google Drive. Please note that HPC/Biowulf does not provide Google Drive accounts.
NIH HPC OneDrive Collection ba595c6f-8822-4905-8ce9-6e072bb49ce4	Transfers to/from NIH OneDrive (the Microsoft cloud service which is part of the NIH Microsoft 365 subscription)
NIH HPC Internet2 - Biowulf /home/data 55bad7bd-4b2b-466a-8019-3666483681c2	This collection is on Internet2 and can access Biowulf /home or /data. Transfers within NIH (e.g. laptop or another NIH collection to/from Biowulf) should NOT use this endpoint. Instead, use the main 'NIH HPC Data Transfer (Biowulf)' collection. This Internet2 endpoint may provide faster transfers between Biowulf and other Internet2 sites, but our tests have been inconclusive

You do not need to have an ssh session on Helix or Biowulf to start or monitor a transfer. You will simply need to authenticate with your NIH PIV/PIN or equivalent to access any of the above endpoints.

Logging into Globus with your NIH login

NIH researchers can use their NIH Login username and password to access Globus. Go to https://www.globus.org/ and click on Log In in the upper right corner of the page.

Type or scroll down to National Institues of Health in the Organization box, and then click Continue.

You will be taken to a familiar-looking page for NIH login.

Use your PIV card as usual. If you are PIV-exempt (e.g. NIH researchers accessing via the NIH Virtual Desktop Interface (VDI), you may be able to enter your NIH login username and password instead.

Installing the Globus client on your desktop

The Globus Connect client is available for Windows, Mac or Linux desktop systems. There are detailed instructions on the Globus website. See links below.

It is best to be off the VPN when installing the Globus client. How to install and configure Globus Connect Personal on

Troubleshooting:

During the login procedure, if your browser says 'Login successful', while the Globus client itself says 'Browser login did not complete. Error: ConnectionError on request', as in the image above, please log off the VPN and try the install again.

Windows Notes:

By default, you will need adminstrative access to install Globus Connect Personal on Windows. This is because it will attempt to install into C:\\Program Files (x86)\Globus Connect Personal. To install as a regular, non-administrative account, change the installation directory to which you have write access, for example Desktop --> Globus Connect Personal.
A windows defender window may pop up during installation (Windows protected your PC). Selecting "More info" should a "Run anyway" button appear that will allow you to proceed.

Note:During installation, you will be offered the option of a 'High Assurance' endpoint. Do not select this option. NIH does not have a High Assurance Globus subscription.

Can't see your network drive in Globus?

By default, Globus Connect Personal on your desktop system can only see your home directory. Any other area, such as a network drive, must be enabled in Globus. To configure your Globus Connect Personal instance to access other paths, see the instructions for
Windows
Mac
Linux

Transferring data between your desktop and Biowulf

Once you have Globus installed, go to the documentation about starting a transfer.