Quick Links NIH HPC Globus Endpoints Globus Setup NIH HPC (Biowulf) endpoints Logging into Globus with your NIH login Installing the Globus client Reading/writing to local drives Globus Data Transfers Biowulf to/from desktop desktop to/from desktop via the command line Recurring/Scheduled transfers to/from AWS to/from Google Cloud Storage to/from Google Drive shared Google Drive data to/from NIH OneDrive to/from NIH Box to/from NIH NCI DME Globus Sharing Sharing Data with Collaborators What to tell your collaborator Deleting a shared endpoint Extras Globus Plus Globus and NIH VDI Encryption and Security TroubleShooting Setting up a managed endpoint

Globus is a service that makes it easy to move, sync, and share large amounts of data. Globus will manage file transfers, monitor performance, retry failures, recover from faults automatically when possible, and report the status of your data transfer. Globus uses GridFTP for more reliable and high-performance file transfer, and will queue file transfers to be performed asynchronously in the background. Globus was developed and is maintained at the University of Chicago and is used extensively at supercomputer centers and major research facilities. [Globus website] No matter how you transfer data in and out of our systems, be aware that PII and PHI data cannot be stored or transferred into the NIH HPC systems. See the links in the Quick Links menu at left for details.

NIH summer interns using the Virtual Desktop Interface can utilize Globus for some, but not all data transfers.

• To transfer data between their VDI desktop and Biowulf: VDI users will not be able to install Globus Connect Personal on their VDI desktop. Instead, they should use WinSCP to transfer files between the VDI and Biowulf. If WinSCP is unavailable on the VDI, please create a service request in the NIH IT Service Desk under the "NIH Virtual Computing (VDI) request" category.

• To transfer data between another Globus endpoint and Biowulf: if data needs to be transferred between another Globus endpoint (e.g. the NIH sequencing center) and Biowulf, VDI users can use Globus. Open a browser on the VDI, go to globus.org, click the Login button on the top right, and log in to Globus. Select 'National Institutes of Health' as the institution, then enter the NIH login username and password. You will need to agree to the Globus Consent.

For some kinds of data transfer or sharing, you need Globus Plus. The NIH Globus subscription includes Globus Plus for all users, but you need to email staff@hpc.nih.gov to request a Globus Plus invite.

When do you not need Globus Plus?

• To transfer or share data between two Globus managed endpoints (e.g. two multi-user systems at different universities, each running a Globus server)
• To transfer data between a managed endpoint (e.g. Helix/Biowulf) to a Globus Connect Personal endpoint (e.g. your desktop)

When do you need Globus Plus?

• To transfer data between two Globus Connect Personal endpoints (e.g. your desktop system and a laptop).
• To share data from a Globus Connect Personal endpoint (e.g. your desktop system)
• To transfer data from/to a shared endpoint that is hosted on a Globus Connect Personal endpoint.

Note 1: if your collaborator needs Globus Plus to download data, and is not at NIH, we cannot provide Globus Plus to that person.

Note 2: By default, files on a Globus Connect Personal endpoint (e.g. your laptop or desktop) may not be shareable. You will need to configure that via the instructions at these links: Linux, Mac, Windows.

Data can be encrypted during Globus file transfers. In some cases encryption cannot be supported by an endpoint, and Globus Online will signal an error.

For more information, see How does Globus Online ensure my data is secure?

In the Transfer Files window, click on 'More options' at the bottom of the 2 panes. Check the 'encrypt transfer' checkbox in the options.

Alternatively, you can encrypt the files before transfer using any method on your local system, then transfer them using Globus, then unencrypt on the other end.

Note that encryption and verification will slow down the data transfer.

50+ transfers were performed with each set of parameters over the course of several days. Each transfer was of a single file, approx 1 GB. In our tests, compared to setting no parameters: Setting the 'Verify file integrity after transfer' (the default setting) added ~25% to the transfer time. Setting the 'Encrypt transfer' parameter added ~13% to the transfer time. Setting both Verify + Encrypt added 40% to the transfer time.

How does Globus Online ensure my data is secure?

Indepedent assessments of Globus security.

Installation error for Globus Connect Personal

During the installation of Globus Connect Personal's login procedure, if your browser says 'Login successful', while the Globus client itself says 'Browser login did not complete. Error: ConnectionError on request', please log off the VPN and try the install again.

If your computer is on the wired NIH network, switch to wireless and use the 'NIH Guest' network, and redo the install.

If your computer is on the 'NIH Staff' wireless network on campus, switch to the 'NIH Guest' network, and then redo the install.

Can't see your network drive in Globus?

By default, Globus Connect Personal on your desktop system can only see your home directory. Any other area, such as a network drive, must be enabled in Globus. To configure your Globus Connect Personal instance to access other paths, see the instructions for
Windows
Mac
Linux

Error when creating a shared endpoint

If you see the following error message when trying to create a shared endpoint: "Sharing state dir has invalid permissions" the likely cause is that your /home/$USER directory is at quota. Move some files to /data, or delete them if unnecessary, and try again.

Error when creating or reading a shared endpoint

If you see one of the following errors while trying to create, or access a previously created shared endpoint:

or

the usual reason is that the permissions on /home/$USER/.globus are misset. Run the following commands:

chmod 0700 /home/$USER/.globus
chmod 0700 /home/$USER/.globus/sharing
chmod 0400 /home/$USER/.globus/sharing/*

Error 'No Effective ACL rules' when trying to access a shared endpoint

If another Globus user has shared an endpoint with you, and when you try to access it you get No effective ACL rules on the endpoint Some Endpoint Name (111111-222222-3333333).

The likely cause is that the owner of the endpoint has shared it with a different email address than you use for Globus. This other email address is not linked to your Globus ID, and therefore you don't have permissions to see the data.

The fix is to find out exactly what email address the owner gave access to. Then you can either link that email address to your Globus ID (see instructions here) or you can ask the owner to give permissions to the email address you normally use for Globus.

Transfer terminated because it hit the deadline

All Globus transfers are assigned a deadline when they are created. So long as a transfer is continuing to make progress, that deadline will get automatically extended as needed until the transfer completes. If a transfer encounters faults, the user will need to "fix" them so that the transfer can continue - for example, the user may need to refresh their credentials, or ensure they have the appropriate permissions, etc. If the transfer is "inactive" (encountering faults without being "fixed") for 3 days, the transfer will expire.