Biowulf High Performance Computing at the NIH
Setting up a Globus managed endpoint at NIH

Globus is a service that makes it easy to move, sync, and share large amounts of data. Globus will manage file transfers, monitor performance, retry failures, recover from faults automatically when possible, and report the status of your data transfer. Globus uses GridFTP for more reliable and high-performance file transfer, and will queue file transfers to be performed asynchronously in the background.

Globus was developed and is maintained at the University of Chicago and is used extensively at supercomputer centers and major research facilities. [Globus website]

The NIH HPC group at CIT has funded a Globus Provider Plan. This allows NIH system administrators to set up managed Globus endpoints at NIH under this provider plan, at no additional cost.

Caveats: The system Fully Qualified Domain Name needs to be resolvable from outside NIH. The Globus systems @globus.org need to be able to resolve the FQDN, since they manage the authentication tokens. The actual data transfer is performed directly between the two endpoints involved.

To set up a managed endpoint, you need to follow these steps.

  1. Select a name for your endpoint. The name should start with 'nih' so NIH users can easily find NIH endpoints. (e.g. 'nihdctg').
  2. Create a Unix group and user on your local system with this name.
  3. su to this newly created account and create ssh keys with 'ssh-keygen -t rsa'.
  4. Create a Globus account at globusid.org for this username ('nihdctg', in this example) and upload your private key. (Instructions here)
  5. Install Globus Connect Server. (Instructions here). Start up Globus and confirm that you can make some test transfers to your endpoint.
  6. Make your Globus endpoint publicly visible by editing the /etc/globus-connect-server.conf file and setting
    [Endpoint]
    Name = myendpointname
    Public = True
    
  7. Once you have your endpoint set up, send email to staff@hpc.nih.gov with the name of your endpoint. We will add this endpoint to the NIH Globus Provider subscription.
  8. Once your endpoint is added to the NIH subscription plan, you will need to make the endpoint managed. You can use the Web GUI and follow the instructions here: https://docs.globus.org/faq/globus-connect-endpoints/#how_do_i_convert_an_existing_endpoint_to_a_managed_endpoint
    Or follow the CLI instructions here: https://docs.globus.org/faq/subscriptions/#how_do_i_convert_an_existing_endpoint_into_a_managed_endpoint

To configure your server to use NIH Login for authentication, edit the /etc/globus-connect-server.conf file and set

IdentityMethod = CILogon
CILogonIdentityProvider = National Institutes of Health

Documentation

Add an ssh key to your globus account

Installing Globus Connect Server.

Configuring an endpoint to use CILogin identity provider