Quick Links
Globus is a service that makes it easy to move, sync, and share large amounts of data. Globus will manage file transfers, monitor performance,
retry failures, recover from faults automatically when possible, and report the status of your data transfer. Globus uses GridFTP for more reliable
and high-performance file transfer, and will queue file transfers to be performed asynchronously in the background.
| Globus was developed and is maintained at the University of Chicago and is used extensively at supercomputer centers and major research facilities. [Globus website] No matter how you transfer data in and out of our systems, be aware that PII and PHI data cannot be stored or transferred into the NIH HPC systems. |
If your data is on the NIH HPC systems, you can easily share it with collaborators who are at NIH or elsewhere. All they will need is a (free) Globus account. The advantage of data sharing via Globus is that you do not need to transfer your data anywhere. This prevents data duplication, wastage of storage space, and saves time. You have full control over which files your collaborator can access, and whether they have read-only or read-write permissions.
Sharing data from NIH HPC systems (e.g. Helix or Biowulf):
- Your collaborator may be another Helix user, or at NIH, or at another institution. Regardless, you can share data from your Helix/Biowulf /home or /data area with that person. He/she will only need a Globus account.
Sharing data from a Globus Connect Personal endpoint (e.g. your desktop system)
- You will need a Globus Plus license to do this. Email staff@hpc.nih.gov to get one.
- If your collaborator is transferring this data to a Globus managed endpoint (e.g. Biowulf, or a server at his/her institution), only you will need a Globus Plus license.
- If your collaborator is transferring this data to another Personal endpoint (e.g. his/her desktop), then he/she will also need a Globus Plus license. If the collaborator is at NIH, they can email staff@hpc.nih.gov to get a Plus license. If they are not at NIH, they may be able to get a license from their own institution. If the collaborator does not have a Globus Plus license, they will not be able to transfer data from your desktop Globus personal endpoint to another Personal endpoint, but only to a managed Globus server.
Important Notes about Sharing:
- You can only share directories, not individual files.
- You should never share your /home or /data directory (the top level). Instead, to share data with a collaborator, you should create a subdirectory (preferably somewhere in your /data/$USER tree) and share that.
- You should share a directory only with a specific colleague or colleagues. Do not select 'all users' to share with.
- Globus allows directory trees to be shared as either read or read/write. This means that any subdirectories within that tree also have the same permissions.
- When you create the shared endpoint and give access to one or more Globus users, you can select whether each person has read or read/write access. If they have write access, they can also delete files within that directory tree, so you should be very careful about providing write access.
- A shared endpoint cannot be made write-only, which means no such thing as a 'blind dropbox'. This feature is being developed, but for now you must create individual endpoints for individual collaborators if you don't want all of them to have access to the same area.
- Once your data transfer or project is complete, and you no longer need to share data, you should delete the shared endpoint.
- Navigate to the Globus File Manager' on globus.org and select the subdirectory you want to share. Click on the 'Share' button in \ the right side pane.
- Click on 'Add a Guest Collection'
- Leave the Path unchanged. Add a Display Name and a Description (optional) and click 'Create Share'.
- Now you will want to share it with other Globus users. Click on 'Add Permissions - Share With'
- You can choose to share with other Globus users by selecting their email address, Globus username, or name. You can search with a name, or an email address. Select the user you want to share with.
- You can select 'Send Email' (default) and enter a message that your colleague will receive. By default, only read access to the directory ia provided. If you wish your colleague to transfer data into the directory, you can a\ lso provide write access by checking the 'write' button. Once the options have been selected, click 'Add Permission'.
- You should then see the share and the people you have shared it with. You can repeat this process for any number of collaborators\ . At any time, you can revoke access for a particular person by clicking the trash can next to the invitee in the last image above.
To see all endpoints you have shared, go to 'Endpoints' in the left bar, then 'Administered by You'.
It is highly recommended that you delete the shared endpoint when your collaborator has completed downloading the data.
If you set up a shared endpoint and want your collaborator to download the data, this is what you need to tell them.
First, the collaborator needs to get a Globus account. The instructions for setting up a Globus account are as described on our webpage above. This account is free. They may already have Globus access via their institution.
If the collaborator is downloading the data to his/her personal workstation, they need to install the Globus Connect client. Globus connect clients are available for Mac, Windows or Linux systems and are free.
When you share data with your collaborator, they will receive an email from Globus which contains a link.
They should click on the files they want to transfer, then 'Transfer or Sync to', enter their own endpoint name and desired path.
and click the 'Start' button near the bottom to start the transfer.
Once your data transfer or project is complete, and you no longer need to share data with your collaborator, you should delete the shared endpoint that you had created. This helps to ensure that you do not inadvertently put data into a directory that you had shared in the past and forgotten about, so that your data is now being shared without your conscious knowledge.
- Go to https://app.globus.org/endpoints.Select the 'Administered by You' tab.
- You should see a list of all the shared endpoints you have created. Click on the right-arrow next to the endpoint you want to delete.
- On the resulting page, you can check who you have shared it with (the 'Permissions' tab), or delete the endpoint.