Command-line tool to upload data to the NCI Cancer Genomics Cloud. This is a fast and secure upload client that has been optimized to efficiently upload files to the CGC, taking advantage of parallelization where possible.
Sample session (user input in bold):
[user@helix ~]$ module load cgc-uploader
[+] Loading cgc-uploader 20190517
[user@helix ~]$ cgc-uploader.sh -h
Upload files to Cancer Genomics Cloud
usage: cgc-uploader.sh [-h] [-l] [-p id] [-t token] [-x url] file ...
-a,--automation Start automation from manifest file.
This option must be used together with
--manifest-file.
--dry-run Dry run the upload (manifest) and/or
metadata setting process.
-f,--folder <arg> Specify optional folder, inside of a
specified project, to upload the files
into.
You can specify nested folder structure
separated by the path separator `/`.
If any of the specified folders is
missing it will be created.
-h,--help Print a short usage summary.
-l,--list-projects Print a list of projects available as
upload targets. The output is a
tab-separated list of project IDs and
names.
--list-tags Print a list of tags present in a project
and exit.
This option must be used together with
--project.
-mf,--manifest-file <arg> Specify manifest tabular file to set
metadata.
This option must be used together with
--project.
-mm,--manifest-metadata <arg> Parse metadata from manifest file.
You can list metadata field names as
argument to this option.
This option must be used together with
--manifest-file.
-p,--project <arg> Specify the ID of the project to upload
files to.
-pf,--preserve-folders Should the folder structure for specified
input folders be preserved while
uploading recursively.
By default, files encountered in the
nested folders are `flattened`, and
uploaded into root target folder.
With this flag, inner folders are created
along the way, and files are uploaded
into them.
--skip-partial Do not attempt to resume incomplete
uploads.
If omitted, the uploader will resume an
upload when the local file matches in
name and size.
-t,--token <arg> Specify an authorization token.
--tag <arg> Apply tag <arg> to all the files in this
upload.
This option may appear multiple times.
-u,--username <arg> Specify username.
If omitted and not using the -t option,
user will be prompted for a username.
-x,--proxy <arg> Specify a proxy server through which the
uploader should connect.
The URL to the proxy server in the form
proto://[user:pass]@host[:port].
Proto can be `http' or `socks'. Supports
SOCKS4 and SOCKS5.
The program outputs a tab-separated list of newly created remote file IDs
and local file names.
Complete documentation is available at:
http://docs.cancergenomicscloud.org/docs/upload-via-the-command-line
[user@helix ~]$