Versions Compared

Old Version 57

changes.mady.by.user Andy Bergman

Saved on

compared with

New Version 58

changes.mady.by.user Cox, Calvin D.

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: small conversion of 'ecommons' to 'hms id' in "transfers on the O2 File Transfer Servers"
Table of Contents

...

  • scp, sftp, rsync - these are automatically installed on Mac and Linux

  • pscp, psftp - Windows-only. These can be installed with the PuTTY ssh program.

  • ftp - available on O2 for downloading from external sites which only accept FTP logins. But, O2 does not accept incoming FTP logins.

  • aspera - a data transport and streaming technology, now owned by IBM.

  • awscli - Amazon AWS command line interface

  • basemount - an Illumina tool to mount BaseSpace Sequence Hub data.

  • bbcp - a point-to-point network file copy application from NERSC

  • lftp - can transfer files via FTP, FTPS, HTTP, HTTPS, FISH, SFTP, BitTorrent, and FTP over HTTP proxy.

  • gcloud - Google Cloud command line interface, including the gsutil command

  • rclone - rsync for cloud storage

...

You can connect to the transfer nodes using ssh at the hostname: transfer.rc.hms.harvard.edu . If you're on Linux or Mac, you can use the native terminal application to connect to the transfer nodes. If you're on Windows, you will need to install a program to connect to the transfer servers; we recommend MobaXterm. In either terminal or MobaXterm, type the following command:

Code Block
ssh yourecommons@transferyourhmsid@transfer.rc.hms.harvard.edu

where you substitute yourecommons yourhmsid for your actual HMS ID (formerly known as eCommons ID) in lowercase. Once you authenticate, you'll be on one of the transfer servers. From here, you can enter commands like scp, sftp, rsync,etc. See the above section on command line tools available on the O2 File Transfer Servers for more details. You can run transfer commands directly on the transfer servers after logging in. We do not have a job scheduler running on the transfer cluster, so you do not need to submit an sbatch job or request an interactive session with srun to run such transfer processes. The transfer servers do not have any research applications (modules) are not available, as well.

...