Page Comparison

...

Connection parameters:

• host: transfer.rc.hms.harvard.edu

• port: 22  (the SFTP port) 

• username: your HMS ID (formerly known as eCommons ID), the ID you use to login to O2, in lowercase, e.g., ab123 (not your Harvard ID or Harvard Key) 

• password: your eCommons HMS ID password, the password you use when logging in to O2

Command line tools available on the O2 File Transfer Servers

• scp, sftp, rsync - these are automatically installed on Mac and Linux

• pscp, psftp - Windows-only. These can be installed with the PuTTY ssh program.

• ftp - available on O2 for downloading from external sites which only accept FTP logins. But, O2 does not accept incoming FTP logins.

• aspera - a data transport and streaming technology, now owned by IBM.

• awscli - Amazon AWS command line interface

• basemount - an Illumina tool to mount BaseSpace Sequence Hub data.

• bbcp - a point-to-point network file copy application from NERSC
  

• lftp - can transfer files via FTP, FTPS, HTTP, HTTPS, FISH, SFTP, BitTorrent, and FTP over HTTP proxy.

• gcloud - Google Cloud command line interface, including the gsutil command

• rclone - rsync for cloud storage

...

me@mydesktop:~$ scp myfile my_o2_id@transfer.rc.hms.harvard.edu:/n/scratch3/users/m/mfk8/

By default, scp will copy to/from your home directory on the remote computer. You need to give the full path, starting with a /, in order to copy to other filesystems.

...

File transfer processes are too resource intensive to run on the O2 login servers, but you can run these interactively from a compute node as you would any other application. Launch an interactive session with the following srun command, and then run your commands once logged into a compute node:

...

1. Use O2's dedicated file transfer servers: 

   1. SSH login to the hostname: transfer.rc.hms.harvard.edu . You will be connected to a system which has access to /n/files .
      

   2. Once logged in, just run your commands (e.g. rsync, scp, cp) normally without using sbatch.

   3. Transfer servers can not submit jobs to the cluster, and research applications (modules) are not available from those systems.

2. If you have a batch job workflow that must use /n/files , you can request access to be able to use the "transfer" job partition. This partition has access to a few lower performance compute nodes which mount /n/files . They are only recommended when using the transfer servers is not an option, as these nodes are slower and generally less available.

...

• Use O2's dedicated file transfer servers.

• SSH login to the hostname: transfer.rc.hms.harvard.edu . You will be connected to a system which has access to /n/standby
  

• Once logged in, just run your transfer commands (rsync, cp, or mv) normally without using sbatch.

• Transfer servers can not submit jobs to the cluster, and research applications (modules) are not available from those systems.

...