Aspera is a tool that can be leveraged to quickly and efficiently download data. This page details how to use it to download files as needed.
Log on to O2
If you need help connecting to O2, please review the How to login to O2 wiki page.
...
From a Mac Terminal, use the ssh command, inserting your eCommons HMS ID instead of user123:
| Code Block |
|---|
|
ssh user123@o2.hms.harvard.edu
|
Log in to transfer cluster and create working folder
| Code Block |
|---|
|
ssh transfer
# Make a scratch3scratch directory for aspera
# This assumes you have a scratch directory already
# "{USER:0:1}" will be replaced by the first letter of your eCommonsHMS ID
# $USER will be replaced by your eCommonsHMS ID
mkdir /n/scratch3scratch/users/${USER:0:1}/$USER/testAspera
# change to the test directory for aspera
cd /n/scratch3scratch/users/${USER:0:1}/$USER/testAspera |
...
| Code Block |
|---|
|
ascp -i /opt/aspera/etc/asperaweb_id_dsa.openssh ...
# for example:
ascp -i /opt/aspera/etc/asperaweb_id_dsa.openssh -QT -l 300m -P33001 era-fasp@fasp.sra.ebi.ac.uk:vol1/fastq/SRR136/029/SRR13673729/SRR13673729_2.fastq.gz . |
Additional tips:
If you need download a lot of data, run the screen command before ssh to transfer cluster, to keep the session alive.
If you have more than a dozen samples to download, running downloads one by one requires a lot of time. You can run them in parallel instead. For example you can submit 5 jobs, but let each job download 100 samples. If you try this method, please share your experience with us. We are always happy to hear feedback and to modify our documentation and recommendations accordingly!
Contact us to let us know if you have any questions.