NOTICE: FULL O2 Cluster Outage, January 3 - January 10th
O2 will be completely offline for a planned HMS IT data center relocation from Friday, Jan 3, 6:00 PM, through Friday, Jan 10
- on Jan 3 (5:30-6:00 PM): O2 login access will be turned off.
- on Jan 3 (6:00 PM): O2 systems will start being powered off.
This project will relocate existing services, consolidate servers, reduce power consumption, and decommission outdated hardware to improve efficiency, enhance resiliency, and lower costs.
Specifically:
- The O2 Cluster will be completely offline, including O2 Portal.
- All data on O2 will be inaccessible.
- Any jobs still pending when the outage begins will need to be resubmitted after O2 is back online.
- Websites on O2 will be completely offline, including all web content.
More details at: https://harvardmed.atlassian.net/l/cp/1BVpyGqm & https://it.hms.harvard.edu/news/upcoming-data-center-relocation
Using Aspera to Download Data
- Lingsheng Dong
- Raffaele Potami
- Neil Coplan
Aspera is a tool that can be leveraged to quickly and efficiently download data. This page details how to use it to download files as needed.
Log on to O2
If you need help connecting to O2, please review the How to login to O2 wiki page.
From Windows, use MobaXterm (preferred) or PuTTY to connect to o2.hms.harvard.edu and make sure the port is set to the default value of 22.
From a Mac Terminal, use the ssh command, inserting your HMS ID instead of user123:
ssh user123@o2.hms.harvard.eduLog in to transfer cluster and create working folder
ssh transfer
# Make a scratch directory for aspera
# This assumes you have a scratch directory already
# "{USER:0:1}" will be replaced by the first letter of your HMS ID
# $USER will be replaced by your HMS ID
mkdir /n/scratch/users/${USER:0:1}/$USER/testAspera
# change to the test directory for aspera
cd /n/scratch/users/${USER:0:1}/$USER/testAspera Download data use aspera follow the instructions from the data center.Â
ascp -i /opt/aspera/etc/asperaweb_id_dsa.openssh ...
# for example:
ascp -i /opt/aspera/etc/asperaweb_id_dsa.openssh -QT -l 300m -P33001 era-fasp@fasp.sra.ebi.ac.uk:vol1/fastq/SRR136/029/SRR13673729/SRR13673729_2.fastq.gz .
Additional tips:Â
If you need download a lot of data, run the screen command before ssh to transfer cluster, to keep the session alive.
If you have more than a dozen samples to download, running downloads one by one requires a lot of time. You can run them in parallel instead. For example you can submit 5 jobs, but let each job download 100 samples. If you try this method, please share your experience with us. We are always happy to hear feedback and to modify our documentation and recommendations accordingly!