NOTICE: FULL O2 Cluster Outage, January 3 - January 10th

O2 will be completely offline for a planned HMS IT data center relocation from Friday, Jan 3, 6:00 PM, through Friday, Jan 10

  • on Jan 3 (5:30-6:00 PM): O2 login access will be turned off.
  • on Jan 3 (6:00 PM): O2 systems will start being powered off.

This project will relocate existing services, consolidate servers, reduce power consumption, and decommission outdated hardware to improve efficiency, enhance resiliency, and lower costs.

Specifically:

  • The O2 Cluster will be completely offline, including O2 Portal.
  • All data on O2 will be inaccessible.
  • Any jobs still pending when the outage begins will need to be resubmitted after O2 is back online.
  • Websites on O2 will be completely offline, including all web content.

More details at: https://harvardmed.atlassian.net/l/cp/1BVpyGqm & https://it.hms.harvard.edu/news/upcoming-data-center-relocation

sratoolkit/2.10.7 to download dbGAP data

Due to Aspera license limitations, users have to install the software under their home to user the software. 

Start interactive job, and create working folder and load sratoolkit modue

For example, for user abc123, the working directory will be

srun --pty -p interactive -t 0-12:0:0 --mem 2000MB -n 1 /bin/bash mkdir /n/scratch/users/${USER:0:1}/${USER}/testDbGaP cd /n/scratch/users/${USER:0:1}/${USER}/testDbGaP module load sratoolkit/2.10.7

Configure sratoolkit. Only need to do this once:

# Configure sratoolkit vdb-config --interactive # Directly press x key to quit # By default, sratoolkit uses working diretory as cache. It is better to use scratch instead: echo /repository/user/main/public/root = \"/n/scratch/users/${USER:0:1}/${USER}/ncbi\" >> ~/.ncbi/user-settings.mkfg

Downoading dbGaP repository Key and upload to O2:



# Upload the dbGaP repository Key to O2: scp ~/Download/prj_phs710EA_test.ngc $USER@transfer.rc.hms.harvard.edu:~/.ncbi

Use sratoolkit prefetch, which try ascp then http,  to download sra data, then convert the data from .sra to .fastq format