NOTICE: FULL O2 Cluster Outage, January 3 - January 10th

O2 will be completely offline for a planned HMS IT data center relocation from Friday, Jan 3, 6:00 PM, through Friday, Jan 10

  • on Jan 3 (5:30-6:00 PM): O2 login access will be turned off.
  • on Jan 3 (6:00 PM): O2 systems will start being powered off.

This project will relocate existing services, consolidate servers, reduce power consumption, and decommission outdated hardware to improve efficiency, enhance resiliency, and lower costs.

Specifically:

  • The O2 Cluster will be completely offline, including O2 Portal.
  • All data on O2 will be inaccessible.
  • Any jobs still pending when the outage begins will need to be resubmitted after O2 is back online.
  • Websites on O2 will be completely offline, including all web content.

More details at: https://harvardmed.atlassian.net/l/cp/1BVpyGqm & https://it.hms.harvard.edu/news/upcoming-data-center-relocation

Juicer

Owned by Lingsheng Dong
Last updated: Dec 11, 2023
2 min read

Please make sure you run this from login node, or submit it as sbatch job.

If you run it under interactive job, the srun command inside the script will not wait for the split jobs to finish and start alignment jobs before split, that will cause problems. 

# login to O2 ssh youraccount@o2.hms.harvard.edu # make a folder in scratch mkdir -p /n/scratch/users/a/abc123/juicerTest/fastq # go to the folder cd /n/scratch/users/a/abc123/juicerTest # copy over some testing data cp /n/shared_db/misc/juicerData/fastq/* fastq/ # run the workflow /n/shared_db/misc/juicer/juicer.short.2hours.2core.sh -g mm10 -d $PWD -s MboI -p /n/shared_db/misc/juicerReference/mm10.chrom.sizes -y /n/shared_db/misc/juicerReference/mm10_MboI.txt -z /n/shared_db/misc/juicerReference/Mus_musculus_assembly10.fasta -D /n/shared_db/misc/juicer/SLURM #output: (-: Looking for fastq files...fastq files exist (-: Aligning files matching /n/scratch/users/a/abc123/juicerTest/fastq/*_R*.fastq* in queue commons to genome mm10 with site file /n/shared_db/misc/juicerReference/mm10_MboI.txt (-: Created /n/scratch/users/a/abc123/juicerTest/splits and /n/scratch/users/a/abc123/juicerTest/aligned. srun: job 13037522 queued and waiting for resources # wait here for a while until the split job finishes srun: job 13037522 queued and waiting for resources srun: job 13037522 has been allocated resources list of splitted fastq files: total 1185828 -rw-rw-r--. 1 ld32 ld32 129231923 Apr 27 12:00 HIC003_S2_L001_R1_001.fastq000.fastq -rw-rw-r--. 1 ld32 ld32 129234906 Apr 27 12:00 HIC003_S2_L001_R1_001.fastq001.fastq -rw-rw-r--. 1 ld32 ld32 129255288 Apr 27 12:00 HIC003_S2_L001_R1_001.fastq002.fastq -rw-rw-r--. 1 ld32 ld32 129266976 Apr 27 12:00 HIC003_S2_L001_R1_001.fastq003.fastq -rw-rw-r--. 1 ld32 ld32 90124479 Apr 27 12:00 HIC003_S2_L001_R1_001.fastq004.fastq -rw-rw-r--. 1 ld32 ld32 129231923 Apr 27 12:00 HIC003_S2_L001_R2_001.fastq000.fastq -rw-rw-r--. 1 ld32 ld32 129234906 Apr 27 12:00 HIC003_S2_L001_R2_001.fastq001.fastq -rw-rw-r--. 1 ld32 ld32 129255288 Apr 27 12:00 HIC003_S2_L001_R2_001.fastq002.fastq -rw-rw-r--. 1 ld32 ld32 129266976 Apr 27 12:00 HIC003_S2_L001_R2_001.fastq003.fastq -rw-rw-r--. 1 ld32 ld32 90124479 Apr 27 12:00 HIC003_S2_L001_R2_001.fastq004.fastq # for large data set, please use /n/shared_db/misc/juicer/juicer.short.12hours.4core.sh ... # for troubleshooting, look log files in: ls -l debug # to run hiccups: module load gcc/4.8.5 java/jdk-1.8u112 cuda/8.0 sbatch -p gpu --gres=gpu:1 -t 120 --mem 3G --wrap "java -Xmx2G -jar /n/shared_db/misc/juicer/SLURM/scripts/juicer_tools.1.7.6_jcuda.0.8.jar hiccups -m 500 -r 5000,10000 -f 0.1,0.1 -p 4,2 -i 7,5 -d 20000,20000 --ignore_sparsity $PWD/aligned/inter.hic $PWD/aligned/hiccups" # this command will create result in: /n/scratch/users/a/abc123/juicerTest/hiccups





Digital Accessibility