...
To provide more robust and reliable storage, HMS IT has deployed a new storage cluster, designated as /n/scratch, to replace /n/scratch3.
| Info |
|---|
The /n/scratch3 filesystem |
...
was retired on Jan 16, 2024 The timeline for this update was:
|
Please update your workflow to use/n/scratch see our documentation about changes in the new Scratch Storage
If you have any questions or concerns, contact Research Computing at rchelp@hms.harvard.edu
...
|
Scratch3 Overview
| Info |
|---|
|
Scratch3 Configuration
Scratch3 is a storage location on O2 for temporary data. In summary:
THE SCRATCH SPACE IS NOT BACKED UP!
Files are AUTOMATICALLY DELETED 30 days after they are last accessed.
There are NO SNAPSHOTS available SNAPSHOTS available for scratch3 folders.
O2 users can have up to 10 TiB and 1 million files/directories in their scratch folder.
Use this space for intermediate files in a longer workflow, or for extra output that doesn't matter much.
To reiterate, if you delete a file on purpose or by accident, or just leave it sitting for 30 days, it is GONE. You can't get it back, and Research Computing can't get it back for you. Using this model lets us provide cost-effective, high-end storage to many more users than would be possible with permanent and backed-up files. But it is important that you understand the risks of using it.
Important - Artificially modifying file access times is against policy and may result in losing your access to scratch3.
Scratch3
...
A new scratch space is now available on O2, mounted at /n/scratch3.
Users cannot write files directly under /n/scratch3 itself. A large number of user directories have been pre-created on scratch3. If you don't have a scratch3 directory already, then you must use the scratch3 create directory script.
** The old /n/scratch2 was retired on June 26 2020 **
...
Directories: Users
On scratch3, user directories are found under:
/n/scratch3/users/<first_eCommonshms_id_char>/<eCommons><HMSID>
Where "<first_eCommonshms_id_char>" is the first character of your eCommons id HMS ID (formerly eCommons ID) and "<eCommons><HMSID>" is is your eCommons HMS ID.
For example, the scratch3 directory for a user with the username "abc123" would be located at:
/n/scratch3/users/a/abc123
For an eCommons HMS ID of zz999 the scratch3 directory will be at:
/n/scratch3/users/z/zz999
...
Scratch3
...
/n/cluster/bin/scratch3_create.sh
Note: As mentioned above, a large number of user's directories have been pre-created on scratch3. If you have a pre-created scratch3 directory and run the script, then it will inform you that you have a directory already.
...
Storage Usage
The storage usage is on a user basis rather than a group. To check the storage usageyour personal and group storage utilization and limits, please run the following command:
quota-v2
More information about the O2 quota-v2 command can be found on this wiki page.
HMS Research Computing retired the old tool for reporting storage utilization and limits called quota on August 8, 2023.
The quota-v2 tool retrieves more comprehensive information than the previous quota tool, executes faster, and runs on all O2 nodes (login, compute, and transfer cluster).