...
User testing will begin on November 22, 2021 and will extend through the end of January 2022 to allow labs to adequately test their workflows and provide feedback.
This is an ongoing pilot and features may be added or removed throughout the process. Additionally, HMS IT internal testing will be ongoing in parallel with user testing.
The cold storage work streams that are being tested are not in production, and any data copied to the pilot platform is not replicated or backed up.
Pilot participants have been asked to identify a dataset(s) that is a representative subset of data you anticipate moving to Cold storage. For the purposes of this pilot testing, please follow this guidance:
Total test data size should be less than ~10 TiB per lab
Please MAKE A COPY of the data you will be testing and add “test” or something similar to the copy. We want to simulate what the data movement would look like in a real world scenario, which means we want to actually remove the data from the source and have the data be moved to Cold.
If you need assistance with copying data, please reach out to rdmhelp@hms.harvard.edu
Storage limits on lab folders where pilot testing is occurring can be temporarily increased during the pilot to accommodate duplicate datasets. If you anticipate needing a temporary increase, email rdmhelp@hms.harvard.edu.
All pilot participant testing will be completed in the Starfish GUI. Starfish and HMS are exploring the possibility of bringing the command line interface to HMS users for work post-pilot.
Both AWS Glacier Services and Azure Cool Blob Archive will be tested by pilot participants. At the start of the pilot, AWS will be tested, and then participants will test the same test workflows with Azure. The goal is to keep the testing of each space separate to avoid complexities with set up, testing, reporting, and feedback. We anticipate starting Azure testing in January; additional instructions will be provided at that time.
During our earlier meetings, we requested feedback on your preferred method for verifying and tracking data transfers to/from Cold storage. We determined that possible options could include email notifications, manifests in the source/destination folders, or a visual indicator in the online interface. We are currently working with Starfish to determine the best option moving forward. In the interim, you should be able to see that data has moved from one storage location to another when viewing the storage folders in the GUI.
...