Mass Storage System Policies

When using NREL's Amazon Web Services (AWS) Mass Storage System (MSS) to archive high-performance computing (HPC) data, please follow these policies.

Quotas

Both the amount of data and the number of files will be managed with quotas.
Users with accounts on any NREL HPC system will be provided with storage space on the AWS MSS system for personal storage. Every system user receives a default 1 terabyte (TB), 1 million file quota automatically when their account is created.
Projects that receive an allocation on HPC systems will be provided with MSS project quota determined as part of the allocation process. Allocations receive a default 2 TB, 2 million file quota when the allocation is created.

AWS MSS data will be retained for 15 months from the date written. This differs from the previous retention policy of minimum 15 months with deletion if needed.
Restore requests of MSS data that are archived in AWS MSS will initially require a request to the HPC Help Desk and may require 48 hours to be able to recover.

The largest file size AWS S3 can manage is limited to 5 TB.
A limit of 1,025 files and/or directories (inodes) at a time has been placed on /shared-projects/MSS/home/<username> and /shared-projects/MSS/projects/<project_handle> to help minimize the cost of data transfers.