Data Retention Policy

To protect against data loss, you must understand the NREL High-Performance Computing (HPC) Data Retention Policy.

File storage areas on Kestrel and the Mass Storage System (MSS) are either user-centric:

  • /home/<username>
  • /scratch/<username>
  • Mass Storage System

or project-centric:

  • /projects/<project-id>  
  • Mass Storage System.

Project-Centric Data

Project data on Kestrel (in /projects/<project-id>) will be retained for 3 months after the project allocation ends to allow project members time to move data either to local resources or to MSS. The ability to write new data to /projects will been disabled for those projects that have expired. Project members will be notified of the need to move their files when their project ends. The HPC leads will be notified again at the end of this 3-month period.

Project data in MSS will be retained for 15 months from the date the data was written to AWS. After this retention period, we reserve the right to delete files if we need to reclaim storage. We can make special arrangements for permanent storage, if needed.

User-Centric Data

The disk quota for /home directories on Kestrel is 50 GB.

Users accounts that are associated only with expired projects will be deactivated 3 months after the last project they are associated with ends. The retention period for files in /home/<username> is 3 months after the last project ends.  During this retention period, the user may log in to transfer data to external storage or to their MSS directories. User files in MSS will be retained for 15 months from the date the data was written, after which we reserve the right to delete files to reclaim MSS storage.

User accounts are locked and access to files is lost when the Appropriate Use Policy (AUP) period expires or contractual arrangements with NREL end. This may occur during an allocation period (depending on when the AUP period began). The typical AUP period for external users (non-NREL employee) is 1 year. Users will be warned about AUP expiration before the expiration date.

User accounts will be deactivated if they are inactive (no logins) for 6 months. Files in /home/<username> will be retained for 3 months after account deactivation. Users may request that accounts be reactivated during this time to enable files to be moved.

Files in /scratch/<username> are subject to purging to prevent the file system from filling up. Files are removed based on the last access time, with the least recently accessed files removed first. Files that haven’t been accessed in 28 days are eligible for deletion. If you would like to be notified in advance before the files are deleted you can add an email address to /scratch/$USER/.notify-email and you will receive notifications if you have files or directories that will be deleted in 7 days. Be sure you make this file hidden using a . (dot) in front of the notify-email file name.

Data Storage and Transfer

To learn about storing and transferring data on NREL HPC systems, see Services.

Share