Storage

User account Space

Each user account has a quota of 50GB in the home directory. Additional space is available, please send email to help-hpc@caltech.edu for information. 

Group Space

Each research group has a quota of 10TB in /central/groups/<groupname>. Additional space is available, please send email to help-hpc@caltech.edu for information. 

Scratch Space

There are two scratch directories available. A 500TB of standard, high speed disk mounted on /central/scratch and a 30TB high IO disk mounted on /central/scratchio.  Best practice dictates creating directory for yourself (e.g. /central/scratch/<username>) and working with files from inside of said directory.  These disks are truly meant as scratch space. Any files not accessed in 14 days will be automatically purged

Checking Quotas for User and Group

To check see how much storage you are using you can use the mmlsquota

mmlsquota -u username  --block-size auto central:home

To check for your group storage

mmlsquota -j  groupname  --block-size auto central

Snapshots

The GPFS based file system uses snapshot technology that will capture file changes in the following way:

  • Every 4 hours for 1 day
  • Every day for 1 week
  • Every week for 4 weeks
The snapshot directory is not listable, but can be found be changing directory to ".snapshots" .

Backup and Archive

There is no formal BCP/DR style back up nor archival system in place. For information on running backups using the Duplicity client see this page. (Duplicity supports saving backups to AWS, Google, Backblaze B2, ssh based hosts and others.)