Search open search form

Storage

User account Space

Each user account has a quota of 50GB in the home directory. Additional space is available, please send email to help-hpc@caltech.edu for information. 

Group Space

Each research group has a quota of 10TB in /central/groups/<groupname>. Additional space up to 30TB is available at no charge, while anything above 30TB will be charged at standard storage rates, please have you PI send an email to help-hpc@caltech.edu for information. 

Scratch Space

There are two scratch directories available. A 500TB of standard, high speed disk mounted on /central/scratch and a 30TB high IO disk mounted on /central/scratchio.  Best practice dictates creating directory for yourself (e.g. /central/scratch/<username>) and working with files from inside of said directory.  These disks are truly meant as scratch space. Any files not accessed in 14 days will be automatically purged

Checking Quotas for User and Group

To check see how much storage you are using you can use the mmlsquota

mmlsquota -u username  --block-size auto central:home

To check for your group storage

mmlsquota -j  groupname  --block-size auto central

Snapshots

The GPFS based file system uses snapshot technology that will capture file changes in the following way:

  • Every 4 hours for 1 day
  • Every day for 1 week
  • Every week for 4 weeks
The snapshot directory is not listable, but can be found be changing directory to ".snapshots" .

Backup and Archive

There is no managed BCP/DR style back up nor archival system in place so on the central hpc cluster. Please be sure to migrate any critical data to systems or services outside of the cluster storage on a routine basis. For information on running backups using the Duplicity client see this page. (Duplicity supports saving backups to AWS, Google, Backblaze B2, ssh based hosts and others.)