We use the GPFS filesystem on the central cluster. It is currently sized at around 2.5 Petabytes. This houses both user home directories and group research directories. It is designed to be scalable as the need arrises.
There are different tiers of storage within the filesystem. We store all metadata in a system SSD tier. This allows metadata operations such as file creates to go faster. The filesystem is also designed to store small files within the metadata inode itself. Each metadata inode is 4k, about 0.5k is used for the metadata, leaving about 3.5k behind. If you file is smaller that the 3.5k, it will be stored in the metadata inode.
Most larger data goes onto spinning disk Do to the large number of drives and the parallel nature of the filesystem, writes can go quite fast to the filesystem.
Some types of workloads perform better on SSD. i some limited cases when can put data on the remainder of our SSD tier. This is done via filesets and policies in the filesystem.
Every user has a home directory on the filesystem. The default quota for each user in the home directory is 50GB.
Groups can also get storage on the system for housing larger datasets and research data. If your group needs this please contact us and we can set up a directory for you.
You can see the rates for storage here
Checking your quota
mmlsquota --block-size AUTO central:home
We have implemented a script to email you when you go over your soft quota. The email is sent to you access.caltech.edu associated account.
You may want to find attributes of you files. These include various dates related to your file (creation, modify, access), file type, where it sits.
To find general information about the file:
[naveed@login2 ~]$ stat myfileFile: ‘myfile’Size: 7 Blocks: 8 IO Block: 8388608 regular fileDevice: 28h/40d Inode: 6078490 Links: 1Access: (0644/-rw-r--r--) Uid: ( 8688/ naveed) Gid: ( 102/citstaff)Access: 2018-03-16 08:00:49.162288264 -0700Modify: 2018-03-16 08:00:47.471020000 -0700Change: 2018-03-16 08:00:47.470986618 -0700Birth: -
[naveed@login2 ~]$ file myfile myfile: ASCII text
[naveed@login2 ~]$ mmlsattr myfilereplication factorsmetadata(max) data(max) file [flags]------------- --------- ---------------1 ( 2) 1 ( 2) myfile[naveed@login2 ~]$ mmlsattr -L myfilefile name: myfilemetadata replication: 1 max 2data replication: 1 max 2immutable: noappendOnly: noflags:storage pool name: datafileset name: homesnapshot name:creation time: Fri Mar 16 08:00:47 2018Misc attributes: ARCHIVEEncrypted: no