Checking my quota and other filesystem attributes

Our Filesystem

We use the GPFS filesystem on the central cluster. It is currently sized at around 2.5 Petabytes. This houses both user home directories and group research directories. It is designed to be scalable as the need arrises.

There are different tiers of storage within the filesystem. We store all metadata in a system SSD tier. This allows metadata operations such as file creates to go faster. The filesystem is also designed to store small files within the metadata inode itself. Each metadata inode is 4k, about 0.5k is used for the metadata, leaving about 3.5k behind. If you file is smaller that the 3.5k, it will be stored in the metadata inode.

Most larger data goes onto spinning disk Do to the large number of drives and the parallel nature of the filesystem, writes can go quite fast to the filesystem.

Some types of workloads perform better on SSD. i some limited cases when can put data on the remainder of our SSD tier. This is done via filesets and policies in the filesystem.

Quotas Limits

Every user has a home directory on the filesystem. The default quota for each user in the home directory is 50GB.

Group Storage

Groups can also get storage on the system for housing larger datasets and research data. If your group needs this please contact us and we can set up a directory for you.

You can see the rates for storage here

Checking your quota

You can check you current quota by running the following command:


mmlsquota --block-size AUTO central:home

Email notification

We have implemented a script to email you when you go over your soft quota. The email is sent to you access.caltech.edu associated account.

File Attributes

You may want to find attributes of you files. These include various dates related to your file (creation, modify, access), file type, where it sits.

To find general information about the file:


[naveed@login2 ~]$ stat myfile
  File: ‘myfile’
  Size: 7         	Blocks: 8          IO Block: 8388608 regular file
Device: 28h/40d	Inode: 6078490     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 8688/  naveed)   Gid: (  102/citstaff)
Access: 2018-03-16 08:00:49.162288264 -0700
Modify: 2018-03-16 08:00:47.471020000 -0700
Change: 2018-03-16 08:00:47.470986618 -0700
 Birth: -

To find out file type information:


[naveed@login2 ~]$ file myfile
myfile: ASCII text

To find out gpfs attributes:


[naveed@login2 ~]$ mmlsattr myfile
  replication factors
metadata(max) data(max) file    [flags]
------------- --------- ---------------
      1 (  2)   1 (  2) myfile
[naveed@login2 ~]$ mmlsattr -L myfile
file name:            myfile
metadata replication: 1 max 2
data replication:     1 max 2
immutable:            no
appendOnly:           no
flags:
storage pool name:    data
fileset name:         home
snapshot name:
creation time:        Fri Mar 16 08:00:47 2018
Misc attributes:      ARCHIVE
Encrypted:            no

Backups

We are currently not backing user data. It is your resposibility to back up your own data. We are working on options for users to back up their data.