Purpose

The purpose of the "myquota" command and associated commands "home-quota", scratch-quota", and "prjspc-quota", (see below) is to enable end-users to query allocated limits and current usage of relevant disk quota. The tools simplify obtaining such an up-to-date overview, tailored to the calling end-user, with virtually no effort on the part of the user.

The tool has the same purpose as the Cartesius version, but it is a total rewrite with a different organisation adapted to gpfs, using gpfs fileset quota for project spaces rather than group quota, and to the Snellius file systems.

• The output format of the different types of quota (for home, scratch, and project spaces) has been made uniform in that usage is consistently displayed as a percentage of quota limits - unless no limits have been set.
• Since scratch and home quota are 'personal', in that they are user quota, pertaining to the files that a user owns on a particular file system, they are automatically displayed for the calling user.
• Project spaces are not included by default.  However, users can customize what specific project space(s )to include in the overview  by naming them in an environment variable: "MYQUOTA_PROJECTSPACES".
• The relevant commands all have built-in help, that is displayed by invoking the command with the "--help" option.

If you see that you are near or at your quota, you might be interested in this tutorial on how to efficiently clean up your home directory or project space.

How to use it

export PATH=$PATH:/gpfs/admin/hpc/usertools/

myquota

Usage: myquota --help [ home-quota | scratch-quota | prjspc-quota ]
       myquota [ projectspace .. ]

Description:

List home and scratch quota and usage pertaining to the current user, and optionally also list project space quota for any specified project space of interest to you. Instead of specifying any project spaces on the command line, a (space-separated list of) project space(s) can also be stored in the environment variable MYQUOTA_PROJECTSPACES. If this variable is non-empty and command-line arguments are specified as well, the specified command-line arguments overrule the contents of the environment variable. In this way, you can easily customize the default working of the myquota, by adding an export statement in your login profile, like so:

export MYQUOTA_PROJECTSPACES="project_x project_y".

Note that there is no need to use the complete pathname. Since we insist on unique project space names across file systems, just the basename(s) will do.  The myquota script is a convenient shorthand for calling the following custom Snellius scripts, that have built-in knowledge of  allocation and quota regimes, to do the actual work:

• home-quota
• scratch-quota
• prjspc-quota

All three can also be used directly and independently of the myquota  tool, and offer the options to see more detail of the reporting of the native gpfs "mmlsquota" command. Consult the help of these commands for more information.

Options:

--help [ home-quota | scratch-quota | prjspc-quota ]
Display the built-in help text of the 'myquota' command and exit.

Alternatively, display the built-in help of a tool handling a specific quota type, to learn
more about the differences, or if you have difficulty interpreting command output.

Example:

# Preferably in your (bash) profile if structurally relevant.
$ export MYQUOTA_PROJECTSPACES="reframe0 reframe4"

$ /gpfs/admin/hpc/usertools/myquota
user_x@home2|GiB:quota=200.00,limit=210.00,usage=1.6821%|Inodes:quota=<none!>,limit=<none!>,usage=81641
user_x@wstor_scratch1|TiB:quota=8.00,limit=10.00,usage=9.6562%|Inodes:quota=3000000,limit=4000000,usage=2.5861%
/gpfs/work4/1/reframe0|TiB:quota=1.00,limit=1.00,usage=68.5083%|Inodes:quota=1000000,limit=1100000,usage=4.8824%
/gpfs/work3/1/reframe4|TiB:quota=20.00,limit=20.00,usage=9.1554%|Inodes:quota=2339732,limit=2573705,usage=9.0111%

home-quota

Usage: home-quota --help
       home-quota [ --raw | --include-raw ] [ username ]

Description:

List usage, as well as capacity and inode quota, pertaining to the user on the file system implied by the /home/<username> path.
If no username is explicitly specified, the current value of the environment variable 'USER' is used. Note that non-privileged users are not permitted to query user quota pertaining to another user.

Home capacity quota and hard limit are reported in GiB. Inode quota and hard limit are reported in exact numbers. Usage of both capacity and inodes is reported as a percentage of quota, unless no quota is set.

Options:

--help
Display the built-in help text and exit

--raw
Output the generated gpfs mmlsquota command and its "raw", uninterpreted, output.

--include-raw
Prefix the normal output with the generated gpfs mmlsquota command and its "raw", uninterpreted, output.

Command output:

The output of the home-quota command is a single line, that is best explained with the help of an example for an anonymized user, "user_x", that has a home directory residing on the home3 file system, that might look like
this:

user_x@home3|GiB:quota=200.00,limit=210.00,usage=31.6831%|Inodes:quota=<none!>,limit=<none!>,usage=171595

The output line consists of three pipe-separate segments. The first segment identifies what the quota and current usage data in the second and third parts refer to. Since this is home directory quota, implemented as user quota pertaining to the specific file system where the user has h(er|is) home directory, the identifier has a <user>@<filesystem device> format: user_x@home3

The second segment shows capacity quota and hard limit in GiB, and current usage, reported as a percentage of quota, unless no quota maximum has been defined: GiB:quota=200.00,limit=210.00,usage=31.6831%

Note that for capacity the hard limit in this example is 110% of quota, which allows peaking for a short grace period above the defined quota without further consequences, as long as the hard limit is not hit, and usage falls below the regular quota limit again before the grace time interval expires.

The third segment shows the inode quota and hard limit, in absolute numbers, and the current number of inodes in use, as a percentage of quota, unless no quota maximum has been defined: Inodes:quota=<none!>,limit=<none!>,usage=171595

scratch-quota

Usage: scratch-quota --help
       scratch-quota [ --raw | --include-raw ] [ username ]

Description:

List usage, as well as capacity and inode quota, pertaining to the user on the scratch file system, where both /scratch-local and /scratch-shared reside. If no username is explicitly specified, the current value of environment variable 'USER' is used. Note that non-privileged users are not permitted to query user quota pertaining to another user.

Scratch capacity quota and hard limit are reported in TiB. Inode quota and hard limit are reported in exact numbers. Usage of both capacity and inodes is reported as a percentage of quota, unless no quota is set.

Options:

--help
Display the built-in help text and exit

--raw
Output the generated gpfs mmlsquota command and its "raw", uninterpreted, output.

--include-raw
Prefix the normal output with the generated gpfs mmlsquota command and its "raw", uninterpreted, output.

Command output:

The output of the scratch-quota command is a single line,that is best explained with the help of an example for an anonymized user, "user_x", that uses the Snellius scratch facilities, /scratch-local and /scratch-shared, both
residing on the wstor_scratch1 file system. The output might look like this:

user_x@wstor_scratch1|TiB:quota=8.00,limit=10.00,usage=9.1562%|Inodes:quota=3000000,limit=4000000,usage=4.0861%

The output line consists of three pipe-separate segments. The first segment identifies what the quota and current usage data in the second and third parts refer to. Since this is the scratch quota, implemented as user quota pertaining tovthe scratch file system, the identifier, like with home directory quota, has a <user>@<filesystem device> format: user_x@wstor_scratch1

The seconds segment shows capacity quota and hard limit in TiB, and current usage, reported as a percentage of quota, unless no quota maximum has been defined: TiB:quota=8.00,limit=10.00,usage=0.0562%

Note that for capacity, the hard limit in this example is 125% of quota, which allows peaking for a short grace period above the defined quota without further consequences, as long as the hard limit is not hit, and usage falls below the regular quota limit again before the grace time interval expires.

The third segment shows the inode quota and hard limit, in absolute numbers, and the current number of inodes in use, as a percentage of quota, unless no quota maximum has been defined: Inodes:quota=3000000,limit=4000000,usage=0.0861%

Note that for inodes, the hard limit in this example is 133.33% of quota, which allows peaking for a short grace period above the defined quota without further consequences, as long as the hard limit is not hit, and usage falls below the regular quota limit again before the grace time interval expires.

prjspc-quota

Usage: prjspc-quota --help
       prjspc-quota [ --raw | --include-raw ] <prjspace>

Description:

List usage, as well as capacity and inode quota, pertaining to <prjspace>, where <prjspace> denotes the root directory of a  project space allocated on Snellius. Project space capacity quota and hard limit are reported in TiB. I-node quota and hard limit are reported in exact numbers. Usage of both capacity and I-nodes is reported as a percentage of quota, unless no quota is set.

Options:

--help
Display the built-in help text and exit

--raw
Output the generated gpfs mmlsquota command and its "raw", uninterpreted, output.

--include-raw
Prefix the normal output with the generated gpfs mmlsquota command and its "raw", uninterpreted, output.

Examples:

 Project spaces can be specified by a pathname (see 1, 2), or,  since there should be no ambiguity in the file system context, just by their unqualified basename (see 3):

  1.: prjspc-quota /projects/0/ourprjspace
  2.: prjspc-quota /gpfs/work2/2/yourprjspace
  3.: prjspc-quota theirprjspace

Command output:

The output of the prjspc-quota command is a single line, that is best explained with the help of an example for an anonymized project space, "project_x", that might look  like this:

/gpfs/work1/0/project_x|TiB:quota=55.00,limit=55.00,usage=92.9427%|Inodes:quota=3971918,limit=4369109,usage=41.0960%

The output line consists of three pipe-separate segments. The first segment identifies what the quota and current usage data in the second and third parts refer to. Since this is  project space quota, implemented as gpfs file set quota, the identifier is the absolute pathname to the project space:  /gpfs/work1/0/project_x

The second segment shows capacity quota and hard limit in TiB, and current usage, reported as a percentage of quota unless  no quota maximum has been defined: TiB:quota=55.00,limit=55.00,usage=92.9427%

 The third segment shows the Inode quota and hard limit, in absolute numbers, and the current number of inodes in use, as a percentage of quota, unless no quota maximum has been defined:  Inodes:quota=3971918,limit=4369109,usage=41.0960%

 Note that for inodes the hard limit in this example is 110% of quota, which allows peaking for a short grace period above the defined quota without further consequences, as long as the hard limit is not hit, and usage falls below the regular quota limit again before the grace time interval expires.