Software environments

Software environments are sets of libraries and software packages available on Lisa and Snellius. The following rules are applied:

1. We provide at most three software environments at a time:
   1. "Production" - the current and the most recent software environment. This environment is fully supported. All new system-wide software installations are added to this environment.
   2. "Previous production" - the software environment released before the "Production" one. We do not install new software in this environment, but we can patch the existing system-wide installation if the request for patching is justified.
   3. "Deprecated" - the software environment released before the "Previous production" one. This environment is not supported and provided "as is". We do not modify the software in this environment.
2. All software environments on our systems are named by the year they were released in. For instance: 2021 ("Production"), 2020 ("Previous production"), 2019 ("Deprecated").
3. Every software environment contains the most recent stable versions of the libraries and software packages available upon release of the environment (see Periodic software maintenance / updates). Exceptions are old versions of libraries that are used as dependencies for other software.
4. Most of the software packages in the provided software environment are installed using the "foss" toolchain (BLAS + ScaLAPACK + OpenMPI + GNU Compilers). Other toolchains, e.g. "intel", are provided for local builds by experienced users only.
5. After significant changes to hardware or OS versions on our machines, we may consider removing the "Previous Production" and/or "Deprecated" software environments. This is due to possible compatibility issues between the software and the new hardware or low-level OS libraries and drivers.
6. Upon introduction of a new machine (for example, a new cluster), we only install the latest software environment on it.
7. Users are strongly advised to always use the most recent ("Production") software environment.

Requests for software installations

We distinguish between two types of software installation on our systems: local installation in the user's home folder; and system-wide installation. You can make a request for the software installation (local or system-wide) through ServiceDesk, provided you take the rules in the following sections into account.

The basic rules for the software installation are as follows:

1. We do not install software that was last released more than 2 years ago system-wide. Exception: if the developer community is still active, the software is relevant to the majority of our users, and the software can be installed on the hardware and OS available on our systems.
2. We do not install software system-wide if it is not versioned. We do not install software from commits in Git systems and we do not install software with a "fake" version, e.g. 0.0.0.
3. We do not install software system-wide if it doesn't have an official release (or still has only an alpha or beta versions).
4. We do not install more than two different versions of software per toolchain system-wide.
5. Approval of the system-wide installation depends on the number of users for whom the software is relevant. We keep track of all software installation requests. As soon as we see significant interest in the software among our users, we consider a system-wide installation of this software. Once the software is installed system-wide, we will do our best to inform all users who requested the installation of this software.
6. The request for the software installation (local and system-wide) should be a part of an EINFRA or a Big NWO Grant proposal and budgeted in the consultancy hours.
7. If a request for new software is not part of an EINFRA or a Big NWO Grant proposal, then it falls under the general rules described in this software policy.
8. If a request does not comply with the software policy, then the user should request an extension of his/her grant to add consultancy hours.

Help with installing software in a user's home folder

We advise users to use the EasyBuild framework to install the software in the user's home folder. We also offer limited general support (unless extended support via consultancy hours was granted). In particular, we can help resolving the site-specific issues, such as:

• Installing software without sudo rights, or in a custom prefix.
• Linking software against libraries already present on the system.
• Advising on the best optimization flags to use when building software.
• Advising on which system libraries to link against for the best performance.

Because of our limited resources and a large number of users on our systems, we can not:

• Offer general Linux/Unix support. You'll have to familiarize yourself with how typical software installation procedures work on Linux/Unix systems. Depending on the type of installation you're trying to do, you may need to learn how to install software from a source code. We advise that you read e.g. our documentation, this tutorial, or get your local ICT support or colleagues to help you. 
• Guarantee that we can perform an installation for you: some installation procedures are broken, ill-documented, or for other reasons take more time than is available within our support.

If you require assistance with the local software installation, please prepare your request well and provide the following information:

• name and desired version of the software
• (a link to) the download page of the software
• (a link to) the installation instructions on the Linux system
• the list of dependencies of the software
• a clear description of the steps you've taken to try and install it yourself
• the (complete) error message that prevented you from completing your software installation

System-wide software installations, not requiring a license

We will only consider installing software system-wide if it complies with the aforementioned rules.

Assessment of these criteria will be done by our HPCV advisors. We will let you know within 3 business days if we grant your request. If your request is granted, we will give you an estimate of how much time we will need to install the software.

To prepare your request well, please provide:

• name and desired version of the software
• (a link to) the download of the software
• (a link to) the installation instructions
• the list of dependencies of the software

If the request is granted, the software will be made available through our module environment.

System-wide software installations, requiring SURF to own a license

Requests for licensed software, where SURF would have to buy the license, will be judged on a case-by-case basis. There is a very limited budget available at SURF for this, so chances of getting such a request granted are limited as well.

The minimum requirement will be:

• Motivational letter from the user as to the need for this software. This should also contain proof that the software is suitable for HPC systems, e.g. it should demonstrate that the software can be used for parallel computation at sufficiently large scales.
• A proven, (very) substantial group of users that benefits, which will be weighed against the cost of the license. This means that SURF will generally wait until we collect multiple requests, before buying a license.

If the request is granted, the software will be made available through our module environment.

System-wide software installations, using a license owned by a third party

Sometimes, it is possible to use a university license to run software at SURF. However, this requires substantial manual configuration on our side, and also requires cooperation of the party hosting the license.

In this situation the user is responsible for:

• arranging with local IT (or whoever hosts the license) that the license server from the university can be approached by SURF infrastructure
• providing proof to SURF that the license allows this type of usage. If the license is unclear about this, the user will be responsible for contacting the vendor, and provide proof to SURF that the vendor approves. Even with this proof, SURF will accept no liability regarding this type of usage: the license owner remains responsible for the way the license is used.
• providing proof that this benefits a reasonably sized user group and that this user group will consume a sizable amount of SURF's compute resources through this setup.

Requests will be decided upon on a case by case basis by our HPC advisors. In this decision, the envisioned effort involved will be weighed against the amount of compute resources targeted through this setup.

If the request is granted, the software will be made available through our module environment, while generally restricting access to the software and/or license file to those users entitled to use the third party license.

System-wide software installations, requiring a (free) personal license

For legal reasons, SURF cannot perform system-wide software installations that require the acceptance of personal licenses. We can help the user to perform such installations in their home directory.

Periodic software maintenance / updates

The entire software stack as it is available in the module environment will periodically be updated. We aim to do this once per year. This means:

• Newer versions of the software already present on the system will be installed.
• This (newer) software will be built with more recent compilers.
• Software that was used very rarely in the past year, will not be included in this update.

In order to be able to maintain a large software stack with a reasonable amount of effort, we use the EasyBuild framework and rely on the EasyBuild community to provide most of the build recipes. In practice, this means that we do not install the very latest versions of software, but usually versions released in the previous year. An additional benefit to the user is that these are often more stable.

This policy is aimed to balance the convenience of a large software stack with the performance of (relatively) up to date code - a good balance for the majority of users. Experienced users, who prefer to use the most recent versions, can of course always install those versions in their home directory.

Support on existing installations

Definition of support on existing installations

Occasionally, software installations available in the module environment stop functioning correctly, for example due to an update to the operating system. If a user reports that a software installation is not functioning correctly, an HPCV advisor inspects the installation and decides the best solution. This could mean e.g. re-installation of the software, but could also mean that a user has to move to a newer version. Support in all cases is best effort and limited to the time specified in the contract with your university / institute. Therefore, support does not mean a guaranteed fix of the installation.

Duration of support

Existing software installations are supported for two years after their installation (according to the above definition of support), before being deprecated. Deprecated software installations will remain on the system for one more year, before they are removed. Note that the users can still use deprecated software, but since it is not supported anymore, the users are advised to update to newer versions as soon as possible.

The table below shows an example of the support period for different software stacks released in different years: