O2 uses the Lmod system of environment modules; this system allows you to manage your environment to access separate applications that are installed on O2. Lmod also automatically tracks dependencies, so you do not need to worry about accidentally running a program in the wrong environment or against the wrong libraries. This is also how Lmod organizes the applications.

Viewing Modules

In order to see a list of all modules that are available to the user on O2, type into the terminal:

This command generates a verbose list of modules and their brief descriptions, which you can page through with <spacebar>, j or k, arrow keys, etc.

To view a list of modules that are available to load immediately, type into the terminal:

This command generates a concise list of modules that you can load given the current state of your environment.

To search all modules for specific keywords use the command: 

which will display information about all modules containing the keyword "string". This command can be useful when module spider does not return any result.

Loading Modules

To load a module, simply take one of the module strings from the output of the module viewing commands, and load it like so (in the example of GCC 6.2.0):

This will add GCC version 6.2.0 to your environment. You can confirm that it is there by typing:

to see a list of modules currently loaded in your environment. Sometimes you will see that there are more modules loaded than you have explicitly loaded yourself; this is because RC has tried to handle any codependencies that a program might have by having the main module load those extra modules in order to provide a more painless user experience (i.e. less need to troubleshoot a faulty module).

To unload a module, simply type:

(to unload gcc/6.2.0.) Any extra modules that were loaded with the module when you first loaded it also get unloaded when you unload that module. For instance, if you load module A, which automatically loads B and C, if you unload A, B and C also get unloaded. However, if you unload B or C, A will remain. We trust you will be careful with your dependencies, as Lmod can only mitigate so much user error.

To remove all modules from your environment, type:

and your environment will no longer have any modules loaded.

Dependency Switching

Lmod uses a hierarchical system in order to organize modules. The main divisions are Core, Linux, Compiler, and MPI.

Core

These modules are modules that - ideally - should function in a vacuum. They are generally compiled against system libraries, and form dependencies themselves for other programs. Tools in this category are compilers and programming languages that were not compiled (e.g. Java binaries).

Linux

These modules are modules that have not been compiled (e.g. are binaries downloaded from the source) that do not belong in Core. This includes programs such as bioinformatics tools that do not provide source code, e.g. Java programs. These tools can generally be loaded and executed regardless of the state of your environment.

Using software provided by BioGrids

Research Computing also offers a module ( biogrids/latest) that provides access to completely independent sets of over 400 pieces of biomedical software relative to the usual modules offered on O2. All In order to access this suite, simply run module load biogrids/latest. For more information, please refer to our Using Software Provided by BioGrids page.

Compiler

These modules are modules that were compiled against a compiler that was in the Core category. For example, if module A was compiled against GCC 6.2.0, it would be categorized under the gcc/6.2.0 hierarchy. In order to view module A, you need gcc/6.2.0 to be loaded in your environment.

The majority of applications on O2 were built against GCC 6.2.0. However, we have several older compilers under which older programs are installed. Feel free to use module spider or load a compiler module to check if we have a module installed.

MPI

These modules are modules that were compiled a certain MPI library, to enable MPI functionality. Any program compiled with MPI functionality will fall into this category.

Programs in this category are doubly nested: they were built against a specific MPI library that was in turn built against a specific compiler version. Therefore, you will need the correct compiler AND MPI module loaded in order to view the set of modules that will function under that library.

For now, the default MPI version on O2 is openMPI 2.0.1, which was built against GCC 6.2.0.

Module Management

Understanding the hierarchy is important to being able to access desired programs. You will not be able to load a module that was built against GCC 6.2.0 in the same environment as a module that was built against GCC 4.8.5. This is to promote a cleaner management of your PATH and environment, and to prevent programs from looking at the wrong libraries and dying.

This means if you have a workflow that might require multiple modules with different hierarchy presence to be utilized, you will no longer to be able to load all of your modules at the beginning of your pipeline; you will need to unload and load correct versions as you proceed with your workflow.

Lmod helpfully tells you when a module is no longer in its current and desired environment when you attempt to switch into a different hierarchy. For example, if you have module A loaded, and it was dependent on GCC 6.2.0, and you decide to load GCC 4.8.5 (which automatically unloads GCC 6.2.0), Lmod will tell you that module A is going inactive. Once you re-load GCC 6.2.0 (which automatically unloads GCC 4.8.5), it will become active again. To illustrate:

You will not be able to use inactive modules until their dependencies have been re-fulfilled.

Managing Environments - Collections

If you have a set of modules you load and use often, you can take a snapshot of your environment and set it up to be reloaded whenever you like. Once you have your list of modules loaded, type:

and it will record the list of modules, saved to the name you specified. You can have as many of these as you'd like.

To load a collection, use the following command:

To view the contents of a collection, use:

To view all of your collections, use:

Creating Your Own Modules

Lmod allows end users to manage their own software in tandem with O2's software stack. RC encourages this, as it allows greater flexibility in installation requirements. For example, if a lab requires a newer version of an existing software and RC cannot help for whatever reason, the lab can set up their own installation, create a modulefile for it, then add it to their list of available modules. This section will briefly describe how to do this.

If you would like to leverage Lmod to handle your own set of installed applications, you need to know how these applications were built and where relevant components are located. See our page on how to install your own software on O2 for more information on this.

Once you have your program installed/the information on how it was installed, you need to create a modulefile for this program. A typical modulefile for a program on the O2 application stack will look something like this (using GCC 6.2.0 as an example), written in Lua:

If your module is for internal consumption only (which will generally be the case), a number of the above fields are optional. For example, you might not need the metadata fields specified by whatis, nor should you need the add_property line (which handles exclusivity management - you cannot have two modules in the same family loaded at the same time). Depending on how complex your local application set is, you might not need the last section either, which exposes a new directory of modulefiles if this module is loaded; this is how Lmod restricts your view of the module tree based on your environment.

For simple programs that are primarily executables, you should only need one variable prepended, the PATH variable. If you need to access libraries as well, you will likely need to also prepend to LD_LIBRARY_PATH. MANPATH is for if you'd like to view the documentation associated with your program (which might not always be available). Where these paths point to is reliant on where your programs are installed.

Notably, in addition to using Lmod modulefile functions, you can also inject logic by writing instructions directly in Lua if necessary (as seen in the the last section of the modulefile above).

A page of the available functions can be found on the Lmod modulefile documentation page.

Organizing Modulefiles - Best Practices

If you are setting up modulefiles for more than one person (e.g. your lab or some colleagues), it is recommended that you choose a directory that is accessible to everyone, perhaps one that is closer to the top level of a group directory, for example. You can simply name it modulefiles or modules or something - just make it abundantly clear what the directory is for. You'll notice that when you type module avail, the modules appear to be categorized by path (which translates to the above categorization). Once you have added your modulefile directory, that will be the path where everything is located.

On O2, modulefiles are structured such that within the modulefile directory, the hierarchy is

So if you have version 5 of program A, the hierarchy would be something like

where 5.lua contains the contents of the modulefile you have written (in lua). You can have multiple subdirectories, and all of them will be mapped.

Once you have set up your modulefile directory tree, add that directory to your Lmod cache with the following command:

To remove a directory (e.g. if you move the directory to somewhere else), simply type

These commands will modify your MODULEPATH environment variable. DO NOT MODIFY THIS VARIABLE MANUALLY; only use module use and module unuse to interact with this environment variable.