|
On O2, we encourage cluster users to install the packages and software they need. One method to install packages and manage environments is to use conda, which is available through the conda2/4.2.13
moduleour miniconda
modules. Conda manages dependencies by default when you install packages, which can make it easier to install software. Packages that can be installed with conda include Python modules, libraries, or executable programs. Conda includes its own version of Python (2.7. 12), though you can explicitly request to use Python 3 if you would prefer.
Commonly used commands, examples
Command | Meaning |
---|---|
|
| shows the versions of conda installed on O2 |
|
| loads an individual conda module (replace version with an actual version) |
| see available conda environments |
| create conda environment named test_env (name the environment whatever you'd like) |
| create conda environment, and install some packages (bwa, bowtie, and star) on the fly |
| "activate" a conda environment |
located at | |
| exit current conda environment |
| delete a conda environment named test_env (may also require full path if installed to nonstandard location) |
| search for a package (replace numpy with the package of your choice) |
| install a package, and must be within a conda environment or this command will fail. (replace numpy with the package of your choice) |
Setup
To install packages on O2 using conda, you must first create a conda environment. Environments are simply directories in ~/.conda/envs/
that contain packages you installed. You "source" an environment to use those packages, and can "deactivate" to exit the environment. You can have multiple environments, and can switch between them.
...
Code Block |
---|
mfk8@compute-a-01-01:~$ module load conda2miniconda3/423.21.130 |
Then the conda
command will be available:
Code Block |
---|
mfk8@compute-a-01-01:~$ which conda /n/app/conda2/miniconda3/23.1.0/bin/conda |
Running conda info
will return information about the current conda installation:
...
You can see available conda environments with conda info --envs.
If you have not created any conda environments yet, the only listing you will see is the root environment in /n/app/conda2miniconda3
. Cluster users do not have access to alter this.
...
Code Block |
---|
(test_env) mfk8@compute-a-01-01:~$ which python /n/app/conda2/miniconda3/23.1.0/bin/python |
The default If you want to use a specific version of Python that's available through the conda module is Python 2.7.12:
Code Block |
---|
(test_env) mfk8@compute-a-01-01:~$ python --version
Python 2.7.12 :: Continuum Analytics, Inc. |
If you want to use conda and Python 3with conda (strongly recommended), you can create a conda environment and install Python 3 to it. For example to create an environment using Python 3.6.5:
...
Code Block |
---|
mfk8@login01:~$ srun --pty -p interactive -t 0-2 bash mfk8@compute-a-01-01:~$ module load conda2miniconda3/423.21.130 mfk8@compute-a-01-01:~$ module list Currently Loaded Modules: 1) conda2miniconda3/423.21.130 (E) Where: E: Experimental mfk8@compute-a-01-01:~$ conda create -n my_env # truncated mfk8@compute-a-01-01:~$ source activate my_env # install example python package, scipy, which is available through conda: (my_env) mfk8@compute-a-01-01:~$ conda install scipy # truncated # see list of packages available in this conda environment: (my_env) mfk8@compute-a-01-01:~$ conda list # truncated # will report scipy in the list # test importing scipy in python to verify it is installed correctly (my_env) mfk8@compute-a-01-01:~$ python -c "import scipy" (my_env) mfk8@compute-a-01-01:~$ # exit environment (my_env) mfk8@compute-a-01-01:~$ source deactivate mfk8@compute-a-01-01:~$ |
...
The centralized conda installation, available through the conda2/4.2.13
modulethe conda modules, includes several channels that we support. Channels are repositories where conda looks for packages. This is done with a centralized .condarc file that contains:
...
If you wish, you can still maintain your own ~/.condarc
file, but we may be unable to assist when using unsupported channels.
Installing difficult packages
We have created a page to document any particularly difficult packages as observed from interacting with the user community, located at Installing Difficult Conda Packages on O2 . We will keep this page updated as necessary as we encounter more.