Installing Difficult Conda Packages on O2
- Alex Truong
Please ensure you are following the instructions in Conda on O2 when attempting to install your own Conda packages. Below is a list of known packages and situations the O2 user community has troubleshooted with Research Computing and their resolutions (this list will be updated as new instances are encountered).
GCC within Conda
When trying to troubleshoot building something within your conda environment, you may notice that the output of which gcc will still point to /usr/bin/gcc. You may attempt to remedy this by trying something like conda install gcc into your environment, only to discover that it either failed, or you are still unable to find your freshly “installed” GCC version. We illustrate an example solution here.
Firstly, the packages to conda install are actually gcc_linux-64 and gxx_linux-64. You will (probably) need both packages. You can specify versions if you’d like, such as gcc_linux-64=9; make sure that if you do so, you specify the same version for both. Once these install successfully, your environment’s bin directory will look something like this:
lrwxrwxrwx 1 *** *** 32 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-addr2line -> x86_64-conda-linux-gnu-addr2line
lrwxrwxrwx 1 *** *** 25 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-ar -> x86_64-conda-linux-gnu-ar
lrwxrwxrwx 1 *** *** 25 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-as -> x86_64-conda-linux-gnu-as
lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-c++ -> x86_64-conda-linux-gnu-c++
lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-c89 -> x86_64-conda-linux-gnu-c89
lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-c99 -> x86_64-conda-linux-gnu-c99
lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-cc -> x86_64-conda-linux-gnu-gcc
lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-c++filt -> x86_64-conda-linux-gnu-c++filt
lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-cpp -> x86_64-conda-linux-gnu-cpp
lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-dwp -> x86_64-conda-linux-gnu-dwp
lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-elfedit -> x86_64-conda-linux-gnu-elfedit
lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-g++ -> x86_64-conda-linux-gnu-g++
lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcc -> x86_64-conda-linux-gnu-gcc
lrwxrwxrwx 1 *** *** 29 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcc-ar -> x86_64-conda-linux-gnu-gcc-ar
lrwxrwxrwx 1 *** *** 29 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcc-nm -> x86_64-conda-linux-gnu-gcc-nm
lrwxrwxrwx 1 *** *** 33 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcc-ranlib -> x86_64-conda-linux-gnu-gcc-ranlib
lrwxrwxrwx 1 *** *** 27 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcov -> x86_64-conda-linux-gnu-gcov
lrwxrwxrwx 1 *** *** 32 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcov-dump -> x86_64-conda-linux-gnu-gcov-dump
lrwxrwxrwx 1 *** *** 32 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcov-tool -> x86_64-conda-linux-gnu-gcov-tool
lrwxrwxrwx 1 *** *** 28 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gprof -> x86_64-conda-linux-gnu-gprof
lrwxrwxrwx 1 *** *** 25 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-ld -> x86_64-conda-linux-gnu-ld
lrwxrwxrwx 1 *** *** 29 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-ld.bfd -> x86_64-conda-linux-gnu-ld.bfd
lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-ld.gold -> x86_64-conda-linux-gnu-ld.gold
lrwxrwxrwx 1 *** *** 25 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-nm -> x86_64-conda-linux-gnu-nm
lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-objcopy -> x86_64-conda-linux-gnu-objcopy
lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-objdump -> x86_64-conda-linux-gnu-objdump
lrwxrwxrwx 1 *** *** 29 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-ranlib -> x86_64-conda-linux-gnu-ranlib
lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-readelf -> x86_64-conda-linux-gnu-readelf
lrwxrwxrwx 1 *** *** 27 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-size -> x86_64-conda-linux-gnu-size
lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-strings -> x86_64-conda-linux-gnu-strings
lrwxrwxrwx 1 *** *** 28 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-strip -> x86_64-conda-linux-gnu-strip
-rwxrwxr-x 1 *** *** 1285536 Nov 20 2022 x86_64-conda-linux-gnu-addr2line
-rwxrwxr-x 1 *** *** 1314056 Nov 20 2022 x86_64-conda-linux-gnu-ar
-rwxrwxr-x 1 *** *** 2057264 Nov 20 2022 x86_64-conda-linux-gnu-as
-rwxrwxr-x 1 *** *** 1092024 Oct 23 2022 x86_64-conda-linux-gnu-c++
-rwxrwxr-x 3 *** *** 225 Oct 23 2022 x86_64-conda-linux-gnu-c89
-rwxrwxr-x 3 *** *** 216 Oct 23 2022 x86_64-conda-linux-gnu-c99
lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda-linux-gnu-cc -> x86_64-conda-linux-gnu-gcc
-rwxrwxr-x 1 *** *** 1280896 Nov 20 2022 x86_64-conda-linux-gnu-c++filt
-rwxrwxr-x 3 *** *** 1092024 Oct 23 2022 x86_64-conda-linux-gnu-cpp
-rwxrwxr-x 1 *** *** 3252560 Nov 20 2022 x86_64-conda-linux-gnu-dwp
-rwxrwxr-x 1 *** *** 43992 Nov 20 2022 x86_64-conda-linux-gnu-elfedit
-rwxrwxr-x 1 *** *** 1092024 Oct 23 2022 x86_64-conda-linux-gnu-g++
-rwxrwxr-x 3 *** *** 1087928 Oct 23 2022 x86_64-conda-linux-gnu-gcc
-rwxrwxr-x 3 *** *** 35296 Oct 23 2022 x86_64-conda-linux-gnu-gcc-ar
-rwxrwxr-x 3 *** *** 35296 Oct 23 2022 x86_64-conda-linux-gnu-gcc-nm
-rwxrwxr-x 3 *** *** 35296 Oct 23 2022 x86_64-conda-linux-gnu-gcc-ranlib
-rwxrwxr-x 3 *** *** 679544 Oct 23 2022 x86_64-conda-linux-gnu-gcov
-rwxrwxr-x 3 *** *** 495120 Oct 23 2022 x86_64-conda-linux-gnu-gcov-dump
-rwxrwxr-x 3 *** *** 511568 Oct 23 2022 x86_64-conda-linux-gnu-gcov-tool
-rwxrwxr-x 1 *** *** 1348096 Nov 20 2022 x86_64-conda-linux-gnu-gprof
-rwxrwxr-x 1 *** *** 2413584 Nov 20 2022 x86_64-conda-linux-gnu-ld
-rwxrwxr-x 1 *** *** 2413584 Nov 20 2022 x86_64-conda-linux-gnu-ld.bfd
-rwxrwxr-x 1 *** *** 5784976 Nov 20 2022 x86_64-conda-linux-gnu-ld.gold
-rwxrwxr-x 1 *** *** 1298752 Nov 20 2022 x86_64-conda-linux-gnu-nm
-rwxrwxr-x 1 *** *** 1428704 Nov 20 2022 x86_64-conda-linux-gnu-objcopy
-rwxrwxr-x 1 *** *** 2713760 Nov 20 2022 x86_64-conda-linux-gnu-objdump
-rwxrwxr-x 1 *** *** 1314088 Nov 20 2022 x86_64-conda-linux-gnu-ranlib
-rwxrwxr-x 1 *** *** 1023128 Nov 20 2022 x86_64-conda-linux-gnu-readelf
-rwxrwxr-x 1 *** *** 1285280 Nov 20 2022 x86_64-conda-linux-gnu-size
-rwxrwxr-x 1 *** *** 1289536 Nov 20 2022 x86_64-conda-linux-gnu-strings
-rwxrwxr-x 1 *** *** 1428704 Nov 20 2022 x86_64-conda-linux-gnu-stripNote that these executables are long-form (and there are already a number of softlinks that have been created); there is no actual executable that is just named gcc. What you will need to do next is create (at least) two more softlinks (replace path/to/env with the actual path to your conda environment - by default, it will be in $HOME/.conda/envs, but if you specified --prefix or -p when creating the environment, it may be somewhere else):
$ cd path/to/env/bin
$ ln -s x86_64-conda-linux-gnu-gcc gcc
$ ln -s x86_64-conda-linux-gnu-g++ g++You should now see the following additional entries in the bin directory:
lrwxrwxrwx 1 *** *** 26 Jul 12 15:05 g++ -> x86_64-conda-linux-gnu-g++
lrwxrwxrwx 1 *** *** 26 Jul 12 15:04 gcc -> x86_64-conda-linux-gnu-gccIn most cases, this should get you far enough. If you encounter build errors that are looking for something like gcc-ar, you will need to set up a similar softlink for that as well. In any case, once you have the above softlinks (at a minimum) created, your conda environment should be able to reliably use your installed gcc installation to build packages instead of inadvertently using the one located at /usr/bin.