Please ensure you are following the instructions in Conda on O2 when attempting to install your own Conda packages. Below is a list of known packages and situations the O2 user community has troubleshooted with Research Computing and their solutions (this list will be updated as new instances are encountered).
GCC within Conda
When trying to troubleshoot building something within your conda environment, you may notice that the output of which gcc will still point to /usr/bin/gcc. You may attempt to remedy this by trying something like conda install gcc into your environment, only to discover that it either failed, or you are still unable to find your freshly “installed” GCC version. We illustrate an example solution here.
Firstly, the packages to conda install are actually gcc_linux-64 and gxx_linux-64. You will (probably) need both packages. You can specify versions if you’d like, such as gcc_linux-64=9; make sure that if you do so, you specify the same version for both. Once these install successfully, your environment’s bin directory will look something like this:
lrwxrwxrwx 1 *** *** 32 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-addr2line -> x86_64-conda-linux-gnu-addr2line lrwxrwxrwx 1 *** *** 25 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-ar -> x86_64-conda-linux-gnu-ar lrwxrwxrwx 1 *** *** 25 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-as -> x86_64-conda-linux-gnu-as lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-c++ -> x86_64-conda-linux-gnu-c++ lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-c89 -> x86_64-conda-linux-gnu-c89 lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-c99 -> x86_64-conda-linux-gnu-c99 lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-cc -> x86_64-conda-linux-gnu-gcc lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-c++filt -> x86_64-conda-linux-gnu-c++filt lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-cpp -> x86_64-conda-linux-gnu-cpp lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-dwp -> x86_64-conda-linux-gnu-dwp lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-elfedit -> x86_64-conda-linux-gnu-elfedit lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-g++ -> x86_64-conda-linux-gnu-g++ lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcc -> x86_64-conda-linux-gnu-gcc lrwxrwxrwx 1 *** *** 29 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcc-ar -> x86_64-conda-linux-gnu-gcc-ar lrwxrwxrwx 1 *** *** 29 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcc-nm -> x86_64-conda-linux-gnu-gcc-nm lrwxrwxrwx 1 *** *** 33 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcc-ranlib -> x86_64-conda-linux-gnu-gcc-ranlib lrwxrwxrwx 1 *** *** 27 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcov -> x86_64-conda-linux-gnu-gcov lrwxrwxrwx 1 *** *** 32 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcov-dump -> x86_64-conda-linux-gnu-gcov-dump lrwxrwxrwx 1 *** *** 32 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gcov-tool -> x86_64-conda-linux-gnu-gcov-tool lrwxrwxrwx 1 *** *** 28 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-gprof -> x86_64-conda-linux-gnu-gprof lrwxrwxrwx 1 *** *** 25 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-ld -> x86_64-conda-linux-gnu-ld lrwxrwxrwx 1 *** *** 29 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-ld.bfd -> x86_64-conda-linux-gnu-ld.bfd lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-ld.gold -> x86_64-conda-linux-gnu-ld.gold lrwxrwxrwx 1 *** *** 25 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-nm -> x86_64-conda-linux-gnu-nm lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-objcopy -> x86_64-conda-linux-gnu-objcopy lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-objdump -> x86_64-conda-linux-gnu-objdump lrwxrwxrwx 1 *** *** 29 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-ranlib -> x86_64-conda-linux-gnu-ranlib lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-readelf -> x86_64-conda-linux-gnu-readelf lrwxrwxrwx 1 *** *** 27 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-size -> x86_64-conda-linux-gnu-size lrwxrwxrwx 1 *** *** 30 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-strings -> x86_64-conda-linux-gnu-strings lrwxrwxrwx 1 *** *** 28 Jul 12 15:01 x86_64-conda_cos6-linux-gnu-strip -> x86_64-conda-linux-gnu-strip -rwxrwxr-x 1 *** *** 1285536 Nov 20 2022 x86_64-conda-linux-gnu-addr2line -rwxrwxr-x 1 *** *** 1314056 Nov 20 2022 x86_64-conda-linux-gnu-ar -rwxrwxr-x 1 *** *** 2057264 Nov 20 2022 x86_64-conda-linux-gnu-as -rwxrwxr-x 1 *** *** 1092024 Oct 23 2022 x86_64-conda-linux-gnu-c++ -rwxrwxr-x 3 *** *** 225 Oct 23 2022 x86_64-conda-linux-gnu-c89 -rwxrwxr-x 3 *** *** 216 Oct 23 2022 x86_64-conda-linux-gnu-c99 lrwxrwxrwx 1 *** *** 26 Jul 12 15:01 x86_64-conda-linux-gnu-cc -> x86_64-conda-linux-gnu-gcc -rwxrwxr-x 1 *** *** 1280896 Nov 20 2022 x86_64-conda-linux-gnu-c++filt -rwxrwxr-x 3 *** *** 1092024 Oct 23 2022 x86_64-conda-linux-gnu-cpp -rwxrwxr-x 1 *** *** 3252560 Nov 20 2022 x86_64-conda-linux-gnu-dwp -rwxrwxr-x 1 *** *** 43992 Nov 20 2022 x86_64-conda-linux-gnu-elfedit -rwxrwxr-x 1 *** *** 1092024 Oct 23 2022 x86_64-conda-linux-gnu-g++ -rwxrwxr-x 3 *** *** 1087928 Oct 23 2022 x86_64-conda-linux-gnu-gcc -rwxrwxr-x 3 *** *** 35296 Oct 23 2022 x86_64-conda-linux-gnu-gcc-ar -rwxrwxr-x 3 *** *** 35296 Oct 23 2022 x86_64-conda-linux-gnu-gcc-nm -rwxrwxr-x 3 *** *** 35296 Oct 23 2022 x86_64-conda-linux-gnu-gcc-ranlib -rwxrwxr-x 3 *** *** 679544 Oct 23 2022 x86_64-conda-linux-gnu-gcov -rwxrwxr-x 3 *** *** 495120 Oct 23 2022 x86_64-conda-linux-gnu-gcov-dump -rwxrwxr-x 3 *** *** 511568 Oct 23 2022 x86_64-conda-linux-gnu-gcov-tool -rwxrwxr-x 1 *** *** 1348096 Nov 20 2022 x86_64-conda-linux-gnu-gprof -rwxrwxr-x 1 *** *** 2413584 Nov 20 2022 x86_64-conda-linux-gnu-ld -rwxrwxr-x 1 *** *** 2413584 Nov 20 2022 x86_64-conda-linux-gnu-ld.bfd -rwxrwxr-x 1 *** *** 5784976 Nov 20 2022 x86_64-conda-linux-gnu-ld.gold -rwxrwxr-x 1 *** *** 1298752 Nov 20 2022 x86_64-conda-linux-gnu-nm -rwxrwxr-x 1 *** *** 1428704 Nov 20 2022 x86_64-conda-linux-gnu-objcopy -rwxrwxr-x 1 *** *** 2713760 Nov 20 2022 x86_64-conda-linux-gnu-objdump -rwxrwxr-x 1 *** *** 1314088 Nov 20 2022 x86_64-conda-linux-gnu-ranlib -rwxrwxr-x 1 *** *** 1023128 Nov 20 2022 x86_64-conda-linux-gnu-readelf -rwxrwxr-x 1 *** *** 1285280 Nov 20 2022 x86_64-conda-linux-gnu-size -rwxrwxr-x 1 *** *** 1289536 Nov 20 2022 x86_64-conda-linux-gnu-strings -rwxrwxr-x 1 *** *** 1428704 Nov 20 2022 x86_64-conda-linux-gnu-strip
Note that these executables are long-form (and there are already a number of softlinks that have been created); there is no actual executable that is just named gcc. What you will need to do next is create (at least) two more softlinks (replace path/to/env with the actual path to your conda environment - by default, it will be in $HOME/.conda/envs, but if you specified --prefix or -p when creating the environment, it may be somewhere else):
$ cd path/to/env/bin $ ln -s x86_64-conda-linux-gnu-gcc gcc $ ln -s x86_64-conda-linux-gnu-g++ g++
In most cases, this should get you far enough. If you encounter build errors that are looking for something like gcc-ar, you will need to set up a similar softlink for that as well. In any case, once you have the above softlinks (at a minimum) created, your conda environment should be able to reliably use your installed gcc installation to build packages instead of inadvertently using the one located at /usr/bin.