Is there a way to deal with initializing conda correctly across multiple shells?

[rdauria]

As this is perhaps a feature request I will chance posting it here and not on https://sourceforge.net/projects/modules/lists/modules-interest.

When creating a module file for anaconda invariably we need to request users to also source a conda shell script that varies according to the user shell, i.e.:

# SH bases shells:
module load anconda
source $CONDA_DIR/etc/profile.d/conda.sh

or:

# CSH based shells:
module load anaconda
source $CONDA_DIR/etc/profile.d/conda.csh

since the file to be sourced are different enough using either sh-to-mod or source-sh would require that the modulefile first check on the user SHELL and branch out accordingly. Is this possible?

Thanks!

[xdelaruelle]

module-info shelltype helps to know the current shell type that module command has been initialized to.

So you may write your modulefile the following way:

switch -- [module-info shelltype] {
  sh {
    source-sh sh $CONDA_DIR/etc/profile.d/conda.sh
  }
  csh {
    source-sh csh $CONDA_DIR/etc/profile.d/conda.csh
  }
  default {
    error {Shell not supported}
  }
}

[rdauria]

Thanks so much for the suggestion. I wanted to point out a strange behavior, when I implemented the module with:

csh {
   source-sh csh $CONDA_DIR/etc/profile.d/conda.csh
}

I would invariably get:

$ conda env list
Badly placed ()'s.

the infuriating thing is that I could see that the conda alias was set correctly by the modulefile, namely:

conda	(source /u/local/apps/anaconda3/2020.11/etc/profile.d/conda.csh)

which is exactly what sourcing the file:

/u/local/apps/anaconda3/2020.11/etc/profile.d/conda.csh

sets it to (but sourcing the file outside of a modulefile does not give the Badly placed ()'s. error).

In the end the only way I could find to make the modulefile work (and the issue of the Badly placed ()'s. go away) was to update the portion of the csh modulefile as follows:

csh {
    #source-sh csh $base_dir/etc/profile.d/conda.csh
    set-alias       conda {source /u/local/apps/anaconda3/2020.11/etc/profile.d/conda.csh}
  }

Any idea why this may be so? In any event I decided to post the trouble and its solution should other people experience the same issue.

[rdauria]

One more question, would you know how can I go about debugging this error which appears upon logging in from a zsh shell?

_module_raw:122: parse error near `()'

The error appears only at log-in and it prevents certain default modules from being loaded. Once the login is finished, however, the user can load the modules with no further error.

We are running env mod version 4.7.0 and gcc 4.8.5 on a CentOS 7 cluster.

Also do you know if there is a way to have a zsh not to give:

zsh compinit: insecure directories and files, run compaudit for list.
Ignore insecure directories and files and continue [y] or abort compinit [n]?

compaudit gives:

There are insecure directories and files:
/u/local/Modules/4.7.0/gcc-4.8.5/init/zsh-functions
/u/local/Modules/4.7.0/gcc-4.8.5/init
/u/local/Modules/4.7.0/gcc-4.8.5/init/zsh-functions/_module

but on a close inspection I am not sure what is it that zsh complaints of being insecure as:

% ls -tlr /u/local/Modules/4.7.0/gcc-4.8.5/init
total 108
-rw-r--r--. 1 instuser instgroup  2734 Mar 19  2021 sh
-rw-r--r--. 1 instuser instgroup  2513 Mar 19  2021 ksh
-rw-r--r--. 1 instuser instgroup  2884 Mar 19  2021 bash
-rw-r--r--. 1 instuser instgroup  2754 Mar 19  2021 zsh
-rw-r--r--. 1 instuser instgroup   424 Mar 19  2021 profile.sh
-rw-r--r--. 1 instuser instgroup 10135 Mar 19  2021 bash_completion
-rw-r--r--. 1 instuser instgroup  1418 Mar 19  2021 csh
-rw-r--r--. 1 instuser instgroup  1419 Mar 19  2021 tcsh
-rw-r--r--. 1 instuser instgroup  5981 Mar 19  2021 tcsh_completion
-rw-r--r--. 1 instuser instgroup   133 Mar 19  2021 profile.csh
-rw-r--r--. 1 instuser instgroup   725 Mar 19  2021 perl.pm
-rw-r--r--. 1 instuser instgroup   981 Mar 19  2021 python.py
-rw-r--r--. 1 instuser instgroup   836 Mar 19  2021 ruby.rb
-rw-r--r--. 1 instuser instgroup  3258 Mar 19  2021 lisp
-rw-r--r--. 1 instuser instgroup  1070 Mar 19  2021 tcl
-rw-r--r--. 1 instuser instgroup  1771 Mar 19  2021 fish
-rw-r--r--. 1 instuser instgroup  1193 Mar 19  2021 cmake
-rw-r--r--. 1 instuser instgroup   960 Mar 19  2021 r.R
-rw-r--r--. 1 instuser instgroup 10217 Mar 19  2021 fish_completion
drwxr-xr-x. 2 instuser instgroup  4096 Mar 19  2021 zsh-functions
drwxr-xr-x. 2 instuser instgroup  4096 Mar 19  2021 ksh-functions
-rw-r--r--. 1 instuser instgroup   409 Mar 19  2021 modulerc
lrwxrwxrwx. 1 instuser instgroup    10 Mar 23  2021 modules.sh -> profile.sh
lrwxrwxrwx. 1 instuser instgroup    11 Mar 23  2021 modules.csh -> profile.csh

and:

% ls -tlr /u/local/Modules/4.7.0/gcc-4.8.5/init/zsh-functions 
total 16
-rw-r--r--. 1 instuser instgroup 13010 Mar 19  2021 _module

Do you happen to have any idea?

Thanks :)

[xdelaruelle]

One more question, would you know how can I go about debugging this error which appears upon logging in from a zsh shell?

_module_raw:122: parse error near `()'

The error appears only at log-in and it prevents certain default modules from being loaded. Once the login is finished, however, the user can load the modules with no further error.

I suggest you add a set +x or set +xv command in the zsh initialization file of Modules to get debugging information.

[xdelaruelle]

Also do you know if there is a way to have a zsh not to give:

zsh compinit: insecure directories and files, run compaudit for list.
Ignore insecure directories and files and continue [y] or abort compinit [n]?

It seems like compinit expect files to be owned by root: https://stackoverflow.com/questions/13762280/zsh-compinit-insecure-directories

[rdauria]

Thanks a million for the suggestions!!! Indeed the zsh shell initialization gets stuck as it tries to load a modulefile for the intel compiler which is basically doing:

source-sh bash $base_dir/compilers_and_libraries_2020/linux/bin/compilervars.sh intel64

if I write the modulefile without sourcing the intel initialization script (but setting all the variables "by hand") the problem goes away. I think the issue is with some of the bash functions that the intel script sets up (I could send you more info in case you are interested).

The interesting fact is that the error only occurs when /bin/zsh is run (we place a file in /etc/profile.d that will load a set of default modulefiles for all of our users), if a user with a zsh shell loads the intel module after logging in then there is no error. At this point I think I will just rewrite the intel modulefile without using the source-sh functionality.

[xdelaruelle]

Thanks a million for the suggestions!!! Indeed the zsh shell initialization gets stuck as it tries to load a modulefile for the intel compiler which is basically doing:

source-sh bash $base_dir/compilers_and_libraries_2020/linux/bin/compilervars.sh intel64

I suggest you adapt the code to use the current running shell rather bash in order to use zsh to source-sh the intel script:

source-sh zsh $base_dir/compilers_and_libraries_2020/linux/bin/compilervars.sh intel64

[xdelaruelle]

Issue found on csh shell with Conda should be fixed. Other fixes have been added to improve support of shell function on sh-kind shells.

It would be great if you could test this new Modules version (current master git branch) and do not hesitate to open new issues if you still encounter errors.