Make StochKv3.mod NEURON 9.0a compatible

[darshanmandge]

The PR makes StochKv3.mod with NEURON 9.0a. The file works with NEURON 8.2.2 and should also work with other version >8.* and the future NEURON 9 release as well. The file comes from BBP's internal repository.

According to @pramodk :
"The changes to mod files like StochKv3 are all technical. There is no change in intrinsic behaviour / results "

[anilbey]

Thanks looks good to me. One point worthy mentioning is that in the requirements.txt we had set the following restriction on the NEURON version for scientific reproducibility of the results.

NEURON>=7.8.0,<8.0.0

Maybe it can be added to the README.md that for the exact reproducibility of paper contents we recommend the aforementioned NEURON version range and the users using the versions outside that range may encounter variety in the results.

[pramodk]

"The changes to mod files like StochKv3 are all technical. There is no change in intrinsic behaviour / results "

Just to remain "true" to the above quote: this change to value variable is the one something that we were a bit unsure about. In the JIRA linked issue we explained why this was necessary and how this was the issue in load-balancing / ExperimentalMechComplex usage with NEURON.

We have ran simulations and results are same and now this change is in production. But if you really want to keep exact old code, you can set the value back to 0.5.

[pramodk]

Just to add some more information if someone will revisit this in the future:

FUNCTION urand() {VERBATIM
    double value;
    if( usingR123 ) {
        value = nrnran123_dblpick((nrnran123_State*)_p_rng);
    } else if (_p_rng) {
#ifndef CORENEURON_BUILD
        value = nrn_random_pick(RANDCAST _p_rng);
#endif
    } else {
        
        value = 0.5;
    }
    _lurand = value;
ENDVERBATIM
}

In the above urand() function we are picking a random number. In all practical usage, we would have an RNG stream setup either using Random123 or MCellRan4. The last else block returns a value 0.5 (or 0) when a user hasn't set up the RNG variable i.e. StochKV is not properly instantiated. This won't be the case if someone is really using StochKV.

So I would say let's ignore my initial comment about this change. Using the new MOD file is perfectly OK.

[darshanmandge]

Thanks for the detailed explanation @pramodk!

[anilbey]

Update: after talking to @pramodk I tried running the tests with NEURON 8.0 with the USE_LEGACY flag BlueBrain/EModelRunner#81. The tests pass. We can remove the NEURON<8.0.0 restriction. I will try to do that in another PR.

@darshanmandge have you tested this PR with NEURON 9.0? I am having a compilation error when I try it with NEURON 9.0.

[darshanmandge]

The StochKv3 file compiled successfully in a new python 3.11.6 virtual environment after installing neuron using pip install NEURON==9.0a0 and running nrnivmodl.
If you are on BB5 using NEURON via module load unstable neuron, you may also have to do module load intel-oneapi-compilers

[darshanmandge]

I tried on linux desktop with python3.11.8 virtual environment and with pip install NEURON==9.0a0. Got this errors:

os.environ["NMODL_PYLIB"] = find_libpython()
~~~~~~~~~~^^^^^^^^^^^^^^^

File "", line 684, in setitem
File "", line 758, in encode
TypeError: str expected, not NoneType

I think we can wait to merge this till the final NEURON 9 release to solve these issues.

[anilbey]

Update: after talking to @pramodk I tried running the tests with NEURON 8.0 with the USE_LEGACY flag BlueBrain/EModelRunner#81. The tests pass.

This statement is not completely true. The optimisation tests do pass but the model-management tests do not pass with NEURON 8.2.4
Source: #36
Below is the diff of installed packages in a working and non-working environment.
https://www.diffchecker.com/pVuhV6xE/

[pramodk]

I tried on linux desktop with python3.11.8 virtual environment and with pip install NEURON==9.0a0. Got this errors:

os.environ["NMODL_PYLIB"] = find_libpython()
~~~~~~~~~~^^^^^^^^^^^^^^^
File "", line 684, in setitem
File "", line 758, in encode
TypeError: str expected, not NoneType

@darshanmandge : Just to understand the above error: how can I reproduce the above? Certainly 9.0a0 is old. We just want to make sure the latest version of NEURON+NMODL doesn't have the same issue. cc: @JCGoran

[darshanmandge]

On my ubuntu desktop with 22.04.4 LTS, I create a python3.11.8 virtual environment using
python3.11 -m venv nrn9
activated the environment using
source nrn9/bin/activate
The python version shows
Python 3.11.8 (main, Feb 25 2024, 16:39:33) [GCC 11.4.0]
I installed neuron 9.0a0 using
pip install NEURON==9.0a0

[JCGoran]

I tried on linux desktop with python3.11.8 virtual environment and with pip install NEURON==9.0a0. Got this errors:

os.environ["NMODL_PYLIB"] = find_libpython()
~~~~~~~~~~^^^^^^^^^^^^^^^

File "", line 684, in setitem File "", line 758, in encode TypeError: str expected, not NoneType

I think we can wait to merge this till the final NEURON 9 release to solve these issues.

Could be this line:
https://github.com/BlueBrain/nmodl/blob/fbf9de61d02b252a1c8da54ef3c7e98175920cdf/pywheel/shim/_binwrapper.py#L35

This indicates that find_libpython cannot find the Python shared library file (like /usr/local/lib/libpython3.11.so.1.0 or similar); NMODL should work just fine on Ubuntu, but NEURON may cause issues in a venv (neuronsimulator/nrn#1744).

Can you tell me which command you ran exactly that would reproduce this error?

[darshanmandge]

Sorry, I didn't specify the command. I compiled mod file using this command as the file was in the same folder:
nrnivmodl

[JCGoran]

Ahh, so it should probably be this line then:
https://github.com/neuronsimulator/nrn/blob/0bc1087e1a3ef996628e346dd1492a47ba113df4/share/lib/python/scripts/_binwrapper.py#L92
Still, it's puzzling that it doesn't work, I tried to reproduce in a Docker container on Debian bullseye with Python 3.11, and I don't encounter this issue; how did you install Python exactly (system package manager, conda, other)? Also, if you run python -m find_libpython in the venv, does it return any kind of output?

If not, could you try finding the Python library file somehow (in the container, the path is /usr/local/lib/libpython3.11.so.1.0; you can try using something like find /usr/lib /usr/local/lib -name 'libpython3.11*' if you installed it system-wide to try to find it), and running export NMODL_PYLIB=[LOCATION_OF_THAT_LIBRARY_FILE], followed by nrnivmodl to see if that works?

[darshanmandge]

python -m find_libpython gives no output in after activating my virtual environment. Without activating what gave command gave an error Command 'python' not found, did you mean: command 'python3' from deb python3 but this one
python3.11 -m find_libpython gave /usr/bin/python3.11: No module named find_libpython
find /usr/lib /usr/local/lib -name 'libpython3.11*' no output in my ubuntu home directory
python3.11 was most likely installed using this. Not sure correctly.
find . -name 'libpython3*' shows among many outputs minconda outputs

...
./miniconda3/lib/libpython3.11.so
./miniconda3/lib/libpython3.11.so.1.0

I had minconda installed but I don't use it create the venv. Also, just FYI have have the default python 3.10 installed.
However, the which python3.11 shows /usr/bin/python3.11 which is used for creating the virtual environment.

[anilbey]

The upper bound of NEURON version is relaxed from <8.0.0 to <=8.2.4. The precision of a test is updated accordingly (it shows small variations across minor NEURON versions).

[darshanmandge]

Thanks a lot @anilbey ! if it is fine, we can merge it.

[anilbey]

Alright. Let me just add notes to reproduce the precision differences in case it is of importance for the NEURON developers.

1. install NEURON 8.0.X
2. run make tests