sourmash not installing in binder: "No module named 'importlib.metadata'" error

[hehouts]

Hello!

I am trying to set up a binder using sourmash, and I am getting the following error:

Traceback (most recent call last):
  File "/srv/conda/envs/notebook/bin/sourmash", line 7, in <module>
    from sourmash.__main__ import main
  File "/srv/conda/envs/notebook/lib/python3.7/site-packages/sourmash/__init__.py", line 113, in <module>
    from . import lca
  File "/srv/conda/envs/notebook/lib/python3.7/site-packages/sourmash/lca/__init__.py", line 8, in <module>
    from .command_index import index
  File "/srv/conda/envs/notebook/lib/python3.7/site-packages/sourmash/lca/command_index.py", line 9, in <module>
    from sourmash import sourmash_args
  File "/srv/conda/envs/notebook/lib/python3.7/site-packages/sourmash/sourmash_args.py", line 46, in <module>
    import screed
  File "/srv/conda/envs/notebook/lib/python3.7/site-packages/screed/__init__.py", line 39, in <module>
    from importlib.metadata import version, PackageNotFoundError
ModuleNotFoundError: No module named 'importlib.metadata'

if I run pip show sourmash in the binder terminal, I get

Name: sourmash
Version: 4.3.0
Summary: tools for comparing DNA sequences with MinHash sketches
Home-page: https://github.com/dib-lab/sourmash
Author: Luiz Irber, Harriet Alexander, Olga Botvinnik, Phillip Brooks, Laurent Gautier, Tim Head, Lisa K. Johnson, Fabian Klötzl, Katrin Leinweber, Ivan Ogasawara, N. Tessa Pierce, Taylor Reiter, Andreas Sjödin, Connor T. Skennerton, Daniel Standage, S. Joshua Swamidass, Connor Tiffany, Erik Young, C. Titus Brown
Author-email: titus@idyll.org
License: BSD 3-clause
Location: /srv/conda/envs/notebook/lib/python3.7/site-packages
Requires: cachetools, cffi, deprecation, matplotlib, numpy, scipy, screed
Required-by:

cc: @luizirber , @ctb

[hehouts]

This is the binder repo, I will not make any changes until its safe :D
https://github.com/hehouts/lab14_binder

this is the binder launch link Ive been using: https://mybinder.org/v2/gh/hehouts/lab14_binder.git/HEAD/?urlpath=rstudio

[ctb]

some oddities 😁

/srv/conda/envs/notebook/bin/ is put in the PATH regardless of what conda environment you're in.

`/srv/conda/envs/notebook/bin/python -V' is Python 3.7.12

default Python in base environment is /srv/conda/bin/python, /srv/conda/bin/python -V shows Python 3.10.9.

yay.

[sapuizait]

HI there - i have the same error when trying to install in on a SLURM cluster using conda

(sourmash) [XXX@hpclogin01 ~]$ pip show sourmash
Name: sourmash
Version: 3.3.0
Summary: tools for comparing DNA sequences with MinHash sketches
Home-page: https://github.com/dib-lab/sourmash
Author: C. Titus Brown
Author-email: titus@idyll.org
License: BSD 3-clause
Location: /home/XXX/miniconda3/envs/sourmash/lib/python3.7/site-packages
Requires: cffi, deprecation, matplotlib, numpy, scipy, screed
Required-by

so... is there a solution?

[ctb]

yep - looks like the solution is to make sure that python >= 3.8. As you can see @sapuizait yours is python 3.7 -

Location: /home/XXX/miniconda3/envs/sourmash/lib/python3.7/site-packages

let us know if that works!

[sapuizait]

yay! seems to be working!

[luizirber]

Food for thought: should we change the conda install instructions from
conda install -c conda-forge -c bioconda sourmash
to
conda install -c conda-forge sourmash-minimal?

We sort of kept the bioconda package because it generates biocontainers, but otherwise it ends up causing compatibility problems if bioconda is not set up in the recommended way...

(or, more extremely, we could rename sourmash-minimal to sourmash in conda-forge and explicitly depend on conda-forge::sourmash in the bioconda recipe, but I think that might be too much confusion...)

[fabianegli]

This is a symptom of not specifying the version of sourmash you want to install. Newer versions are not compatible with Python 3.7 which means the installer looks for previous versions which are compatible and they find sourmash version 3.3.0. And this release is apparently buggy.

A solution would be to specify a recent version of sourmash in the docs. e.g.

conda install -c conda-forge -c bioconda sourmash=4.7.0

This will tell the user that is not possible to resolve the dependencies if it is run in a conda environment using Python versions < 3.8

[ctb]

This is a symptom of not specifying the version of sourmash you want to install. Newer versions are not compatible with Python 3.7 which means the installer looks for previous versions which are compatible and they find sourmash version 3.3.0. And this release is apparently buggy.

right, two issues here -

• why is sourmash failing with that error message...!
• why are binders defaulting to python 3.7!

A solution would be to specify a recent version of sourmash in the docs. e.g.

conda install -c conda-forge -c bioconda sourmash=4.7.0

This will tell the user that is not possible to resolve the dependencies if it is run in a conda environment using Python versions < 3.8

yep. I'd like to avoid having sourmash version numbers hardcoded in the docs too much, because we always forget to update them, but it might also be something we can put into the release procedure.

[fabianegli]

I am not sure if this would have the desired effect, but you could try and release a new build restricintg the Python versions for older versions of sourmash. See: https://conda-forge.org/docs/maintainer/updating_pkgs.html#updating-recipes

[ctb]

our default binder/environment.yml in the sourmash repo installs python 3.10 and sourmash 4.8.2 without any version spec, so that's good, I think. not sure if binder updated the default python version or what.

(you can run binder from here, for example)