Merge pull request #121 from metagenlab/nj/update_conda_envs

Update conda environments

CHANGELOG.md

@@ -4,6 +4,12 @@
 Versions are of the form MAJOR.MINOR.PATCH and this changelog tries to conform
 to [Common Changelog](https://common-changelog.org)
 
+## Unreleased
+
+### Fixed
+
+- Fix checkm conda environment. ([#121](https://github.com/metagenlab/zDB/pull/121)) (Niklaus Johner)
+
 
 ## 1.3.2 - 2024-07-30
 

README.md

@@ -285,13 +285,12 @@ The changes you make in the web server code will then reflect directly in the we
 ### Testing
 
 #### Nextflow pipelines
-The nextflow pipeline is tested using a python integration of nextflow [nextflow.py](https://github.com/goodwright/nextflow.py) and standard [unittests](https://docs.python.org/3/library/unittest.html). You'll therefore need to install [nextflow.py](https://github.com/goodwright/nextflow.py).
+The nextflow pipeline is tested using a python integration of nextflow [nextflow.py](https://github.com/goodwright/nextflow.py) and standard [unittests](https://docs.python.org/3/library/unittest.html).
 
 To run the tests you need a python environment with the required dependencies:
 ```
 conda env create -f conda/testing.yaml
 conda activate testing
-pip install nextflowpy
 ```
 
 You can then simply call
@@ -328,6 +327,7 @@ If you want to contribute, feel free to open a PR describing your changes and ma
 ### Making a release
 
 - Adapt CHANGELOG.md with release number and date.
+- Run the tests (also checking that all conda envs build correctly).
 - Create docker containers:
     - Containers are defined in https://github.com/metagenlab/docker-images/
     - Update `requirements.txt` and `Dockerfile`s for `zdb` and `annotation-pipeline`

conda/checkm.yaml

@@ -1,6 +1,7 @@
 name: checkm
 channels:
+    - conda-forge
     - bioconda
-    - metagenlab
 dependencies:
     - checkm-genome=1.2.3
+    - setuptools

conda/testing.yaml

@@ -10,5 +10,7 @@ dependencies:
     - numpy
     - pandas
     - singularity=3.8.4
-    - nextflow <=22.10
+    - nextflow=22.10
     - sqlalchemy
+    - pip:
+        - nextflowpy

testing/conda/test_environments.py

@@ -0,0 +1,47 @@
+import glob
+import os
+import subprocess
+from tempfile import TemporaryDirectory
+
+from testing.pipelines.base import BaseTestCase
+
+
+class TestCondaEnvironments(BaseTestCase):
+
+    env_commands = {
+        "amrfinderplus.yaml": "amrfinder --version",
+        'annotation.yaml': 'python -c "import Bio"',
+        "blast.yaml": "blastp -h",
+        "checkm.yaml": "checkm -h",
+        "fasttree.yaml": "fasttree -expert",
+        "kofamscan.yaml": "exec_annotation -h",
+        "mafft.yaml": "mafft --version",
+        "main.yaml": "nextflow -v && singularity version",
+        "orthofinder.yaml": "orthofinder -h",
+        "pfam_scan.yaml": "pfam_scan.pl -h",
+        'testing.yaml': "nextflow -v && singularity version",
+        'webapp.yaml': 'python -c "import django"',
+        'zdb.yaml': "zdb",
+        }
+
+    @property
+    def conda_dir(self):
+        basedir = os.path.dirname(os.path.dirname(self.test_dir))
+        return os.path.join(basedir, "conda")
+
+    def create_env(self, filename, envdir):
+        filepath = os.path.join(self.conda_dir, filename)
+        subprocess.check_call(f"mamba env create -f {filepath} -p {envdir}", shell=True)
+
+    def run_with_env(self, command, envdir):
+        subprocess.check_call(f"conda run -p {envdir} {command}", shell=True)
+
+    def test_all_environments_are_tested(self):
+        yaml_files = glob.glob("*.yaml", root_dir=self.conda_dir)
+        self.assertItemsEqual(self.env_commands.keys(), yaml_files)
+
+    def test_all_environments_work(self):
+        for filename, command in self.env_commands.items():
+            with TemporaryDirectory() as tempdir:
+                self.create_env(filename, tempdir)
+                self.run_with_env(command, tempdir)