Merge pull request #15 from jsalignon/Updating_for_compatibility_to_n…

…ewest_nextflow_versions

Updating for compatibility to newest nextflow versions

CHANGELOG.md

@@ -27,6 +27,31 @@ Types of changes
 
 
 
+## [v0.8.4](https://github.com/jsalignon/cactus/releases/tag/v0.8.4) - Matucana, 09-06-2023
+
+### Added
+
+### Changed
+
+- Changed the code to make Cactus work with the latest versions of Nextflow, Singularity and Docker. [commit](https://github.com/jsalignon/cactus/commit/ce2b6dd3d8ca8beee5479e85a4d24e4c4b022641)
+- Changed the documentation to clarify the dependencies. [commit](https://github.com/jsalignon/cactus/commit/0f45173c61a78dd8ce742ff6413c90d1c699575e)
+- Changed the CITATIONS.md and the README.md to add a link to Cactus preprint. [commit](https://github.com/jsalignon/cactus/commit/0e7b6ba4cb4269708897f8ad27adc57048dc9229)
+
+### Fixed
+
+- Fixed a bug happening in conda/Mamba when users have a .Renviron or a .Rprofile file. [commit](https://github.com/jsalignon/cactus/commit/a188dbb31dfe34547aaab427175dde28237ac36a)
+- Fixed a bug with conda/Mamba for merging pdf. [commit](https://github.com/jsalignon/cactus/commit/a188dbb31dfe34547aaab427175dde28237ac36a)
+
+### Deprecated
+
+- Deprecated the use of conda and Mamba temporarily until a bug in conda is resolved. [commit](https://github.com/jsalignon/cactus/commit/0f45173c61a78dd8ce742ff6413c90d1c699575e)
+
+### Removed
+
+### Dependencies
+
+
+
 ## [v0.8.3](https://github.com/jsalignon/cactus/releases/tag/v0.8.3) - Espostoopsis, 15-05-2023
 
 ### Added

CITATIONS.md

@@ -25,7 +25,9 @@
 
 ## jsalignon/cactus
 
-> Manuscript under preparation
+- [Cactus](https://doi.org/10.1101/2023.05.11.540110)
+
+> Salignon J, Millan-Ariño L, Garcia M, Riedel C G. Cactus: a user-friendly and reproducible ATAC-Seq and mRNA-Seq analysis pipeline for data preprocessing, differential analysis, and enrichment analysis. doi: 10.1101/2023.05.11.540110.
 
 
 ## Languages

README.md

@@ -23,6 +23,6 @@ This introductory section provides a quick overview of how Cactus works, with:
  - A [flowchart](/docs/1_Intro/Flowchart.md) that details all the key steps of the analysis
  - An overview of the [output](/docs/1_Intro/Outputs_structure.md) files generated by Cactus 
 
-**Reference**: The preprint titled "*Cactus: a user-friendly and reproducible ATAC-Seq and mRNA-Seq analysis pipeline for data preprocessing, differential analysis, and enrichment analysis*" can be found [here](https://www.biorxiv.org/content/10.1101/2023.05.11.540110v1).
+**Reference**: The preprint titled "*Cactus: a user-friendly and reproducible ATAC-Seq and mRNA-Seq analysis pipeline for data preprocessing, differential analysis, and enrichment analysis*" can be found [here](https://doi.org/10.1101/2023.05.11.540110).
 
 **Licence**: This source code is released under the MIT license, included [here](LICENCE.md).

conf/containers.config

@@ -1,3 +1,14 @@
+
+
+// Export these variables to prevent local Python/R libraries from conflicting 
+// with those in the container
+env {
+    PYTHONNOUSERSITE = 1
+    R_PROFILE_USER   = "/.Rprofile"
+    R_ENVIRON_USER   = "/.Renviron"
+}
+
+
 profiles {
   conda {
     conda.enabled       = true
@@ -28,7 +39,7 @@ profiles {
       libraryDir        = params.singularity_cache_dir
       enabled           = true
       autoMounts        = true
-      runOptions        = "--containall --cleanenv --home ${PWD} --workdir /dev/shm"
+      runOptions        = "--containall --cleanenv --home ${PWD} --bind ${params.references_dir} --workdir /dev/shm"
       // runOptions        = "--containall --cleanenv --home ${PWD} --workdir ${PWD}"
       // runOptions        = "--containall --cleanenv --home ${PWD} --workdir ${params.cactus_dir}/tmp"
 
@@ -90,9 +101,9 @@ process {
     conda     = (params.enable_conda ? "bioconda::homer=4.9.1" : null)
     container = (!params.enable_conda ? "${params.depot_url}/homer:4.9.1--pl5.22.0_5" : null)
   }
-  withLabel:pdftools {
-    conda     = (params.enable_conda ? "johannr::pdftools=1.0.6" : null)
-    container = (!params.enable_conda ? "xaratustrah/pdftools" : null)
+  withLabel:pypdf2 {
+    conda     = (params.enable_conda ? "conda-forge::pypdf2=2.11.1" : null)
+    container = (!params.enable_conda ? "demisto/chromium:1.0.0.35353" : null)
   }
 
 

conf/reports.config

@@ -17,8 +17,9 @@ report {
 }
 
 dag {
-	enabled = true
-	file    = "${params.report_dir}/dag.mmd"
+	enabled       = true
+	file          = "${params.report_dir}/dag.mmd"
+	dag.overwrite = true
 }
 
 trace {

docs/1_Intro/Quick_start.md

@@ -86,7 +86,7 @@ nextflow run jsalignon/cactus -r release_tag -profile {singulariy,docker,conda,m
 ```
 For instance:
 ```
-nextflow run jsalignon/cactus -r v0.8.1 -profile singularity -params-file parameters/full_test.yml
+nextflow run jsalignon/cactus -r v0.8.4 -profile singularity -params-file parameters/full_test.yml
 ```
 
 # Troubleshooting

docs/2_Install/Dependencies.md

@@ -11,27 +11,23 @@
 [](END_OF_MENU)
 
 
-Cactus was built with the goal to make the installation as pain-free as possible. This was achieved by using only tools within containers or virtual environments for all analysis. Installation of most tools is done automatically the first time the pipeline is run. However, two key dependencies should still be installed. These are the workflow language [Nextflow](https://doi.org/10.1038/nbt.3820) ([Install](https://www.nextflow.io/docs/latest/getstarted.html#installation)) and a tools manager that can be any of:
- - [SingularityCE](https://doi.org/10.1371/journal.pone.0177459) ([Install](https://docs.sylabs.io/guides/latest/admin-guide/installation.html))
- - [Docker](https://dl.acm.org/doi/10.5555/2600239.2600241) ([Install](https://docs.docker.com/get-docker/))
- - [conda](https://docs.anaconda.com/anaconda/reference/release-notes/) ([Install](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html))
- - [Mamba](https://medium.com/@QuantStack/open-software-packaging-for-science-61cecee7fc23) ([Install](https://mamba.readthedocs.io/en/latest/installation.html))
+Cactus was built with the goal to make the installation as pain-free as possible. This was achieved by using only tools within containers or virtual environments for all analysis. Installation of most tools is done automatically the first time the pipeline is run (see the [Quick Start](/docs/1_Intro/Quick_start.md) section). However, two key dependencies should still be installed: [Nextflow](https://doi.org/10.1038/nbt.3820), and a package manager; one of these tools: [SingularityCE](https://doi.org/10.1371/journal.pone.0177459), [Docker](https://dl.acm.org/doi/10.5555/2600239.2600241), [conda](https://docs.anaconda.com/anaconda/reference/release-notes/) and [Mamba](https://medium.com/@QuantStack/open-software-packaging-for-science-61cecee7fc23). Among these, Singularity is the recommended package manager to use if possible.
 
-When using conda or mamba, the set-up should be as described in [Bioconda's usage section](https://bioconda.github.io/#usage):
-```
-conda config --add channels defaults
-conda config --add channels bioconda
-conda config --add channels conda-forge
-conda config --set channel_priority strict
-```
+The current stable release of Cactus (v0.8.4) was developed and tested using these versions of the dependencies:
+ - Nextflow: 22.10.8.5859
+ - SingularityCE: 3.11.3-focal
+ - Docker: 20.10.21-0ubuntu1~20.04.2
+ - conda/Mamba: currently, running Cactus using conda or Mamba is not possible due to a [conda bug](https://github.com/conda/conda/issues/12786). 
+
+In case of issues with running Cactus, please make sure you are using the same version of the dependencies.
 
-During Cactus development, Mamba was installed with this command: `conda install mamba -n base -c conda-forge`
+You can find here installation links for: [Nextflow](https://www.nextflow.io/docs/latest/getstarted.html#installation), [SingularityCE](https://docs.sylabs.io/guides/latest/admin-guide/installation.html), [Docker](https://docs.docker.com/get-docker/), [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html) and [Mamba](https://mamba.readthedocs.io/en/latest/installation.html).
 
-In case of issue with the pipeline and/or for reproducibility purposes, it might be useful to try to run Cactus with the version of these tools that were used when developing the current Cactus release. These are:  
-  - Nextflow: 22.09.2-edge.5765
-  
-  - SingularityCE: 3.10.0+91-g13f189977
-  - Conda: 22.9.0
-  - Mamba 0.27.0 (conda 22.9.0)
+Please, note that Cactus is currently written in DSL1, which is [not supported anymore](https://nextflow.io/podcast/2023/ep9_end_of_dsl1_chatting_to_bots.html) by the latest default versions of Nextflow. For now, Cactus users should use Nextflow version 22.10.X or earlier, such as v22.10.8 released on April 15 2023. This can be achieved with these commands:
+```
+wget -qO- https://github.com/nextflow-io/nextflow/releases/download/v22.10.8/nextflow-22.10.8-all | bash
+chmod +x nextflow
+sudo mv nextflow /usr/local/bin
+```
 
-Docker version 18.06.1-ce, build e68fc7a
+Please check out the [Quick Start](/docs/1_Intro/Quick_start.md) section to learn how to run Cactus once the dependencies are installed.

main.nf

@@ -4758,7 +4758,7 @@ Merging_pdfs_channel = Merging_pdfs_channel.dump(tag: 'merging_pdf')
 process Figures__merging_pdfs {
   tag "${file_name}"
 
-  label "pdftools"
+  label "pypdf2"
 
   publishDir path: "${out_fig_merge}/${out_path}", mode: "${pub_mode}"
 
@@ -4770,8 +4770,16 @@ process Figures__merging_pdfs {
 
   script:
     """
+    #!/usr/bin/env python
 
-    pdfunite `ls *pdf | sort` ${file_name}.pdf
+    from glob import glob
+    from PyPDF2 import PdfMerger
+
+    merger = PdfMerger()
+    allpdfs = [a for a in glob("*.pdf")]
+    [merger.append(pdf) for pdf in allpdfs]
+    with open("${file_name}.pdf", "wb") as new_file:
+      merger.write(new_file)
 
     """
 }