Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Install subworkflows with modules from different remotes #3083

Open
wants to merge 71 commits into
base: dev
Choose a base branch
from

Commits on Jun 21, 2024

  1. tests: Add test case for cross-organization subwf

    Strangely it is passing at the moment, it seems the install command
    returns a passing exit code regardless of installing all modules.
    
    - Addresses #2
    jvfe committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    9825d89 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. Configuration menu
    Copy the full SHA
    568363a View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Configuration menu
    Copy the full SHA
    534da92 View commit details
    Browse the repository at this point in the history
  2. Revert "chore: Dummy commit to test action"

    This reverts commit 534da92.
    jvfe committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    286ab86 View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. Configuration menu
    Copy the full SHA
    423517e View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. Merge pull request #6 from sanger-tol/cross-org-test

    tests: Add test case for cross-organization subwf
    jvfe authored Jul 8, 2024
    Configuration menu
    Copy the full SHA
    6816cc1 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    af816fd View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. Configuration menu
    Copy the full SHA
    d0e6031 View commit details
    Browse the repository at this point in the history
  2. fix: Add correct return type

    jvfe committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    1737c88 View commit details
    Browse the repository at this point in the history
  3. fix: Use any for values

    jvfe committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    f7aaea2 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2024

  1. Configuration menu
    Copy the full SHA
    d1c490d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6fa5cf4 View commit details
    Browse the repository at this point in the history
  3. refact: Copy parent attribute

    jvfe committed Jul 28, 2024
    Configuration menu
    Copy the full SHA
    d903282 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    7f095fa View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    012e9d6 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    d77fccb View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    faa7faf View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    0f08bbe View commit details
    Browse the repository at this point in the history
  9. fix: Pass branch in install too

    jvfe committed Jul 28, 2024
    Configuration menu
    Copy the full SHA
    6d052de View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. Configuration menu
    Copy the full SHA
    c6ce67f View commit details
    Browse the repository at this point in the history
  2. Merge pull request #9 from sanger-tol/fix/duplicated-downloads

    fix: Avoid duplicate downloads in nested subworkflows
    jvfe authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    792f68b View commit details
    Browse the repository at this point in the history
  3. Merge branch 'dev' into fix/1927

    * dev: (299 commits)
      Update nf_core/pipelines/create/utils.py
      [automated] Update CHANGELOG.md
      allow numbers in custom pipeline name
      [automated] Update CHANGELOG.md
      Update pre-commit hook pre-commit/mirrors-mypy to v1.11.1
      Fixed linting
      Updated changelog
      Added process_high_memory to create/lint list
      update chagelog
      use pathlib instead of os.path
      Apply suggestions from code review
      add option to skip code linters
      Update gitpod/workspace-base Docker digest to f189a41
      Update pre-commit hook pre-commit/mirrors-mypy to v1.11.0
      [automated] Update CHANGELOG.md
      Update python:3.12-slim Docker digest to 740d94a
      Update .pre-commit-config.yaml
      Update nf_core/__main__.py
      Add default version
      Automatically add a default version in pipelines_create
      ...
    jvfe committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    7d23f15 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2024

  1. Merge branch 'dev' into review-round-1

    * dev: (35 commits)
      don't remove creating a pipeline to test it 😶
      update textual snapshots
      update textual snapshots
      fix indentation
      Update nf_core/pipelines/create/create.py
      [automated] Update CHANGELOG.md
      [automated] Update CHANGELOG.md
      add option to exclude changelog from custom pipeline template
      add option to exclude codespaces from pipeline template
      remove multiqc images from the template docs
      and the same for subworkflows
      run tests also after pytest-migrate
      update textual snapshots
      add multiqc potion to test data
      update textual snapshots
      fix tests
      [automated] Update CHANGELOG.md
      add option to exclude multiqc from pipeline template
      rename regenerae-snapshots to update-textual-snapshots
      Apply suggestions from code review
      ...
    jvfe committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    6f9a60e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    fca6f27 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    092bf91 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    208d796 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2024

  1. Configuration menu
    Copy the full SHA
    71c43be View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8bcf737 View commit details
    Browse the repository at this point in the history
  3. fix: Change modules to subwfs

    jvfe committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    efde7b7 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8e25587 View commit details
    Browse the repository at this point in the history
  5. Revert "fix: Use name value in recreate deps"

    This reverts commit 8e25587.
    jvfe committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    4947913 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    4888d32 View commit details
    Browse the repository at this point in the history
  7. fix: Use sw_name in appends too

    jvfe committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    aa265d8 View commit details
    Browse the repository at this point in the history
  8. Merge branch 'dev' into fix/1927

    * dev: (35 commits)
      don't remove creating a pipeline to test it 😶
      update textual snapshots
      update textual snapshots
      fix indentation
      Update nf_core/pipelines/create/create.py
      [automated] Update CHANGELOG.md
      [automated] Update CHANGELOG.md
      add option to exclude changelog from custom pipeline template
      add option to exclude codespaces from pipeline template
      remove multiqc images from the template docs
      and the same for subworkflows
      run tests also after pytest-migrate
      update textual snapshots
      add multiqc potion to test data
      update textual snapshots
      fix tests
      [automated] Update CHANGELOG.md
      add option to exclude multiqc from pipeline template
      rename regenerae-snapshots to update-textual-snapshots
      Apply suggestions from code review
      ...
    jvfe committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    ab0f398 View commit details
    Browse the repository at this point in the history
  9. Merge branch 'fix/1927' into review-round-1

    * fix/1927:
    jvfe committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    33c36cf View commit details
    Browse the repository at this point in the history
  10. Merge branch 'feat/use-subwf' into review-round-1

    * feat/use-subwf:
      fix: Use sw_name in appends too
      fix: Use subworkflow name in recreate deps
      Revert "fix: Use name value in recreate deps"
      fix: Use name value in recreate deps
      fix: Change modules to subwfs
      refact: Support subwf dict in recreate deps
      refact: Rework logic to use subwfs as well
    jvfe committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    9356cd3 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    a764c76 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. fix: Handle incomplete meta.yml

    jvfe committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    869d816 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c3ac1b8 View commit details
    Browse the repository at this point in the history
  3. style: Format meta_yml.py

    jvfe committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    4eb95e6 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    e9bf238 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    c8b8a83 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    eba8391 View commit details
    Browse the repository at this point in the history
  7. Merge pull request #10 from sanger-tol/review-round-1

    refact: Add suggestions from 1st review round
    jvfe authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    e3f7b3f View commit details
    Browse the repository at this point in the history
  8. Merge branch 'dev' into merge-dev

    * dev: (102 commits)
      [automated] Update CHANGELOG.md
      add templatefeatures.yml to python package
      fix typing
      Update nf_core/pipelines/create/utils.py
      set force to true when creating a pipeline from the app
      load feature yaml file with a utils function
      apply suggestions from code review by @mashehu
      reset lintconfigtype
      don't read from json on matrix creation
      try reading yaml keys with a list format
      update textual snapshots
      use template features yaml file to generate textual layout
      [automated] Update CHANGELOG.md
      handle template features with a yaml file
      update ruff
      fix tests
      fix refgenie tests
      add type hints to utils functions
      fix tests
      remove unnecessary string conversion
      ...
    jvfe committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    ddb3dd0 View commit details
    Browse the repository at this point in the history
  9. Merge pull request #11 from sanger-tol/merge-dev

    chore: Merge changes from dev
    jvfe authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    634ff8f View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2024

  1. Configuration menu
    Copy the full SHA
    c72b94a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    bb31d78 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. Configuration menu
    Copy the full SHA
    7f4cce8 View commit details
    Browse the repository at this point in the history
  2. refact: Don't default to master

    jvfe committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    e342678 View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. refact: Move instance check up

    jvfe committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    50216c4 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0e83d9a View commit details
    Browse the repository at this point in the history
  3. Revert "Refactoring of component_utils.py"

    This reverts commit 0e83d9a.
    jvfe committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    f8c8251 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2024

  1. Configuration menu
    Copy the full SHA
    313853e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e8e4da0 View commit details
    Browse the repository at this point in the history
  3. refact: Use get in modules_json

    jvfe committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    f783d58 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    eb59308 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    be4d78f View commit details
    Browse the repository at this point in the history
  6. Merge pull request #13 from sanger-tol/refact/change_structure_levera…

    …ge_get
    
    Use the power of get to skip if tests
    jvfe authored Aug 21, 2024
    Configuration menu
    Copy the full SHA
    3c493b9 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2024

  1. Configuration menu
    Copy the full SHA
    b13a406 View commit details
    Browse the repository at this point in the history
  2. style: Run ruff format

    jvfe committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    e96341e View commit details
    Browse the repository at this point in the history
  3. refact: Change UserWarning message

    Co-authored-by: Matthieu Muffato <mm49@sanger.ac.uk>
    jvfe and muffato authored Aug 22, 2024
    Configuration menu
    Copy the full SHA
    2b9a573 View commit details
    Browse the repository at this point in the history
  4. style: Run ruff format

    jvfe committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    e55c74d View commit details
    Browse the repository at this point in the history
  5. Merge pull request #12 from sanger-tol/refact/change_structure

    refact: Change function structure to use dicts not lists
    jvfe authored Aug 22, 2024
    Configuration menu
    Copy the full SHA
    4b9fea5 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. Merge remote-tracking branch 'upstream/dev' into merge-dev2

    * upstream/dev: (45 commits)
      Apply suggestions from code review
      update textual snapshot
      add option to exclude adaptivecard and slackreport from pipeline template
      update pytest test_make_pipeline_schema
      [automated] Update CHANGELOG.md
      add option to exclude email from pipeline template
      update textual snapshots
      Update nf_core/pipelines/create/templatefeatures.yml
      [automated] Update CHANGELOG.md
      add option to exclude license from pipeline template
      [automated] Update CHANGELOG.md
      Update python:3.12-slim Docker digest to 59c7332
      [automated] Update CHANGELOG.md
      Update pre-commit hook astral-sh/ruff-pre-commit to v0.6.0
      Ran pre commit
      add matthias' suggestion woops
      Overwrite the verbose in ctx in case the user uses it
      Overwrite the verbose in ctx in case the user uses it
      Add a verbose click option
      run prettier on the pipeline output directory
      ...
    jvfe committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    ea7d1ca View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2024

  1. Merge remote-tracking branch 'upstream/dev' into fix/1927

    * upstream/dev: (120 commits)
      markdown header
      add .gitignore to base required files
      update textual snapshots
      don't remove .gitignore when unselecting github
      update textual snapshots
      some more minimal template fixes
      udpate changelog
      handle cases where the directory path contains the name of the component
      switch to os.walk to support <3.10 versions of Python
      fix pipeline linting when skipping nf_schema
      update test to use template_features.yml and add missing files to a feature group
      rename templatefeatures.yml to template_features.yml
      update changelog
      fix linting
      add tests to ensure all files are part of a template customisation group
      add process cpus, memory and time to nextflow.config if we remove base.config
      update textual snapshots
      Revert "don't remove base.config when removing nf-core components"
      don't remove base.config when removing nf-core components
      apply review comments from @mashehu
      ...
    jvfe committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    a40d5d9 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2024

  1. Configuration menu
    Copy the full SHA
    2491207 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a870787 View commit details
    Browse the repository at this point in the history
  3. Merge remote-tracking branch 'upstream/dev' into second-review

    * upstream/dev:
      Move if statements to top of YAML blocks
      Don't look for dev, look for not master
      Remove tests involving environment.yml with 'name'
      Remove name from conda enviroment.yml in module template
      Don't test conda `environment.yml` `name` attribute (which should no longer be there)
      Update CHANGELOG.md
      Add --update-all flag to remove defaults channels dependents
      add a tabix tabix test
      pre commit
      fix issue where tool and subtool have the same name
      udpate changelog
      run nf-core lint --release on PRs to master
    jvfe committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    7ad4a8c View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. Merge pull request #14 from sanger-tol/second-review

    refact: Add suggestions from second review
    jvfe authored Sep 9, 2024
    Configuration menu
    Copy the full SHA
    f3259bd View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. docs: Add comment for clarification about the feature

    Co-authored-by: Júlia Mir Pedrol <mirp.julia@gmail.com>
    jvfe and mirpedrol authored Sep 13, 2024
    Configuration menu
    Copy the full SHA
    1a8c3be View commit details
    Browse the repository at this point in the history