plugin/nf-validation error with old version of Nextflow #293

pecko-g · 2024-05-17T10:35:13Z

Description of the bug

Hi again,

I'm having trouble running the test profile of the pgsc calc in a new environment. When I run the test command, I get the error line "Module path must start with / or ./ prefix -- Offending module: plugin/nf-validation".

I had a look around some folders and couldn't find any "nf-validation" file/folder and I am unsure which config files I could correct.

I also just read that nf-validation is deprecated. I'm running nextflow version 22.04.5, could that be the problem? Unfortunately I don't have permission to update Nextflow on my own, so could there be a way around?

Thank you in advance.

Best wishes

Command used and terminal output

$ nextflow run pgscatalog/pgsc_calc -profile test,singularity


.nextflow.log file:

May-17 10:03:27.557 [main] DEBUG nextflow.cli.Launcher - $> nextflow run pgscatalog/pgsc_calc -profile test,singularity
May-17 10:03:27.661 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 22.04.5
May-17 10:03:29.588 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/jupyter/.nextflow/assets/pgscatalog/pgsc_calc/.git/config; branch: main; remote: origin; url: https://github.com/PGScatalog/pgsc_calc.git
May-17 10:03:29.608 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/jupyter/.nextflow/assets/pgscatalog/pgsc_calc/.git/config; branch: main; remote: origin; url: https://github.com/PGScatalog/pgsc_calc.git
May-17 10:03:30.288 [main] DEBUG nextflow.config.ConfigBuilder - Found config home: /home/jupyter/.nextflow/config
May-17 10:03:30.288 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/jupyter/.nextflow/assets/pgscatalog/pgsc_calc/nextflow.config
May-17 10:03:30.290 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/jupyter/.nextflow/config
May-17 10:03:30.290 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/jupyter/.nextflow/assets/pgscatalog/pgsc_calc/nextflow.config
May-17 10:03:30.303 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `test,singularity`
May-17 10:03:30.360 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `test,singularity`
May-17 10:03:30.905 [main] DEBUG nextflow.plugin.PluginsFacade - Using Default plugins manager
May-17 10:03:30.921 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
May-17 10:03:30.922 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
May-17 10:03:30.927 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
May-17 10:03:30.933 [main] DEBUG nextflow.plugin.PluginsFacade - Using Default plugins manager
May-17 10:03:31.714 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [cfc_dev, uzl_omics, ifb_core, denbi_qbic, alice, mjolnir_globe, uppmax, incliva, ilifu, uge, rosalind_uge, lugh, mccleary, unibe_ibu, vai, czbiohub_aws, jax, ccga_med, scw, unc_longleaf, tigem, tubingen_apg, google, apollo, gls, ipop_up, vsc_calcua, pdc_kth, googlels, daisybio, eddie, medair, biowulf, apptainer, bi, bigpurple, adcra, cedars, pawsey_setonix, vsc_kul_uhasselt, pawsey_nimbus, ucl_myriad, utd_ganymede, charliecloud, icr_davros, ceres, munin, arm, rosalind, hasta, cfc, uzh, ebi_codon_slurm, ebc, ku_sund_dangpu, ccga_dx, crick, marvin, biohpc_gen, shifter, mana, mamba, york_viking, unc_lccc, wehi, awsbatch, wustl_htcf, arcc, imperial, maestro, software_license, utd_europa, genotoul, nci_gadi, abims, janelia, nu_genomics, googlebatch, oist, sahmri, mpcdf, leicester, vsc_ugent, create, sage, cambridge, jex, podman, ebi_codon, cheaha, xanadu, nyu_hpc, test, marjorie, computerome, ucd_sonic, seg_globe, sanger, dkfz, pasteur, einstein, ethz_euler, m3c, test_full, imb, ucl_cscluster, tuos_stanage, azurebatch, hki, crukmi, csiro_petrichor, qmul_apocrita, docker, engaging, gis, hypatia, psmn, eva, nygc, fgcz, conda, crg, singularity, self_hosted_runner, tufts, uw_hyak_pedslabs, utd_sysbio, debug, genouest, cbe, phoenix, gitpod, seawulf, uod_hpc, fub_curta, uct_hpc, aws_tower, binac]
May-17 10:03:31.754 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
May-17 10:03:31.755 [main] INFO  nextflow.cli.CmdRun - Launching `https://github.com/pgscatalog/pgsc_calc` [intergalactic_dalembert] DSL2 - revision: 8bdf287d55 [main]
May-17 10:03:31.757 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; plugins-dir=/home/jupyter/.nextflow/plugins; core-plugins: nf-amazon@1.7.2,nf-azure@0.13.2,nf-console@1.0.3,nf-ga4gh@1.0.3,nf-google@1.1.4,nf-sqldb@0.4.0,nf-tower@1.4.0
May-17 10:03:31.758 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
May-17 10:03:31.762 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
May-17 10:03:31.762 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
May-17 10:03:31.764 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
May-17 10:03:31.774 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
May-17 10:03:31.840 [main] DEBUG nextflow.Session - Session uuid: 400e4acd-eeb6-4f2d-a2ad-9dbe431a0ceb
May-17 10:03:31.840 [main] DEBUG nextflow.Session - Run name: intergalactic_dalembert
May-17 10:03:31.841 [main] DEBUG nextflow.Session - Executor pool size: 4
May-17 10:03:31.880 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 22.04.5 build 5708
  Created: 15-07-2022 16:09 UTC 
  System: Linux 5.15.146+
  Runtime: Groovy 3.0.10 on OpenJDK 64-Bit Server VM 11.0.20.1+1-post-Ubuntu-0ubuntu120.04
  Encoding: UTF-8 (UTF-8)
  Process: 696@906f4f5ddf8c [172.18.0.4]
  CPUs: 4 - Mem: 13.2 GB (12.2 GB) - Swap: 0 (0)
May-17 10:03:31.901 [main] DEBUG nextflow.Session - Work-dir: /home/jupyter/workspaces/lifetimeriskestimationinmultiancestrybiobanks/work [ext2/ext3]
May-17 10:03:31.920 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
May-17 10:03:31.933 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
May-17 10:03:32.050 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
May-17 10:03:32.062 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 5; maxThreads: 1000
May-17 10:03:32.182 [main] DEBUG nextflow.Session - Session start invoked
May-17 10:03:32.188 [main] DEBUG nextflow.trace.TraceFileObserver - Flow starting -- trace file: /home/jupyter/.nextflow/assets/pgscatalog/pgsc_calc/results/pipeline_info/execution_trace_2024-05-17_10-03-31.txt
May-17 10:03:32.201 [main] DEBUG nextflow.Session - Using default localLib path: /home/jupyter/.nextflow/assets/pgscatalog/pgsc_calc/lib
May-17 10:03:32.206 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/jupyter/.nextflow/assets/pgscatalog/pgsc_calc/lib
May-17 10:03:33.070 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
May-17 10:03:33.875 [main] DEBUG nextflow.Session - Session aborted -- Cause: Module path must start with / or ./ prefix -- Offending module: plugin/nf-validation
May-17 10:03:33.894 [main] ERROR nextflow.cli.Launcher - @unknown
nextflow.exception.IllegalModulePath: Module path must start with / or ./ prefix -- Offending module: plugin/nf-validation
	at nextflow.script.IncludeDef.checkValidPath(IncludeDef.groovy:190)
	at nextflow.script.IncludeDef.load0(IncludeDef.groovy:110)
	at nextflow.script.IncludeDef$load0$1.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
	at Script_5cb38c3c.runScript(Script_5cb38c3c:7)
	at nextflow.script.BaseScript.runDsl2(BaseScript.groovy:170)
	at nextflow.script.BaseScript.run(BaseScript.groovy:217)
	at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:220)
	at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:206)
	at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:200)
	at nextflow.script.IncludeDef.memoizedMethodPriv$loadModule0PathMapSession(IncludeDef.groovy:142)
	at nextflow.script.IncludeDef.access$0(IncludeDef.groovy)
	at nextflow.script.IncludeDef$__clinit__closure1.doCall(IncludeDef.groovy)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
	at groovy.lang.Closure.call(Closure.java:412)
	at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.lambda$call$0(Memoize.java:137)
	at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:137)
	at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:113)
	at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.call(Memoize.java:136)
	at nextflow.script.IncludeDef.loadModule0(IncludeDef.groovy)
	at nextflow.script.IncludeDef.load0(IncludeDef.groovy:114)
	at nextflow.script.IncludeDef$load0$1.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
	at Script_f7d3e816.runScript(Script_f7d3e816:35)
	at nextflow.script.BaseScript.runDsl2(BaseScript.groovy:170)
	at nextflow.script.BaseScript.run(BaseScript.groovy:217)
	at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:220)
	at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:212)
	at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:120)
	at nextflow.cli.CmdRun.run(CmdRun.groovy:337)
	at nextflow.cli.Launcher.run(Launcher.groovy:480)
	at nextflow.cli.Launcher.main(Launcher.groovy:639)

Relevant files

No response

System information

Nextflow version 22.04.5

smlmbrt · 2024-05-24T10:43:45Z

@pecko-g is there any way for you to ask the system administrators to update nextflow? This might be the easiest fix.

nebfield · 2024-05-24T11:03:09Z

It should be possible to install a local copy of Nextflow for your user account:

$ curl -s https://get.nextflow.io | bash

This will download the Nextflow executable file to your current directory.

You can then put the executable file in your home bin folder. I do this on a HPC which provides an old version of Nextflow.

pecko-g · 2024-05-27T10:53:55Z

Thank you @nebfield, that worked! Is there a similar solution for installing Singularity or Docker? I don't have permission to use sudo.

I thought I could run the pipeline with Conda, but it's telling me it can't create an environment...

ERROR ~ Error executing process > 'PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1)'

Caused by:
Failed to create Conda environment
command: conda env create --prefix /home/jupyter/workspaces/lifetimeriskestimationinmultiancestrybiobanks/work/conda/pgscatalog_utils-509d76d1ac8db7e37291670bd7917365 --file /home/jupyter/.nextflow/assets/pgscatalog/pgsc_calc/environments/pgscatalog_utils/environment.yml
status : 1
message:
Collecting package metadata (repodata.json): ...working... failed

  # >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

      Traceback (most recent call last):
        File "/opt/conda/lib/python3.10/site-packages/conda/exception_handler.py", line 17, in __call__
          return func(*args, **kwargs)
        File "/opt/conda/lib/python3.10/site-packages/conda_env/cli/main.py", line 56, in do_call
          exit_code = getattr(module, func_name)(arguments, parser)
        File "/opt/conda/lib/python3.10/site-packages/conda/notices/core.py", line 124, in wrapper
          return func(*args, **kwargs)
        File "/opt/conda/lib/python3.10/site-packages/conda_env/cli/main_create.py", line 162, in execute
          result[installer_type] = installer.install(
        File "/opt/conda/lib/python3.10/site-packages/conda_env/installers/conda.py", line 51, in install
          unlink_link_transaction = solver.solve_for_transaction(
        File "/opt/conda/lib/python3.10/site-packages/conda/core/solve.py", line 154, in solve_for_transaction
          unlink_precs, link_precs = self.solve_for_diff(
        File "/opt/conda/lib/python3.10/site-packages/conda/core/solve.py", line 215, in solve_for_diff
          final_precs = self.solve_final_state(
        File "/opt/conda/lib/python3.10/site-packages/conda/core/solve.py", line 358, in solve_final_state
          ssc = self._collect_all_metadata(ssc)
        File "/opt/conda/lib/python3.10/site-packages/conda/common/io.py", line 84, in decorated
          return f(*args, **kwds)
        File "/opt/conda/lib/python3.10/site-packages/conda/core/solve.py", line 572, in _collect_all_metadata
          index, r = self._prepare(prepared_specs)
        File "/opt/conda/lib/python3.10/site-packages/conda/core/solve.py", line 1286, in _prepare
          reduced_index = get_reduced_index(
        File "/opt/conda/lib/python3.10/site-packages/conda/core/index.py", line 287, in get_reduced_index
          new_records = SubdirData.query_all(
        File "/opt/conda/lib/python3.10/site-packages/conda/core/subdir_data.py", line 158, in query_all
          result = tuple(
        File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
          yield _result_or_cancel(fs.pop())
        File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
          return fut.result(timeout)
        File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 451, in result
          return self.__get_result()
        File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
          raise self._exception
        File "/opt/conda/lib/python3.10/concurrent/futures/thread.py", line 58, in run
          result = self.fn(*self.args, **self.kwargs)
        File "/opt/conda/lib/python3.10/site-packages/conda/core/subdir_data.py", line 143, in subdir_query
          return tuple(
        File "/opt/conda/lib/python3.10/site-packages/conda/core/subdir_data.py", line 165, in query
          self.load()
        File "/opt/conda/lib/python3.10/site-packages/conda/core/subdir_data.py", line 265, in load
          _internal_state = self._load()
        File "/opt/conda/lib/python3.10/site-packages/conda/core/subdir_data.py", line 322, in _load
          repodata, state = fetcher.fetch_latest_parsed()
        File "/opt/conda/lib/python3.10/site-packages/conda/gateways/repodata/__init__.py", line 758, in fetch_latest_parsed
          parsed, state = self.fetch_latest()
        File "/opt/conda/lib/python3.10/site-packages/conda/gateways/repodata/__init__.py", line 820, in fetch_latest
          cache.load_state()
        File "/opt/conda/lib/python3.10/site-packages/conda/gateways/repodata/__init__.py", line 626, in load_state
          self.load(state_only=True)
        File "/opt/conda/lib/python3.10/site-packages/conda/gateways/repodata/__init__.py", line 573, in load
          with self.cache_path_state.open("r+") as state_file, lock(state_file):
        File "/opt/conda/lib/python3.10/pathlib.py", line 1119, in open
          return self._accessor.open(self, mode, buffering, encoding, errors,
      PermissionError: [Errno 13] Permission denied: '/opt/conda/pkgs/cache/497deca9.info.json'

  `$ /opt/conda/bin/conda-env create --prefix /home/jupyter/workspaces/lifetimeriskestimationinmultiancestrybiobanks/work/conda/pgscatalog_utils-509d76d1ac8db7e37291670bd7917365 --file /home/jupyter/.nextflow/assets/pgscatalog/pgsc_calc/environments/pgscatalog_utils/environment.yml`

    environment variables:
           CDR_STORAGE_PATH=gs://fc-aou-datasets-controlled/v7
                   CIO_TEST=<not set>
    CONDA_AUTO_UPDATE_CONDA=false
                 CONDA_ROOT=/opt/conda
             CURL_CA_BUNDLE=<not set>
              GATK_ZIP_PATH=/tmp/gatk-4.3.0.0.zip
                     GOPATH=/home/jupyter/go
            LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda/lib:/usr/local/lib/x86_64-linux-
                            gnu:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/nvidia/li
                            b:/usr/local/nvidia/lib64
                 LD_PRELOAD=<not set>
               LIBRARY_PATH=/usr/local/cuda/lib64/stubs
   LONG_READS_MANIFEST_PATH=gs://fc-aou-datasets-controlled/v7/wgs/long_read/manifest.csv
  MICROARRAY_HAIL_STORAGE_PATH=gs://fc-aou-datasets-controlled/v7/microarray/hail.mt_v7.1
  MICROARRAY_IDAT_MANIFEST_PATH=gs://fc-aou-datasets-controlled/v7/microarray/idat/manifest.csv
  MICROARRAY_VCF_MANIFEST_PATH=gs://fc-aou-datasets-controlled/v7/microarray/vcf/manifest.csv
                       PATH=/home/jupyter/go/bin:/home/jupyter/bin:/opt/conda/bin:/usr/local/nvidi
                            a/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/us
                            r/bin:/sbin:/bin:/home/jupyter/.local/bin:/home/jupyter/packages/bin:/
                            home/jupyter/go/bin:/home/jupyter/go/bin
                 PYTHONPATH=/etc/jupyter/custom:/usr/lib/spark/python:/home/jupyter/packages
         REQUESTS_CA_BUNDLE=<not set>
              SSL_CERT_FILE=<not set>
  WGS_ACAF_THRESHOLD_MULTI_HAIL_PATH=gs://fc-aou-datasets-controlled/v7/wgs/short_read/snpindel/acaf_thresh
                            old_v7.1/multiMT/hail.mt
  WGS_ACAF_THRESHOLD_SPLIT_HAIL_PATH=gs://fc-aou-datasets-controlled/v7/wgs/short_read/snpindel/acaf_thresh
                            old_v7.1/splitMT/hail.mt
  WGS_ACAF_THRESHOLD_VCF_PATH=gs://fc-aou-datasets-
                            controlled/v7/wgs/short_read/snpindel/acaf_threshold_v7.1/vcf
  WGS_CLINVAR_MULTI_HAIL_PATH=gs://fc-aou-datasets-
                            controlled/v7/wgs/short_read/snpindel/clinvar_v7.1/splitMT/hail.mt
  WGS_CLINVAR_SPLIT_HAIL_PATH=gs://fc-aou-datasets-
                            controlled/v7/wgs/short_read/snpindel/clinvar_v7.1/splitMT/hail.mt
       WGS_CLINVAR_VCF_PATH=gs://fc-aou-datasets-
                            controlled/v7/wgs/short_read/snpindel/clinvar_v7.1/vcf
     WGS_CRAM_MANIFEST_PATH=gs://fc-aou-datasets-controlled/v7/wgs/cram/manifest.csv
  WGS_EXOME_MULTI_HAIL_PATH=gs://fc-aou-datasets-
                            controlled/v7/wgs/short_read/snpindel/exome_v7.1/multiMT/hail.mt
  WGS_EXOME_SPLIT_HAIL_PATH=gs://fc-aou-datasets-
                            controlled/v7/wgs/short_read/snpindel/exome_v7.1/splitMT/hail.mt
         WGS_EXOME_VCF_PATH=gs://fc-aou-datasets-
                            controlled/v7/wgs/short_read/snpindel/exome_v7.1/vcf
  WGS_LONGREADS_HAIL_GRCH38_PATH=gs://fc-aou-datasets-controlled/v7/wgs/long_read/hail.mt/GRCh38
  WGS_LONGREADS_HAIL_T2T_PATH=gs://fc-aou-datasets-controlled/v7/wgs/long_read/hail.mt/T2T
  WGS_LONGREADS_JOINT_SNP_INDEL_VCF_GRCH38_PATH=gs://fc-aou-datasets-controlled/v7/wgs/long_read/joint_vcf/GRCh38
  WGS_LONGREADS_JOINT_SNP_INDEL_VCF_T2T_PATH=gs://fc-aou-datasets-controlled/v7/wgs/long_read/joint_vcf/T2T
               WGS_VDS_PATH=gs://fc-aou-datasets-
                            controlled/v7/wgs/short_read/snpindel/vds/hail.vds

       active environment : None
         user config file : /home/jupyter/.condarc
   populated config files : /opt/conda/.condarc
                            /home/jupyter/.condarc
            conda version : 23.7.3
      conda-build version : not installed
           python version : 3.10.12.final.0
         virtual packages : __archspec=1=x86_64
                            __glibc=2.31=0
                            __linux=5.15.146=0
                            __unix=0=0
         base environment : /opt/conda  (writable)
        conda av data dir : /opt/conda/etc/conda
    conda av metadata url : None
             channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                            https://conda.anaconda.org/conda-forge/noarch
                            https://conda.anaconda.org/bioconda/linux-64
                            https://conda.anaconda.org/bioconda/noarch
                            https://repo.anaconda.com/pkgs/main/linux-64
                            https://repo.anaconda.com/pkgs/main/noarch
                            https://repo.anaconda.com/pkgs/r/linux-64
                            https://repo.anaconda.com/pkgs/r/noarch
            package cache : /opt/conda/pkgs
                            /home/jupyter/.conda/pkgs
         envs directories : /opt/conda/envs
                            /home/jupyter/.conda/envs
                 platform : linux-64
               user-agent : conda/23.7.3 requests/2.29.0 CPython/3.10.12 Linux/5.15.146+ ubuntu/20.04.6 glibc/2.31
                  UID:GID : 1000:100
               netrc file : None
             offline mode : False


  An unexpected error has occurred. Conda has prepared the above report.
  If you suspect this error is being caused by a malfunctioning plugin,
  consider using the --no-plugins option to turn off plugins.

  Example: conda --no-plugins install <package>

  Alternatively, you can set the CONDA_NO_PLUGINS environment variable on
  the command line to run the command without plugins enabled.

  Example: CONDA_NO_PLUGINS=true conda install <package>

-- Check '.nextflow.log' file for details

nebfield · 2024-05-28T08:17:40Z

Unfortunately both docker and singularity require administrator access to set up properly. Normally HPC sysadmins are happy to help you with this kind of thing.

Perhaps you could try installing a user version of miniconda with the same approach you used to install nextflow.

pecko-g · 2024-05-28T08:59:48Z

Thank you very much again! It seems we're getting closer. This time it managed to create the environment. Now it says it's missing the 'pgscatalog_utils' module. Maybe I can download it manually into the right folder?

Would that be one of the two?
./work/conda/env-8d4d376488d027827c26cf625add7a32
./work/conda/pgscatalog_utils-509d76d1ac8db7e37291670bd7917365

ERROR ~ Error executing process > 'PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1)'

Caused by:
Process PGSCATALOG_PGSCCALC:PGSCCALC:INPUT_CHECK:COMBINE_SCOREFILES (1) terminated with an error exit status (1)

Command executed:

combine_scorefiles -s PGS001229_22.txt -t GRCh37 -o scorefiles.txt.gz -l log_scorefil
es.json -v

cat <<-END_VERSIONS > versions.yml
COMBINE_SCOREFILES:
pgscatalog_utils: $(echo $(python -c 'import pgscatalog_utils; print(pgscatalog_utils.version)'))
END_VERSIONS

Command exit status:
1

Command output:
(empty)

Command error:
Traceback (most recent call last):
File "/home/jupyter/.local/bin/combine_scorefiles", line 7, in
from pgscatalog_utils.scorefile.combine_scorefiles import combine_scorefiles
ModuleNotFoundError: No module named 'pgscatalog_utils'

Work dir:
/home/jupyter/workspaces/lifetimeriskestimationinmultiancestrybiobanks/work/4b/e32fd63e45a59ef9437ea1125e584b

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run

-- Check '.nextflow.log' file for details
ERROR ~ ERROR: Matching subworkflow failed

-- Check '.nextflow.log' file for details
ERROR ~ ERROR: No scores calculated!

-- Check '.nextflow.log' file for details
ERROR ~ ERROR: No results report written!

-- Check '.nextflow.log' file for details

nebfield · 2024-05-28T10:10:12Z

Could you test with:

$ rm -rf ~/.nextflow/assets
$ rm -r work 
$ nextflow run pgscatalog/pgsc_calc -profile test,conda -r v2.0.0-alpha.5

This should reset your installation. We did a new release last Friday but found some problems over the weekend, so we've withdrawn it, which might have caused some problems for you.

pecko-g · 2024-05-28T10:34:59Z

I got the same error again.

nebfield · 2024-05-28T14:23:07Z

I'm not sure how to fix your local environment, sorry. The workflow runs OK with the conda profile in our test suite. You should never need to download or install any environments manually (e.g. you won't need to install pgscatalog_utils)

Singularity or docker are the best way to deploy the pipeline - we only use conda as a backup. Is it possible you could get support from your sysadmins to work with singularity?

pecko-g · 2024-05-29T08:09:19Z

Thank you so much for your help still! I will try to get them to install singularity.

Best wishes

nebfield · 2024-05-30T09:11:41Z

I had an idea to create a big conda environment containing every dependency, but it should only be tried as a last resort:

#302 (comment)

Please let me know if this approach works for you 😅

pecko-g · 2024-06-03T11:28:32Z

Thank you for your suggestion, I read shortly into that discussion and it should be possible to make it work, since I'm also working with the AllofUs cloud environment.

Before I try that though, I was sent this page by the support team of AoU: https://support.researchallofus.org/hc/en-us/articles/21179878475028-Using-Docker-Images-on-the-Workbench

And there's this short paragraph about using a docker image with Nextflow, would there be a way to adapt it to pgsc calc?

Thank you again!

nebfield · 2024-06-19T19:08:14Z

@pecko-g could you try the conda profile with the latest release please. Sorry for the delay! 😄

https://pgsc-calc.readthedocs.io/en/latest/how-to/change_version.html

pecko-g · 2024-06-20T08:55:00Z

The test profile works now! Thank you so much! 👏🏼

pecko-g added the bug Something isn't working label May 17, 2024

smlmbrt assigned nebfield May 23, 2024

nebfield changed the title ~~Module path must start with / or ./ prefix -- Offending module: plugin/nf-validation~~ plugin/nf-validation error with old version of Nextflow May 24, 2024

nebfield removed the bug Something isn't working label May 24, 2024

Carldeboer mentioned this issue Jun 12, 2024

nf-validation cannot find the manifest path #312

Closed

pecko-g closed this as completed Jun 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

plugin/nf-validation error with old version of Nextflow #293

plugin/nf-validation error with old version of Nextflow #293

pecko-g commented May 17, 2024

smlmbrt commented May 24, 2024

nebfield commented May 24, 2024

pecko-g commented May 27, 2024 •

edited

Loading

nebfield commented May 28, 2024 •

edited

Loading

pecko-g commented May 28, 2024

nebfield commented May 28, 2024

pecko-g commented May 28, 2024

nebfield commented May 28, 2024 •

edited

Loading

pecko-g commented May 29, 2024

nebfield commented May 30, 2024

pecko-g commented Jun 3, 2024

nebfield commented Jun 19, 2024 •

edited

Loading

pecko-g commented Jun 20, 2024

plugin/nf-validation error with old version of Nextflow #293

plugin/nf-validation error with old version of Nextflow #293

Comments

pecko-g commented May 17, 2024

Description of the bug

Command used and terminal output

Relevant files

System information

smlmbrt commented May 24, 2024

nebfield commented May 24, 2024

pecko-g commented May 27, 2024 • edited Loading

nebfield commented May 28, 2024 • edited Loading

pecko-g commented May 28, 2024

nebfield commented May 28, 2024

pecko-g commented May 28, 2024

nebfield commented May 28, 2024 • edited Loading

pecko-g commented May 29, 2024

nebfield commented May 30, 2024

pecko-g commented Jun 3, 2024

nebfield commented Jun 19, 2024 • edited Loading

pecko-g commented Jun 20, 2024

pecko-g commented May 27, 2024 •

edited

Loading

nebfield commented May 28, 2024 •

edited

Loading

nebfield commented May 28, 2024 •

edited

Loading

nebfield commented Jun 19, 2024 •

edited

Loading