caper.nb_subproc_thread|ERROR| Cromwell failed. returncode=1

[ritetsu]

Describe the bug

While running nohup caper run chip.wdl -i https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI_subsampled_chr19_only.json --conda &
I got error as: Conda environment name not found in WDL metadata. and caper.nb_subproc_thread|ERROR| Cromwell failed. returncode=1 and a cromwell.out with nothing in it.

2023-02-23 10:45:50,046|autouri.autouri|INFO| cp: (468c5614) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI/fastq_subsampled/rep1.subsampled.25.fastq.gz, dest=/home/encode-dcc/41d3a8658bf4f565c08290e386fddccf/rep1.subsampled.25.fastq.gz
2023-02-23 10:45:53,986|autouri.autouri|INFO| cp: (468c5614) done.
2023-02-23 10:45:53,987|autouri.autouri|INFO| cp: (1e490fb8) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI/fastq_subsampled/rep2.subsampled.20.fastq.gz, dest=/home/encode-dcc/0d5b102d7f21dd009798ae9b2c318b65/rep2.subsampled.20.fastq.gz
2023-02-23 10:45:57,226|autouri.autouri|INFO| cp: (1e490fb8) done.
2023-02-23 10:45:57,227|autouri.autouri|INFO| cp: (7176279f) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI/fastq_subsampled/ctl1.subsampled.25.fastq.gz, dest=/home/encode-dcc/f4dba75d4430f4702a75554c44265119/ctl1.subsampled.25.fastq.gz
2023-02-23 10:45:59,741|autouri.autouri|INFO| cp: (7176279f) done.
2023-02-23 10:45:59,742|autouri.autouri|INFO| cp: (89ae1f80) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI/fastq_subsampled/ctl2.subsampled.25.fastq.gz, dest=/home/encode-dcc/7a610dd5397c45a806678efaad824abf/ctl2.subsampled.25.fastq.gz
2023-02-23 10:46:02,442|autouri.autouri|INFO| cp: (89ae1f80) done.
2023-02-23 10:46:03,271|caper.caper_workflow_opts|INFO| Conda environment name not found in WDL metadata. wdl=/home/lvwei/chipseq/encode/chip.wdl
2023-02-23 10:46:03,289|caper.cromwell|INFO| Validating WDL/inputs/imports with Womtool...
2023-02-23 10:46:06,100|caper.nb_subproc_thread|INFO| Subprocess finished successfully.
2023-02-23 10:46:06,100|caper.cromwell|INFO| Passed Womtool validation.
2023-02-23 10:46:06,101|caper.caper_runner|INFO| launching run: wdl=/home/lvwei/chipseq/encode/chip.wdl, inputs=/home/encode-dcc/15b4ff439de58d859f4c3ee4482c3bd2/ENCSR000DYI_subsampled_chr19_only.local.json, backend_conf=/home/encode-dcc/chip/20230223_104456_225709/backend.conf
2023-02-23 10:46:06,129|caper.nb_subproc_thread|ERROR| Cromwell failed. returncode=1
2023-02-23 10:46:06,130|caper.cli|ERROR| Check stdout in /home/lvwei/chipseq/encode/cromwell.out.1

I tried to follow issue #271 to remove and re-install caper and conda_env, but still get the same output.

OS/Platform

• OS/Platform: CentOS 7.5.1804
• Conda version: conda 23.1.0
• Pipeline version: v2.2.1
• Caper version: v2.2.3

Caper configuration file

Paste contents of ~/.caper/default.conf.

backend=local

# Local directory for localized files and Cromwell's intermediate files.
# If not defined then Caper will make .caper_tmp/ on CWD or `local-out-dir`.
# /tmp is not recommended since Caper store localized data files here.
local-loc-dir=/home/encode-dcc

cromwell=/home/.caper/cromwell_jar/cromwell-82.jar
womtool=/home/.caper/womtool_jar/womtool-82.jar

Input JSON file

Paste contents of your input JSON file.

{
    "chip.pipeline_type" : "tf",
    "chip.genome_tsv" : "https://storage.googleapis.com/encode-pipeline-genome-data/genome_tsv/v3/hg38_chr19_chrM.tsv",
    "chip.fastqs_rep1_R1" : ["https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI/fastq_subsampled/rep1.subsampled.25.fastq.gz"
    ],
    "chip.fastqs_rep2_R1" : ["https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI/fastq_subsampled/rep2.subsampled.20.fastq.gz"
    ],
    "chip.ctl_fastqs_rep1_R1" : ["https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI/fastq_subsampled/ctl1.subsampled.25.fastq.gz"
    ],
    "chip.ctl_fastqs_rep2_R1" : ["https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI/fastq_subsampled/ctl2.subsampled.25.fastq.gz"
    ],
    "chip.paired_end" : false,
    "chip.title" : "ENCSR000DYI (subsampled 1/25, chr19_chrM only)",
    "chip.description" : "CEBPB ChIP-seq on human A549 produced by the Snyder lab"
}

Troubleshooting result

If you ran caper run without Caper server then Caper automatically runs a troubleshooter for failed workflows. Find troubleshooting result in the bottom of Caper's screen log.

If you ran caper submit with a running Caper server then first find your workflow ID (1st column) with caper list and run caper debug [WORKFLOW_ID].

Paste troubleshooting result.

2023-02-23 11:02:03,383|caper.cli|INFO| Cromwell stdout: /home/lvwei/chipseq/encode/cromwell.out.3
2023-02-23 11:02:03,387|caper.caper_base|INFO| Creating a timestamped temporary directory. /home/encode-dcc/chip/20230223_110203_384959
2023-02-23 11:02:03,387|caper.caper_runner|INFO| Localizing files on work_dir. /home/encode-dcc/chip/20230223_110203_384959
2023-02-23 11:02:04,211|caper.cromwell|INFO| Validating WDL/inputs/imports with Womtool...
2023-02-23 11:02:06,884|caper.nb_subproc_thread|INFO| Subprocess finished successfully.
2023-02-23 11:02:06,885|caper.cromwell|INFO| Passed Womtool validation.
2023-02-23 11:02:06,885|caper.caper_runner|INFO| launching run: wdl=/home/lvwei/chipseq/encode/chip.wdl, inputs=None, backend_conf=/home/encode-dcc/chip/20230223_110203_384959/backend.conf
2023-02-23 11:02:06,911|caper.nb_subproc_thread|ERROR| Cromwell failed. returncode=1
2023-02-23 11:02:06,912|caper.cli|ERROR| Check stdout in /home/lvwei/chipseq/encode/cromwell.out.3

[ritetsu]

Seems to be a caper version issue, I solved this problem by using cromwell-65.jar and womtool-65.jar instead of -82.jar.