Not all definitions in JSON schema have a "properties", leading to an error.

[mahesh-panchal]

Description of the bug

Relevant code:

tools/nf_core/pipeline-template/lib/NfcoreSchema.groovy

Lines 390 to 402 in a1bf2ad

	raw_schema.definitions.each { definition ->
	for (key in definition.keySet()){
	if (definition[key].get("properties").keySet().contains(ignore_param)){
	// Remove the param to ignore
	definition[key].get("properties").remove(ignore_param)
	// If the param was required, change this
	if (definition[key].has("required")) {
	def cleaned_required = removeElement(definition[key].required, ignore_param)
	definition[key].put("required", cleaned_required)
	}
	}
	}
	}

Nextflow prints an error when including an nf-core workflow as a subworkflow and the schema doesn't match.

N E X T F L O W  ~  version 21.10.6
Launching `main.nf` [nasty_noyce] - revision: 31a6c995e0
JSONObject["properties"] not found.

When using nf-core schema build to make the JSON schema, not all entries have the "properties" key, but the code expects it to be there. I'm not sure if this an issue for the builder or for the checker.

Command used and terminal output

Error:

$ nextflow run main.nf -profile docker
Picked up JAVA_TOOL_OPTIONS:  -Xmx3435m
N E X T F L O W  ~  version 21.10.6
Launching `main.nf` [nasty_noyce] - revision: 31a6c995e0
JSONObject["properties"] not found.

 -- Check script 'subworkflows/fetchngs/main.nf' at line: 20 or see '.nextflow.log' file for more details

Relevant part of the JSON Schema (built using nf-core schema build)

{
    "$schema": "http://json-schema.org/draft-07/schema",
    "$id": "https://raw.githubusercontent.com/nf-core/fetchngs/master/nextflow_schema.json",
    "title": "nf-core/fetchngs pipeline parameters",
    "description": "Pipeline to fetch metadata and raw FastQ files from public databases",
    "type": "object",
    "definitions": {
        "input_output_options": {
            "title": "Input/output options",
            "type": "object",
            "fa_icon": "fas fa-terminal",
            "description": "Define where the pipeline should find input data and save output data.",
            "required": [
                "input",
                "outdir"
            ],
            "properties": {
                "input": {
                    "type": "string",
                    "format": "file-path",
                    "mimetype": "text/csv",
                    "pattern": "^\\S+\\.txt$",
                    "schema": "assets/schema_input.json",
                    "description": "Path to text file containing information about the samples in the experiment.",
                    "help_text": "You will need to create a design file with information about the samples in your experiment before running the pipeline. Use this parameter to specify its location. It has to be a comma-separated file with 3 columns, and a header row. See [usage docs](https://nf-co.re//usage#samplesheet-input).",
                    "fa_icon": "fas fa-file-csv"
                },
                "outdir": {
                    "type": "string",
                    "description": "Path to the output directory where the results will be saved.",
                    "default": "./results",
                    "fa_icon": "fas fa-folder-open"
                },
                "email": {
                    "type": "string",
                    "description": "Email address for completion summary.",
                    "fa_icon": "fas fa-envelope",
                    "help_text": "Set this parameter to your e-mail address to get a summary e-mail with details of the run sent to you when the workflow exits. If set in your user config file (`~/.nextflow/config`) then you don't need to specify this on the command line for every run.",
                    "pattern": "^([a-zA-Z0-9_\\-\\.]+)@([a-zA-Z0-9_\\-\\.]+)\\.([a-zA-Z]{2,5})$"
                }
            }
        },
        "reference_genome_options": {
            "title": "Reference genome options",
            "type": "object",
            "fa_icon": "fas fa-dna",
            "description": "Reference genome related files and options required for the workflow."
        },

reference_genome_options has no properties tag.

[mashehu]

Hmm, probably a problem of the builder. Did you create this schema by (re)moving all of the parameters from the reference_genome_options group?

[mahesh-panchal]

It was the default setting after running nf-core schema build and opening in the browser. I didn't do anything to it.

[mashehu]

which pipeline?

[mahesh-panchal]

https://github.com/mahesh-panchal/test_nfcore_workflow_chain

[mashehu]

Sorry, I can't reproduce your error (cloned your repo and ran the test profile). I don't see reference_genome_options in the schema of the linked repo.

[mahesh-panchal]

The workflow works at the moment. I got rid of the "reference_genome_options" which was causing the error using the builder. To be honest, I'm not even sure why the "reference_genome_options" is there, but I assume because it pulled it from some subworkflow.

But to recreate the issue, delete the nextflow_schema.json and then run nf-core schema build. That should recreate it.

[mashehu]

Okay, I think I found the problem. When building a schema it takes a skeleton schema from the template (which contains the reference_genome_options group containing fasta etc.) and then removes the unneeded parameters, but not the now empty group. This step should probably be part of the remove_schema_notfound_configs() function in schema.py.

[ewels]

@mashehu sounds like you've figured this one out - would you mind putting in a PR to fix it please?

[drpatelh]

To reproduce run:

1. nf-core create to create a fresh pipeline from the template
2. Delete these 3 lines in the nextflow.config
3. Delete this line in main.nf
4. Run rm nextflow_schema.json && nf-core schema build

This should update the scheme to remove all of the params in the Reference genome options group but will still leave the group in the schema without any properties which is causing the error.

[ewels]

You can also just create a new group in the web builder and not add any parameters to it. That has the same effect of leaving a dangling definition with no properties.

Edit: Not quite the same as when you do that there is no properties key at all, rather than a properties key with an empty dict.