setting sub-params from command line fails when following params block has self-references

[TonyWildish-ETX]

Bug report

Expected behavior and actual behavior

I have a config file with two params blocks. The first block contains nested objects, the other contains a self-reference (see code example below). When run with no extra command line parameters, the params are all set as expected. When run with a command line parameter to set one of the nested object values, the other nested values are lost.

If I reverse the order of the two param blocks, it works correctly again.

Steps to reproduce the problem

# cat nextflow.config 

params {
    module_2 {
      flag_1 = 3
      flag_2 = 4
    }
}

params {
    outdir = "./results"
    tracedir = "${params.outdir}/pipeline_info"
}

# cat main.nf 
nextflow.enable.dsl=2

log.info """
  params: ${params}
  """

Running nextflow with no command line parameters sets the params block as expected. the module_2 subobject has both flag_1 and flag_2 set.

# nextflow run main.nf
N E X T F L O W  ~  version 21.04.0-edge
Launching main.nf [drunk_ramanujan] - revision: c62c740cf9

  params: [module_2:[flag_1:3, flag_2:4], outdir:./results, tracedir:./results/pipeline_info]

Attempting to set one of the nested param objects from the command line succeeds, but the other flag is lost:

# nextflow run main.nf --module_2.flag_1 99
N E X T F L O W  ~  version 21.04.0-edge
Launching main.nf [soggy_keller] - revision: c62c740cf9

  params: [module_2:[flag_1:99], outdir:./results, tracedir:./results/pipeline_info]

Note that module_2.flag_1 has been set, but module_2.flag_2 has been lost completely.

If I swap the order of the two param blocks, the problem goes away. If the params blocks are left in the order above, but the self-reference to ${params.outdir} is removed, the problem goes away. I.e. the problem only happens when the second block has a self reference and a sub-object in the first block is set via the command line.

Program output

Environment

• Nextflow version: 21.04.0-edge

• Java version:
  openjdk version "11.0.10" 2021-01-19 OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.10+9) OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.10+9, mixed mode)

• Operating system: macOS 11.6 (Big Sur)

• Bash version: GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin20)

Additional context

This was also seen by a colleague using Windows.

[pditommaso]

Solved by #2485. It will be included in next release. Thanks for reporting this problem.