html-reporter: urlParams checkboxes broken (hidepassed, noglobals, notrycatch)

[bastimeyer]

Tell us about your runtime:

• QUnit version:
  2.17.1
• Which environment are you using? (e.g., browser, Node):
  NW.js - both: Chromium 93 + Node 16
• How are you running QUnit? (e.g., QUnit CLI, Grunt, Karma, manually in browser):
  manually in NW.js (dev) / custom QUnit reporter/bridge via Chrome Debugging Protocol (CLI / headless)
  with autostart=false and delayed/injected QUnit.start() call

Issue

The hidepassed, noglobals and notrycatch checkboxes are completely broken. Neither the checked state is set correctly, nor does the toggle work properly.

This is especially frustrating when default config values are set.

import QUnit from "qunit";

const { config } = QUnit;
config.autostart = false;
config.hidepassed = true;
config.noglobals = true;
config.notrycatch = false;

do_something_async( () => QUnit.start() );

Observation

QUnit seems to build the urlParams object right at the start when loading the module.
https://github.com/qunitjs/qunit/blob/2.17.1/src/html-reporter/urlparams.js#L12-L14

However, when building the checkboxes in getUrlConfigHtml, the config values are not set yet from the urlParams, so it's always setting the checkboxes' checked state from the default config values:
https://github.com/qunitjs/qunit/blob/2.17.1/src/html-reporter/html.js#L156-L162

That's because the begin callback where the config values are set from the urlParams is run afterwards and too late:
https://github.com/qunitjs/qunit/blob/2.17.1/src/html-reporter/urlparams.js#L56-L72

Also, when deselecting a checkbox and there's no entry in the window.location.search query string, default config values are always applied instead of explicit false values representing the state of the unchecked checkboxes. For hidepassed, this is no problem because it doesn't refresh the page, but for the other urlParams entries it is.

Expected behavior

QUnit should allow setting default config values and should correctly handle the checked state of the urlParams checkboxes, in addition to false values of urlParams entries for being able to toggle the state despite of the default values.

Thanks!

[Krinkle]

Thanks @bastimeyer. It was embarrassing to see this slip through our tests. We have lots of tests in place that verify that config options set at run-time (such as testId and filter) work as expected, as well as numerous end-to-end test that verify other config options registered by plugins correctly go from URL parameter to actual behaviour.

What we lacked was an end-to-end test for the three most important config options (those in the UI), and to simply check that the UI displays their current values correctly.

Expected behaviour:
QUnit should allow setting default config values

I understand the use case and I'd like to explore this in a separate issue. My concern here is backward-compatibility as this is not something we have supported in the past and is potentially incompatible with some of the callback features for plugins and configuration. My understanding is that most use of config at runtime is specifically to override the defaults so we may have to come up with a different way to support this. For example, something like QUnit.config.setDefaults().