-
Notifications
You must be signed in to change notification settings - Fork 39
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
testament: support per-target
knownIssue
s (#834)
## Summary Add support for per-target `knownIssue` specification to testament. Previously, only the full test (including all targets) could be marked as a "known issue". To emulate per-target `knownIssue`s, tests had to explicitly disable the target for which the test was failing and add a separate comment, but this has the issue of `--tryfailing` not applying. The `knownIssue` key in the test specifications now supports a *sub-key*, like so: `knownIssue.target`, where `target` is one of `c`, `js`, or `vm`. A `knownIssue` key without a sub-key means "applies to all targets", and specifying multiple targets works by separating them via spaces. ## Details At the core of the changes is that `knownIssues` are now per-target instead of per-spec. The spec parser already supports dots in keys, so the only addition there is splitting the parsed key string into the main and sub part prior to processing the key. Since "has known issue" is now a per-run attribute instead of a per-spec one, disabling a test early by returning `reKnownIssue` from `computeEarly` cannot work anymore -- everything related to whole-spec known issues is removed. Instead, test-run descriptions are now always generated, but then later skipped by `run` (which now accepts a `runKnownIssues` parameter) if they have known issues and trying those is disabled. Since skipped "known issue" test runs would have no associated "known issue" information in the backend otherwise, `addResult` for `TestRun` now always includes the runs "known issues" information. Tests using the disable + comment approach are adjusted to use per- target `knownIssue`s, but without checking whether the issue is still current. --------- Co-authored-by: Saem Ghani <saemghani+github@gmail.com>
- Loading branch information
Showing
25 changed files
with
119 additions
and
104 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
8 changes: 4 additions & 4 deletions
8
tests/lang_callable/closure/tinfer_closure_for_nestedproc.nim
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.