Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Negative UI test error annotations #129460

Open
jieyouxu opened this issue Aug 23, 2024 · 3 comments
Open

Negative UI test error annotations #129460

jieyouxu opened this issue Aug 23, 2024 · 3 comments
Labels
A-compiletest Area: The compiletest test runner A-testsuite Area: The testsuite used to check the correctness of rustc C-enhancement Category: An issue proposing an enhancement or a PR with one. E-medium Call for participation: Medium difficulty. Experience needed to fix: Intermediate. E-needs-design This issue needs exploration and design to see how and if we can fix/implement it E-needs-investigation Call for partcipation: This issues needs some investigation to determine current status T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@jieyouxu
Copy link
Member

jieyouxu commented Aug 23, 2024

I wish there is a negative assertion, saying that here one shall not expect a HELP note.

Originally posted by @dingxiangfei2009 in #107251 (comment)


Something that might be nice-to-have is negative ui test error annotations, i.e. that we explicitly want to make sure some diagnostic is not emitted (analogous to FileCheck check-NOT annotations).

Obviously, you can rely on checking the blessed stderr, but a negative ui test annotation could help communicate intent in the test itself and is less prone to accidentally letting the blessed stderr though.

Note: we should probably try to make the directive parser more robust and less hacky before attempting to add more syntax.

@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Aug 23, 2024
@jieyouxu jieyouxu added A-testsuite Area: The testsuite used to check the correctness of rustc C-enhancement Category: An issue proposing an enhancement or a PR with one. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. A-compiletest Area: The compiletest test runner labels Aug 23, 2024
@jieyouxu jieyouxu removed the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Aug 23, 2024
@dingxiangfei2009
Copy link
Contributor

dingxiangfei2009 commented Aug 23, 2024

Let me know if we would like to go ahead with this feature. I am getting my hands dirty enough with compiletest that I feel like I know how to. 😄

@jieyouxu
Copy link
Member Author

jieyouxu commented Aug 23, 2024

@dingxiangfei2009 Thanks for the interest. I would be happy to aid and review this feature, but I feel like we want to make compiletest a bit more maintainable first, see e.g. #128070. I think we should not hack on more syntax for directives and error annotations before we figure out how to make the directive and annotation parser itself a bit more robust, lol.

@fmease
Copy link
Member

fmease commented Aug 29, 2024

Related: #46667 (2017).

@jieyouxu jieyouxu added E-needs-design This issue needs exploration and design to see how and if we can fix/implement it E-needs-investigation Call for partcipation: This issues needs some investigation to determine current status E-medium Call for participation: Medium difficulty. Experience needed to fix: Intermediate. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) and removed T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-compiletest Area: The compiletest test runner A-testsuite Area: The testsuite used to check the correctness of rustc C-enhancement Category: An issue proposing an enhancement or a PR with one. E-medium Call for participation: Medium difficulty. Experience needed to fix: Intermediate. E-needs-design This issue needs exploration and design to see how and if we can fix/implement it E-needs-investigation Call for partcipation: This issues needs some investigation to determine current status T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
Development

No branches or pull requests

4 participants