Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustdoc: Add more test coverage #91113

Open
camelid opened this issue Nov 21, 2021 · 6 comments
Open

rustdoc: Add more test coverage #91113

camelid opened this issue Nov 21, 2021 · 6 comments
Assignees
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.

Comments

@camelid
Copy link
Member

camelid commented Nov 21, 2021

Summary

Currently, rustdoc doesn't have much test coverage, but it does have a lot of edge cases. We would like to have more test coverage, so we can improve the code with less risk of introducing bugs.

It's even helpful to add tests for incorrect behavior since it will help us find, fix, and test the fixes for bugs.

What to test

  • Behavior of re-exports (e.g., pub use self::foo::bar) and inlining (some re-exports are rendered as re-exports, while others have their docs copied as if the re-export is the definition)
  • Trait implementations, including normal impls, blanket impls (e.g., impl<T> MyTrait for &T), and auto impls (like Sized, Send, and Sync)
  • Notable traits popup (appears as an (i) next to return types); it has some bugs currently that would be good to fix.

How to test

Here is the documentation for rustdoc's main tests (the tests in src/test/rustdoc):

rust/src/etc/htmldocck.py

Lines 28 to 100 in 49d4232

# Commands
Commands start with an `@` followed by a command name (letters and
hyphens), and zero or more arguments separated by one or more whitespace
characters and optionally delimited with single or double quotes. The `@`
mark cannot be preceded by a non-whitespace character. Other lines
(including every text up to the first `@`) are ignored, but it is
recommended to avoid the use of `@` in the template file.
There are a number of supported commands:
* `@has PATH` checks for the existence of the given file.
`PATH` is relative to the output directory. It can be given as `-`
which repeats the most recently used `PATH`.
* `@has PATH PATTERN` and `@matches PATH PATTERN` checks for
the occurrence of the given pattern `PATTERN` in the specified file.
Only one occurrence of the pattern is enough.
For `@has`, `PATTERN` is a whitespace-normalized (every consecutive
whitespace being replaced by one single space character) string.
The entire file is also whitespace-normalized including newlines.
For `@matches`, `PATTERN` is a Python-supported regular expression.
The file remains intact but the regexp is matched without the `MULTILINE`
and `IGNORECASE` options. You can still use a prefix `(?m)` or `(?i)`
to override them, and `\A` and `\Z` for definitely matching
the beginning and end of the file.
(The same distinction goes to other variants of these commands.)
* `@has PATH XPATH PATTERN` and `@matches PATH XPATH PATTERN` checks for
the presence of the given XPath `XPATH` in the specified HTML file,
and also the occurrence of the given pattern `PATTERN` in the matching
node or attribute. Only one occurrence of the pattern in the match
is enough.
`PATH` should be a valid and well-formed HTML file. It does *not*
accept arbitrary HTML5; it should have matching open and close tags
and correct entity references at least.
`XPATH` is an XPath expression to match. The XPath is fairly limited:
`tag`, `*`, `.`, `//`, `..`, `[@attr]`, `[@attr='value']`, `[tag]`,
`[POS]` (element located in given `POS`), `[last()-POS]`, `text()`
and `@attr` (both as the last segment) are supported. Some examples:
- `//pre` or `.//pre` matches any element with a name `pre`.
- `//a[@href]` matches any element with an `href` attribute.
- `//*[@class="impl"]//code` matches any element with a name `code`,
which is an ancestor of some element which `class` attr is `impl`.
- `//h1[@class="fqn"]/span[1]/a[last()]/@class` matches a value of
`class` attribute in the last `a` element (can be followed by more
elements that are not `a`) inside the first `span` in the `h1` with
a class of `fqn`. Note that there cannot be any additional elements
between them due to the use of `/` instead of `//`.
Do not try to use non-absolute paths, it won't work due to the flawed
ElementTree implementation. The script rejects them.
For the text matches (i.e. paths not ending with `@attr`), any
subelements are flattened into one string; this is handy for ignoring
highlights for example. If you want to simply check for the presence of
a given node or attribute, use an empty string (`""`) as a `PATTERN`.
* `@count PATH XPATH COUNT' checks for the occurrence of the given XPath
in the specified file. The number of occurrences must match the given
count.
* `@has-dir PATH` checks for the existence of the given directory.
All conditions can be negated with `!`. `@!has foo/type.NoSuch.html`
checks if the given file does not exist, for example.

You may also want to look at some existing tests in src/test/rustdoc.

Please note that you do not need to check that rustdoc's behavior is correct. We will check during code review, and as mentioned earlier, tests for incorrect behavior can be helpful too.

If you have any questions or get stuck, please ask here or on Zulip in #rustdoc!

@camelid camelid added E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. A-testsuite Area: The testsuite used to check the correctness of rustc T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC labels Nov 21, 2021
@camelid
Copy link
Member Author

camelid commented Nov 21, 2021

cc @rust-lang/rustdoc

@cameron1024
Copy link
Contributor

@rustbot claim

@jyn514
Copy link
Member

jyn514 commented Nov 22, 2021

@cameron1024 Since this is a tracking issue, I don't think it makes sense to claim - I plan to add some examples of tests soon, and it's definitely possible for multiple people to work on it in parallel :) feel free to work on it in the meantime though!

JohnTitor added a commit to JohnTitor/rust that referenced this issue Jun 1, 2022
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Jun 2, 2022
…=GuillaumeGomez

rustdoc: Add more test coverage

Related issue rust-lang#91113
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Jun 2, 2022
…=GuillaumeGomez

rustdoc: Add more test coverage

Related issue rust-lang#91113
@NoobProgrammer31
Copy link

@rustbot claim

@ismailarilik
Copy link
Contributor

I think the description should be updated. Tests are in tests/rustdoc now, not in src/test/rustdoc; actually it is not exist anymore.

@ismailarilik
Copy link
Contributor

I was thinking on this issue and just wonder if there is any coverage tool we can use to see the blind points; it might be helpful for this case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

5 participants