feat: add golden testing #262
Conversation
test: omit gapic-combo test: include gapic-combo again in goldens
|
|
||
| out_dir = build_dir / "html/docfx_yaml" | ||
|
|
||
| # Install dependencies. |
There was a problem hiding this comment.
Why aren't these installed normally? It would surprise me if a unit test installed something when I ran this locally.
There was a problem hiding this comment.
This mimics how Sphinx works in the repositories as well as for the plugin repo. It must have the packages installed before generating documentation. I could put this in tox.ini or in requirements.txt I think, but it'd be test data specific and thought it'd be better to keep it generic based on the given package.
Recommonmark is required in order to run sphinx-build with the extension. It seems that it's required within docuploader's shell command, so either I needed to add it here or add it in docuploader. Since this is specific to this repository's usage I found it more appropriate to install it here locally for test usage.
There was a problem hiding this comment.
Any time you use sphinx-docfx-yaml, you'll need Recommonmark, right? So, it seems reasonable to include it as a dependency, even if it's only directly used by this repo for tests? I may be misunderstanding.
Do you have a link to how docuploader's shell requires it (what is it?)?
There was a problem hiding this comment.
Not quite - it's used for the extension but not directly in sphinx-docfx-yaml. That's why I included it as dependencies on each doc generation script (example: https://github.com/googleapis/python-logging/blob/77e621cb85b726a801227da85c31319f25969d19/noxfile.py#L312). When I tried including this in requirements.txt or tox.ini as part of dependency, docuploader.shell has trouble finding recommonmark required to run sphinx-build command and thus that's why I included it as a dependency.
tests/test_goldens.py
Outdated
| # Settings to be used for sphinx-build | ||
| "-D", | ||
| # Extensions to be added | ||
| ("extensions=sphinx.ext.autodoc,sphinx.ext.autosummary,docfx_yaml.extension," + |
There was a problem hiding this comment.
I don't think the + is needed at the end?
There was a problem hiding this comment.
Done! Did a bit of formatting as well.
tests/test_goldens.py
Outdated
|
|
||
| if update_goldens: | ||
| shutil.rmtree(golden_dir, ignore_errors=True) | ||
| files_to_move = [file for file in out_dir.rglob("*")] |
There was a problem hiding this comment.
No need! Updated.
|
Please take a look again! |
| # Place the Overview page at the top of the list. | ||
| app.env.markdown_pages.insert( | ||
| 0, | ||
| index_page_entry, |
There was a problem hiding this comment.
What if there is no index page entry?
There was a problem hiding this comment.
Hopefully that's not the case - that means that the README page from GitHub is missing. However, I've added an if clause to check and return early if not found.
|
|
||
| out_dir = build_dir / "html/docfx_yaml" | ||
|
|
||
| # Install dependencies. |
There was a problem hiding this comment.
Any time you use sphinx-docfx-yaml, you'll need Recommonmark, right? So, it seems reasonable to include it as a dependency, even if it's only directly used by this repo for tests? I may be misunderstanding.
Do you have a link to how docuploader's shell requires it (what is it?)?
Adds goldens that I finally wanted to add!
Grabbed one of each so that I can easily take a look at the updated values across most libraries.
Removed a bunch of old artifacts for tests and on top directory. If it's distracting, I'll move it to another PR.
Unified tox unit test to a single
tests. Addedupdate-goldenssession to be used locally if needed.3.7 and 3.8 tests on GitHub Action seem to be causing weird discrepancies that I'm not sure why it's happening - sometimes it confuses "starting Line" and on other times just makes pure formatting issues that I'm not sure why it happens on GH actions. I was able to force this with 3.9 with sorting the TOC. I'll look into enabling it for 3.7 and 3.8 if needed.
Fixes #44