✨ Add core testing mode to help test SDKs #984
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently, the majority of our SDKs (JavaScript SDKs) have access to sdk-utils' test server that mocks the core API. This is the primary method of testing these SDKs since their implementations use sdk-utils internally anyway. For other SDKs, we implement the required utils on a per-language basis, including how they're tested. Testing typically involves mocking HTTP modules or creating stub APIs to simulate the CLI exec command running. With minimal changes to our existing core API, we can add a testing mode. This new testing mode accomplishes two main tasks. First is to silence all logs and enable dry-run mode to disable uploads and asset discovery. Second is to enable a few extra API endpoints to allow all SDKs to be more easily tested. New API endpoints include access to raw logs, a simple HTML document to snapshot, and a set of commands to manipulate how the API responds to test various circumstances such as errors, disconnects, or missing/invalid core version information. Other changes made include a small adjustment to client's request util to allow `false` as a JSON body, and updated percy-css tests which were failing from unrelated, accidentally unstashed, changes.
wwilsman
force-pushed
the
ww/cli-testing-mode
branch
from
197d792 to
951ce54
Compare
Robdel12
approved these changes
Jul 5, 2022
kodiakhq bot
referenced
this pull request
in carbon-design-system/carbon-for-ibm-dotcom
Jul 13, 2022
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [@percy/cli](https://github.com/percy/cli) | [`~1.5.0` -> `~1.6.0`](https://renovatebot.com/diffs/npm/@percy%2fcli/1.5.1/1.6.0) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [@percy/dom](https://github.com/percy/cli) | [`~1.5.0` -> `~1.6.0`](https://renovatebot.com/diffs/npm/@percy%2fdom/1.5.1/1.6.0) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | | [@percy/sdk-utils](https://github.com/percy/cli) | [`~1.5.0` -> `~1.6.0`](https://renovatebot.com/diffs/npm/@percy%2fsdk-utils/1.5.1/1.6.0) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>percy/cli</summary> ### [`v1.6.0`](https://github.com/percy/cli/releases/tag/v1.6.0) [Compare Source](https://github.com/percy/cli/compare/v1.5.1...v1.6.0) <!-- Release notes generated using configuration in .github/release.yml at master --> #### What's Changed ##### ✨ Enhancements - ✨ Add core testing mode to help test SDKs by [@​wwilsman](https://github.com/wwilsman) in [https://github.com/percy/cli/pull/984](https://github.com/percy/cli/pull/984) - ✨ Allow PascalCase options (and other casing) by [@​wwilsman](https://github.com/wwilsman) in [https://github.com/percy/cli/pull/985](https://github.com/percy/cli/pull/985) ##### ⬆️⬇️ Dependency Updates - ⬆️ Bump [@​babel/cli](https://github.com/babel/cli) from 7.17.10 to 7.18.6 by [@​dependabot](https://github.com/dependabot) in [https://github.com/percy/cli/pull/981](https://github.com/percy/cli/pull/981) - ⬆️ Bump [@​babel/register](https://github.com/babel/register) from 7.17.7 to 7.18.6 by [@​dependabot](https://github.com/dependabot) in [https://github.com/percy/cli/pull/979](https://github.com/percy/cli/pull/979) - ⬆️ Bump [@​babel/core](https://github.com/babel/core) from 7.18.5 to 7.18.6 by [@​dependabot](https://github.com/dependabot) in [https://github.com/percy/cli/pull/980](https://github.com/percy/cli/pull/980) - ⬆️ Bump tsd from 0.21.0 to 0.22.0 by [@​dependabot](https://github.com/dependabot) in [https://github.com/percy/cli/pull/983](https://github.com/percy/cli/pull/983) - ⬆️ Bump [@​babel/preset-env](https://github.com/babel/preset-env) from 7.18.2 to 7.18.6 by [@​dependabot](https://github.com/dependabot) in [https://github.com/percy/cli/pull/982](https://github.com/percy/cli/pull/982) **Full Changelog**: percy/cli@v1.5.1...v1.6.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "every weekend" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/carbon-design-system/carbon-for-ibm-dotcom).
](https://renovatebot.com){kind=link}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is this?
Currently, the majority of our SDKs (JavaScript SDKs) have access to sdk-utils' test server that mocks the core API. This is the primary method of testing these SDKs since their implementations use sdk-utils internally anyway. For other SDKs, we implement the required utils on a per-language basis, including how they're tested. Testing typically involves mocking HTTP modules or creating stub APIs to simulate the CLI exec command running.
With minimal changes to our existing core API, we can add a testing mode. This new testing mode accomplishes two main tasks. First is to silence all logs and enable dry-run mode to disable uploads and asset discovery. Second is to enable a few extra API endpoints to allow all SDKs to be more easily tested. New API endpoints include access to raw logs, a simple HTML document to snapshot, and a set of commands to manipulate how the API responds to test various circumstances such as errors, disconnects, or missing/invalid core version information.
For now, this new testing mode will be used with the upcoming .NET SDK. In the future however, we should update all of our other tests across all languages and eventually remove the old test server from sdk-utils.
Other changes made include a small adjustment to client's request util to allow
falseas a JSON body, and updated percy-css tests which were failing for me locally from accidentally unstashed, unrelated, changes.