Introduce approvaltest module #4112
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
💔 Tests Failed
Expand to view the summary
Build stats
Test stats 🧪
Test errorsExpand to view the tests failures
Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
axw
force-pushed
the
approvaltest
branch
2 times, most recently
from
22fdef6
to
875f630
Compare
We move tests/approvals and tests/scripts/approvals.go intoa new module, github.com/elastic/apm-server/approvaltest. This new module will be used by both apm-server (integration tests) and the systemtest module. We also reduce the API surface area to two functions: ApproveJSON, and ApproveEventDocs. The ApproveJSON function can be used for approving arbitrary JSON-encoded values, while ApproveEventDocs is specifically intended for approving JSON-encoded Elasticsearch event documents. To enable the API reduction, we introduce beatertest.EncodeEventDoc(s), which encode events using the libbeat Elasticsearch output. In some tests we previously used the libbeat JSON codec directly, which led to @metadata being included in diffs. The output removes @metadata, hence it no longer features in our approval diffs.
- Since we now use the Elasticsearch output, rather than directly
using the libbeat JSON codec, `@metadata` is no longer part of the
diff, and `@timestamp` is encoded slightly differently.
- For dynamic fields like `profile.id`, we now replace the value
with a known string ("dynamic") rather than ignoring the field.
simitt
approved these changes
Aug 31, 2020
jalvz
pushed a commit
to jalvz/apm-server
that referenced
this pull request
Sep 4, 2020
* Introduce new approvaltest module We move tests/approvals and tests/scripts/approvals.go intoa new module, github.com/elastic/apm-server/approvaltest. This new module will be used by both apm-server (integration tests) and the systemtest module. We also reduce the API surface area to two functions: ApproveJSON, and ApproveEventDocs. The ApproveJSON function can be used for approving arbitrary JSON-encoded values, while ApproveEventDocs is specifically intended for approving JSON-encoded Elasticsearch event documents. To enable the API reduction, we introduce beatertest.EncodeEventDoc(s), which encode events using the libbeat Elasticsearch output. In some tests we previously used the libbeat JSON codec directly, which led to @metadata being included in diffs. The output removes @metadata, hence it no longer features in our approval diffs. * Update approved files - Since we now use the Elasticsearch output, rather than directly using the libbeat JSON codec, `@metadata` is no longer part of the diff, and `@timestamp` is encoded slightly differently. - For dynamic fields like `profile.id`, we now replace the value with a known string ("dynamic") rather than ignoring the field. * .ci/docker/golang-mage: support multiple modules
axw
added a commit
to axw/apm-server
that referenced
this pull request
Sep 7, 2020
* Introduce new approvaltest module We move tests/approvals and tests/scripts/approvals.go intoa new module, github.com/elastic/apm-server/approvaltest. This new module will be used by both apm-server (integration tests) and the systemtest module. We also reduce the API surface area to two functions: ApproveJSON, and ApproveEventDocs. The ApproveJSON function can be used for approving arbitrary JSON-encoded values, while ApproveEventDocs is specifically intended for approving JSON-encoded Elasticsearch event documents. To enable the API reduction, we introduce beatertest.EncodeEventDoc(s), which encode events using the libbeat Elasticsearch output. In some tests we previously used the libbeat JSON codec directly, which led to @metadata being included in diffs. The output removes @metadata, hence it no longer features in our approval diffs. * Update approved files - Since we now use the Elasticsearch output, rather than directly using the libbeat JSON codec, `@metadata` is no longer part of the diff, and `@timestamp` is encoded slightly differently. - For dynamic fields like `profile.id`, we now replace the value with a known string ("dynamic") rather than ignoring the field. * .ci/docker/golang-mage: support multiple modules
axw
added a commit
that referenced
this pull request
Sep 8, 2020
* Introduce approvaltest module (#4112) * Introduce new approvaltest module We move tests/approvals and tests/scripts/approvals.go intoa new module, github.com/elastic/apm-server/approvaltest. This new module will be used by both apm-server (integration tests) and the systemtest module. We also reduce the API surface area to two functions: ApproveJSON, and ApproveEventDocs. The ApproveJSON function can be used for approving arbitrary JSON-encoded values, while ApproveEventDocs is specifically intended for approving JSON-encoded Elasticsearch event documents. To enable the API reduction, we introduce beatertest.EncodeEventDoc(s), which encode events using the libbeat Elasticsearch output. In some tests we previously used the libbeat JSON codec directly, which led to @metadata being included in diffs. The output removes @metadata, hence it no longer features in our approval diffs. * Update approved files - Since we now use the Elasticsearch output, rather than directly using the libbeat JSON codec, `@metadata` is no longer part of the diff, and `@timestamp` is encoded slightly differently. - For dynamic fields like `profile.id`, we now replace the value with a known string ("dynamic") rather than ignoring the field. * .ci/docker/golang-mage: support multiple modules * tests: fix docker-system-tests (#4123) Copy in approvaltest/go.mod (and systemtest/go.mod for good measure) to fix `go mod download` at the top level. Also run `go mod download` for the sub-modules, to speed up repeat runs of docker-system-tests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation/summary
Move tests/approvals and tests/scripts/approvals.go into a new module,
github.com/elastic/apm-server/approvaltest. This new module will be used
by both apm-server (in integration tests) and the systemtest module,
enabling us to have those two modules not import from one another.
We also reduce the approvals API surface area to two functions: ApproveJSON,
and ApproveEventDocs. The ApproveJSON function can be used for approving
arbitrary JSON-encoded values, while ApproveEventDocs is specifically intended
for approving JSON-encoded Elasticsearch event documents.
To enable the API reduction, we introduce beatertest.EncodeEventDoc(s), which
encode events using the libbeat Elasticsearch output. In some tests we previously
used the libbeat JSON codec directly, which led to
@metadatabeing included indiffs. The output removes
@metadata, hence it no longer features in our approvaldiffs.
For dynamic fields like
profile.id, we now replace the value with a known string("dynamic") rather than ignoring the field completely. This minimises the diffs of
.approved.jsonfiles, and also allows us to detect added/removed fields.Checklist
- [ ] I have updated CHANGELOG.asciidocI have considered changes for:
- [ ] documentation- [ ] logging (add log lines, choose appropriate log selector, etc.)- [ ] metrics and monitoring (create issue for Kibana team to add metrics to visualizations, e.g. Kibana#44001)- [ ] telemetry- [ ] Elasticsearch Service (https://cloud.elastic.co)- [ ] Elastic Cloud Enterprise (https://www.elastic.co/products/ece)- [ ] Elastic Cloud on Kubernetes (https://www.elastic.co/elastic-cloud-kubernetes)How to test these changes
make testRelated issues
See #4084 (comment)