Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove timeseries.instance #9565

Merged
merged 2 commits into from
Nov 15, 2022
Merged

Remove timeseries.instance #9565

merged 2 commits into from
Nov 15, 2022

Conversation

axw
Copy link
Member

@axw axw commented Nov 15, 2022

Motivation/summary

Remove this field as we are not using it, and we can use _tsid instead when we migrate to TSDS.

Checklist

How to test these changes

  1. Ensure transaction metrics are recorded
  2. Check that timeseries.instance is not indexed

Related issues

Closes #9018

@axw axw added the v8.6.0 label Nov 15, 2022
@axw axw force-pushed the remove-timeseries-instance branch from 10dba5e to 3ff4cb3 Compare November 15, 2022 07:34
@mergify
Copy link
Contributor

mergify bot commented Nov 15, 2022

This pull request does not have a backport label. Could you fix it @axw? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-7.x is the label to automatically backport to the 7.x branch.
  • backport-7./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Nov 15, 2022
@axw axw marked this pull request as ready for review November 15, 2022 07:36
@axw axw requested a review from a team November 15, 2022 07:36
@apmmachine
Copy link
Contributor

apmmachine commented Nov 15, 2022 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-11-15T12:00:04.643+0000

  • Duration: 28 min 24 sec

Test stats 🧪

Test Results
Failed 0
Passed 153
Skipped 0
Total 153

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate and publish the docker images.

  • /test windows : Build & tests on Windows.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@apmmachine
Copy link
Contributor

apmmachine commented Nov 15, 2022 •
edited
Loading

📚 Go benchmark report

Diff with the main branch

name                                                                                              old time/op    new time/op     delta
pkg:github.com/elastic/apm-server/internal/agentcfg goos:linux goarch:amd64
FetchAndAdd/FetchFromCache-12                                                                       41.2ns ± 0%     46.2ns ± 0%   +12.22%  (p=0.008 n=5+5)
FetchAndAdd/FetchAndAddToCache-12                                                                   91.8ns ± 3%    104.1ns ± 1%   +13.37%  (p=0.008 n=5+5)
pkg:github.com/elastic/apm-server/internal/beater/request goos:linux goarch:amd64
ContextResetContentEncoding/empty-12                                                                 118ns ± 1%      132ns ± 1%   +12.24%  (p=0.008 n=5+5)
ContextResetContentEncoding/uncompressed-12                                                          140ns ± 1%      160ns ± 5%   +14.21%  (p=0.008 n=5+5)
pkg:github.com/elastic/apm-server/internal/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/processor/stream goos:linux goarch:amd64
BackendProcessor/invalid-metadata.ndjson-12                                                         7.91µs ±11%     6.29µs ±21%   -20.47%  (p=0.016 n=5+5)
BackendProcessor/logs.ndjson-12                                                                     78.6µs ±21%     58.4µs ±32%   -25.74%  (p=0.032 n=5+5)
BackendProcessor/minimal-service.ndjson-12                                                          15.4µs ±16%     12.6µs ±14%   -18.21%  (p=0.032 n=5+5)
BackendProcessor/otel-bridge.ndjson-12                                                              41.6µs ± 9%     32.0µs ±29%   -23.09%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/invalid-json-event.ndjson-12            3.74µs ± 4%     4.12µs ± 6%   +10.15%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/invalid-metadata.ndjson-12              1.58µs ± 1%     1.91µs ± 9%   +20.84%  (p=0.016 n=4+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/metadata.ndjson-12                      4.36µs ± 4%     4.74µs ±12%    +8.81%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/minimal-service.ndjson-12               3.32µs ± 4%     3.63µs ± 6%    +9.48%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/optional-timestamps.ndjson-12           4.86µs ± 4%     5.27µs ±13%    +8.38%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/spans.ndjson-12                         32.1µs ±11%     34.7µs ± 9%    +8.20%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/transactions.ndjson-12                  32.6µs ± 8%     39.9µs ±20%   +22.32%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/invalid-metadata.ndjson-12              1.24µs ±10%     1.39µs ± 7%   +12.13%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/metadata.ndjson-12                      2.83µs ± 4%     3.35µs ±11%   +18.45%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/metricsets.ndjson-12                    4.99µs ± 1%     5.08µs ± 2%    +1.74%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions.ndjson-12                  11.7µs ± 1%     11.4µs ± 2%    -2.61%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions_spans_rum.ndjson-12        2.14µs ± 2%     2.07µs ± 1%    -3.43%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/heavy.ndjson-12                        536µs ± 3%      504µs ± 2%    -5.99%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-json-event.ndjson-12          1.14µs ± 1%     1.10µs ± 0%    -3.19%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/minimal-service.ndjson-12             1.01µs ± 1%     0.98µs ± 2%    -2.90%  (p=0.008 n=5+5)
ReadBatch/errors_2.ndjson-12                                                                         109µs ±24%       82µs ±17%   -24.59%  (p=0.032 n=5+5)
ReadBatch/metadata-null-values.ndjson-12                                                            9.59µs ±26%     7.56µs ±27%   -21.21%  (p=0.032 n=5+5)
ReadBatch/transactions_spans_rum_2.ndjson-12                                                        19.6µs ±11%     16.0µs ±21%   -17.98%  (p=0.016 n=5+5)
pkg:github.com/elastic/apm-server/internal/publish goos:linux goarch:amd64
Publisher-12                                                                                         1.00s ± 0%      1.00s ± 0%    -0.09%  (p=0.032 n=5+5)
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage goos:linux goarch:amd64
WriteTransaction/json_codec-12                                                                      4.48µs ±20%     5.67µs ±11%   +26.58%  (p=0.032 n=5+5)
WriteTransaction/json_codec_big_tx-12                                                               5.03µs ± 6%    10.46µs ±54%  +108.04%  (p=0.008 n=5+5)
ReadEvents/json_codec/0_events-12                                                                    301ns ± 4%      348ns ± 8%   +15.94%  (p=0.008 n=5+5)
ReadEvents/json_codec/1_events-12                                                                   9.67µs ± 7%    10.46µs ± 4%    +8.14%  (p=0.008 n=5+5)
ReadEvents/json_codec_big_tx/0_events-12                                                             306ns ± 5%      343ns ± 7%   +12.17%  (p=0.008 n=5+5)
ReadEvents/nop_codec/0_events-12                                                                     307ns ± 5%      339ns ± 4%   +10.62%  (p=0.008 n=5+5)
ReadEvents/nop_codec_big_tx/0_events-12                                                              299ns ± 5%      344ns ± 6%   +15.00%  (p=0.008 n=5+5)
IsTraceSampled/sampled-12                                                                           69.4ns ± 4%     75.3ns ± 3%    +8.39%  (p=0.008 n=5+5)
IsTraceSampled/unsampled-12                                                                         72.0ns ± 2%     77.2ns ± 2%    +7.25%  (p=0.008 n=5+5)
IsTraceSampled/unknown-12                                                                            371ns ± 1%      421ns ± 1%   +13.45%  (p=0.008 n=5+5)

name                                                                                              old alloc/op   new alloc/op    delta
pkg:github.com/elastic/apm-server/internal/agentcfg goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/beater/request goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/processor/stream goos:linux goarch:amd64
BackendProcessor/heavy.ndjson-12                                                                     856kB ± 1%      845kB ± 1%    -1.18%  (p=0.032 n=5+5)
BackendProcessor/invalid-event.ndjson-12                                                            7.92kB ± 2%     7.74kB ± 2%    -2.31%  (p=0.016 n=5+5)
BackendProcessor/invalid-json-metadata.ndjson-12                                                    5.26kB ± 1%     5.18kB ± 1%    -1.49%  (p=0.040 n=5+5)
BackendProcessor/transactions-huge_traces.ndjson-12                                                 15.1kB ± 2%     14.8kB ± 1%    -1.99%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/events.ndjson-12                        41.7kB ± 1%     41.9kB ± 1%    +0.49%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/heavy.ndjson-12                          867kB ± 0%      853kB ± 1%    -1.58%  (p=0.016 n=5+4)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/metadata-null-values.ndjson-12          3.85kB ± 1%     3.93kB ± 1%    +2.15%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/errors_rum.ndjson-12                    8.59kB ± 1%     8.75kB ± 1%    +1.83%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/heavy.ndjson-12                          873kB ± 1%      856kB ± 0%    -1.93%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/metricsets.ndjson-12                    14.4kB ± 1%     14.3kB ± 1%    -1.06%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions.ndjson-12                  27.9kB ± 0%     27.6kB ± 0%    -0.85%  (p=0.008 n=5+5)
ReadBatch/errors_2.ndjson-12                                                                        22.7kB ± 0%     22.7kB ± 0%    -0.06%  (p=0.048 n=5+5)
ReadBatch/events.ndjson-12                                                                          38.7kB ± 0%     38.7kB ± 0%    -0.25%  (p=0.032 n=5+5)
ReadBatch/heavy.ndjson-12                                                                            838kB ± 0%      832kB ± 0%    -0.77%  (p=0.008 n=5+5)
ReadBatch/invalid-json-metadata.ndjson-12                                                           8.46kB ± 1%     8.40kB ± 0%    -0.68%  (p=0.024 n=5+5)
ReadBatch/invalid-metadata-2.ndjson-12                                                              5.80kB ± 0%     5.78kB ± 0%    -0.27%  (p=0.008 n=5+5)
ReadBatch/invalid-metadata.ndjson-12                                                                5.81kB ± 0%     5.79kB ± 0%    -0.30%  (p=0.008 n=5+5)
ReadBatch/logs.ndjson-12                                                                            18.4kB ± 0%     18.4kB ± 0%    -0.03%  (p=0.048 n=5+5)
ReadBatch/metricsets.ndjson-12                                                                      15.6kB ± 0%     15.4kB ± 1%    -1.46%  (p=0.008 n=5+5)
ReadBatch/minimal-service.ndjson-12                                                                 2.79kB ± 0%     2.77kB ± 0%    -0.61%  (p=0.008 n=5+5)
ReadBatch/minimal.ndjson-12                                                                         4.78kB ± 0%     4.76kB ± 0%    -0.45%  (p=0.016 n=5+4)
ReadBatch/optional-timestamps.ndjson-12                                                             3.62kB ± 0%     3.60kB ± 0%    -0.46%  (p=0.008 n=5+5)
ReadBatch/ratelimit.ndjson-12                                                                       10.5kB ± 0%     10.4kB ± 0%    -0.63%  (p=0.008 n=5+5)
ReadBatch/spans.ndjson-12                                                                           27.5kB ± 0%     27.5kB ± 0%    -0.07%  (p=0.008 n=5+5)
ReadBatch/unknown-span-type.ndjson-12                                                               16.8kB ± 0%     16.8kB ± 0%    -0.12%  (p=0.032 n=5+5)
pkg:github.com/elastic/apm-server/internal/publish goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage goos:linux goarch:amd64
ReadEvents/nop_codec_big_tx/10_events-12                                                            25.7kB ± 0%     25.7kB ± 0%    -0.06%  (p=0.016 n=5+4)
ReadEvents/nop_codec_big_tx/100_events-12                                                            251kB ± 0%      250kB ± 0%    -0.11%  (p=0.016 n=5+4)

name                                                                                              old allocs/op  new allocs/op   delta
pkg:github.com/elastic/apm-server/internal/agentcfg goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/beater/request goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/processor/stream goos:linux goarch:amd64
BackendProcessor/events.ndjson-12                                                                      717 ± 0%        716 ± 0%    -0.14%  (p=0.000 n=5+4)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/heavy.ndjson-12                          22.3k ± 0%      22.3k ± 0%    -0.02%  (p=0.048 n=5+5)
pkg:github.com/elastic/apm-server/internal/publish goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage goos:linux goarch:amd64

name                                                                                              old speed      new speed       delta
pkg:github.com/elastic/apm-server/internal/model/modelindexer goos:linux goarch:amd64
pkg:github.com/elastic/apm-server/internal/processor/stream goos:linux goarch:amd64
BackendProcessor/invalid-metadata.ndjson-12                                                       56.6MB/s ±12%   72.4MB/s ±24%   +27.88%  (p=0.016 n=5+5)
BackendProcessor/logs.ndjson-12                                                                   45.4MB/s ±22%   61.4MB/s ±26%   +35.31%  (p=0.032 n=5+5)
BackendProcessor/minimal-service.ndjson-12                                                        27.9MB/s ±15%   34.1MB/s ±15%   +21.98%  (p=0.032 n=5+5)
BackendProcessor/otel-bridge.ndjson-12                                                            45.4MB/s ± 9%   61.7MB/s ±34%   +35.90%  (p=0.032 n=5+5)
BackendProcessor/ratelimit.ndjson-12                                                              92.5MB/s ± 5%  100.4MB/s ± 8%    +8.60%  (p=0.016 n=4+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/invalid-json-event.ndjson-12           157MB/s ± 4%    143MB/s ± 6%    -9.07%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/invalid-metadata.ndjson-12             282MB/s ± 1%    235MB/s ± 9%   -16.98%  (p=0.016 n=4+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/metadata.ndjson-12                     285MB/s ± 4%    263MB/s ±11%    -7.76%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/minimal-service.ndjson-12              128MB/s ± 4%    117MB/s ± 6%    -8.60%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/optional-timestamps.ndjson-12          211MB/s ± 4%    196MB/s ±12%    -7.39%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/spans.ndjson-12                        251MB/s ±12%    232MB/s ± 8%    -7.70%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel2/transactions.ndjson-12                 173MB/s ± 8%    144MB/s ±20%   -16.85%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/invalid-metadata.ndjson-12             361MB/s ±11%    321MB/s ± 7%   -11.02%  (p=0.032 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel4/metadata.ndjson-12                     439MB/s ± 4%    372MB/s ±10%   -15.21%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/metricsets.ndjson-12                   510MB/s ± 1%    501MB/s ± 2%    -1.71%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions.ndjson-12                 484MB/s ± 1%    497MB/s ± 2%    +2.70%  (p=0.016 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel8/transactions_spans_rum.ndjson-12       539MB/s ± 2%    559MB/s ± 1%    +3.54%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/heavy.ndjson-12                      745MB/s ± 3%    792MB/s ± 2%    +6.35%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/invalid-json-event.ndjson-12         515MB/s ± 1%    532MB/s ± 0%    +3.29%  (p=0.008 n=5+5)
BackendProcessorParallel/BenchmarkBackendProcessorParallel200/minimal-service.ndjson-12            420MB/s ± 1%    432MB/s ± 2%    +3.01%  (p=0.008 n=5+5)
ReadBatch/errors_2.ndjson-12                                                                      44.0MB/s ±30%   58.1MB/s ±19%   +32.12%  (p=0.032 n=5+5)
ReadBatch/metadata-null-values.ndjson-12                                                          55.9MB/s ±22%   71.5MB/s ±34%   +27.77%  (p=0.032 n=5+5)
ReadBatch/transactions_spans_rum_2.ndjson-12                                                      57.3MB/s ±12%   70.8MB/s ±25%   +23.49%  (p=0.016 n=5+5)

report generated with https://pkg.go.dev/golang.org/x/perf/cmd/benchstat

@axw
Drop timeseries.instance on migration
7273255 
... and fix system tests
@axw axw enabled auto-merge (squash) November 15, 2022 12:00
@axw axw merged commit 774c29e into elastic:main Nov 15, 2022
@axw axw deleted the remove-timeseries-instance branch November 16, 2022 02:19
@elasticmachine
Copy link
Contributor

Package apm - 8.6.0-preview-1670294014 containing this change is available at https://epr.elastic.co/search?package=apm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@marclop marclop marclop left review comments

@simitt simitt simitt approved these changes

Assignees
No one assigned
Labels
backport-skip Skip notification from the automated backport with mergify v8.6.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

txmetrics: remove timeseries.instance field
5 participants
@axw @apmmachine @elasticmachine @simitt @marclop