Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transform wasm #633

Closed
wants to merge 13 commits into from
Closed

Transform wasm #633

wants to merge 13 commits into from

Conversation

anuraaga
Copy link
Owner

@anuraaga anuraaga commented Apr 28, 2022

Plugin implementation https://github.com/anuraaga/opentelemetry-collector-contrib/pull/633/files#diff-3bae4a8ec0d5af632e586e8e1c3f87c81704bbd96a9760fa0ee7b7e71e5844f4

Identical code to built-in keep_keys function except for working around tinygo not supporting reflect yet for functions, which should be implementable

Plugin framework https://github.com/anuraaga/opentelemetry-collector-contrib/pull/633/files#diff-b3e4bc5021aed817d7157a1fc6a7f83f6b72e735e517e9cc47343b9775746fb0

Implements OTel data model (pdata) and transformation API by delegating to host functions and appropriate marshaling.

Host functions https://github.com/anuraaga/opentelemetry-collector-contrib/pull/633/files

Function resolution and dispatch

https://github.com/anuraaga/opentelemetry-collector-contrib/pull/633/files#diff-a48d4b6a3df6770ea57f07c4c977333e72f136b73dfc85884d00c9bae98fc795R58

General flow

  • wasm function .start calls host function to tell host what function factory signatures it is adding to the collector. host registers signature
  • host parses user config file. If function name in config matches one of the wasm registered names, it resolves the config's function parameters in the host and passes them to the wasm function factory
  • wasm function factory registers the instantiated transformation function and returns index to it to host
  • host receives telemetry, calls instantiated transformation function by passing index and telemetry context to wasm

To allow wasm to allocate objects on the host where needed, we take advantage of the lifecycle being well defined - the function invocation has the same lifetime as the piece of telemetry. So we can just store references into context scoped to the telemetry to prevent GC. A poorly behaved wasm function could of course store references globally, but there is no use case for this.

Other binders will serialize objects between host and wasm, sometimes even as JSON (?!). Here the only serialization we do is of referenced strings, everything else is an opaque pointer that is passed between host and wasm as needed.

@github-actions
Copy link

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Jan 26, 2023
@github-actions
Copy link

Closed as inactive. Feel free to reopen if this PR is still being worked on.

@github-actions github-actions bot closed this Feb 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant