Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws_ec2_metadata transform assumes event is an object, even in Vector namespace #17193

Closed
Tracked by #15003
StephenWakely opened this issue Apr 21, 2023 · 1 comment · Fixed by #17819
Closed
Tracked by #15003
Assignees
Labels
transform: aws_ec2_metadata Anything `aws_ec2_metadata` transform related type: bug A code related bug.

Comments

@StephenWakely
Copy link
Contributor

A note for the community

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

The aws_ec2_metadata transform assumes the incoming object is an Object. This is fine for the Legacy namespace (currently in use everywhere), but in the Vector namespace this is not always the case.

Currently Vector will panic (once the transform definitions work goes in) on load if the incoming event definition is not an object.

Configuration

schema:
  enabled: true

sources:
  in:
    type: syslog
    address: 0.0.0.0:9000
    mode: tcp
    log_namespace: true

transforms:
  thing:
    type: aws_ec2_metadata
    inputs:
      - in
      
sinks:
  console:
    type: console
    inputs:
      - thing
    encoding:
      codec: json
    target: stdout

Version

v0.30

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response

@StephenWakely StephenWakely added type: bug A code related bug. transform: aws_ec2_metadata Anything `aws_ec2_metadata` transform related labels Apr 21, 2023
@fuchsnj
Copy link
Member

fuchsnj commented Jun 30, 2023

Long term this transform should probably be deprecated. It doesn't provide enough flexibility to decide how to enrich an event with the new data. A better alternative is to allow this enrichment in VRL with something like #15110

@fuchsnj fuchsnj self-assigned this Jun 30, 2023
github-merge-queue bot pushed a commit that referenced this issue Jul 1, 2023
closes: #17193

This keeps the same runtime behavior (if the even is not an object, it
is converted to an object), but fixes the schema definition calculation
to not panic.

The behavior might not be optimal since it may result in data loss.
Users can remap before hand and convert to an object if needed, and
longer term this functionality should be made available in VRL for more
flexibility.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
transform: aws_ec2_metadata Anything `aws_ec2_metadata` transform related type: bug A code related bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants