Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] add encode form and decode ndjson to httpjson input #23521

Merged
merged 2 commits into from
Jan 15, 2021

Conversation

leehinman
Copy link
Contributor

@leehinman leehinman commented Jan 15, 2021

What does this PR do?

  • adds support "application/x-www-form-urlencoded", as supported
    request.encode_as option. url parameters are encoded and sent in
    the body
  • adds support "application/x-ndjson", as supported
    response.decode_as option. json objects are decoded and returned
    in an array at root level

Why is it important?

Some API endpoints require posting data as
application/x-www-form-urlencoded

Some API endpoints only return application/x-ndjson not json

Checklist

  • My code follows the style guidelines of this project
    - [ ] I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 15, 2021
- support "application/x-www-form-urlencoded", url parameters are
  encoded and sent in the body
- support "application/x-ndjson", json objects are decoded and
  returned in an array at root level
@leehinman leehinman added enhancement Filebeat Filebeat needs_backport PR is waiting to be backported to other branches. Team:Security-External Integrations labels Jan 15, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 15, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jan 15, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #23521 updated

    • Start Time: 2021-01-15T17:43:15.047+0000
  • Duration: 48 min 1 sec

  • Commit: 5dc7a17

Test stats 🧪

Test Results
Failed 0
Passed 5135
Skipped 574
Total 5709

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 5135
Skipped 574
Total 5709

}

func encodeAsJSON(trReq transformable) ([]byte, error) {
if len(trReq.body()) == 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

Comment on lines 113 to 122
for {
var o interface{}
err := dec.Decode(&o)
if err == io.EOF {
break
} else if err != nil {
return err
}
results = append(results, o)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for {
var o interface{}
err := dec.Decode(&o)
if err == io.EOF {
break
} else if err != nil {
return err
}
results = append(results, o)
}
for dec.More() {
var o interface{}
if err := dec.Decode(&o); err != nil {
return err
}
results = append(results, o)
}

Copy link
Contributor

@marc-gr marc-gr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, feel free to ignore or not the suggested change.

@leehinman leehinman merged commit 1327562 into elastic:master Jan 15, 2021
@leehinman leehinman added v7.12.0 and removed needs_backport PR is waiting to be backported to other branches. labels Feb 1, 2021
leehinman added a commit to leehinman/beats that referenced this pull request Feb 1, 2021
…ic#23521)

* add encode form and decode ndjson to httpjson

- support "application/x-www-form-urlencoded", url parameters are
  encoded and sent in the body
- support "application/x-ndjson", json objects are decoded and
  returned in an array at root level

(cherry picked from commit 1327562)
@leehinman leehinman deleted the post_x_www_form_urlencoded branch February 1, 2021 21:23
leehinman added a commit that referenced this pull request Feb 2, 2021
… (#23794)

* add encode form and decode ndjson to httpjson

- support "application/x-www-form-urlencoded", url parameters are
  encoded and sent in the body
- support "application/x-ndjson", json objects are decoded and
  returned in an array at root level

(cherry picked from commit 1327562)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants