Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metricbeat]Log prometheus errors instead of parsing families #15712

Merged

Conversation

odacremolbap
Copy link
Contributor

Solves #15709

Some users have been reporting errors fetching prometheus endpoints that include logs stating format errors decoding families.

Most probably metricbeat's prometheus helper has been trying to parse errors received as metric families. This PR adds the HTTP status code to the error sent upstream from the helper and also allows for writing the error body to log when debug is enabled.

@odacremolbap odacremolbap added the Team:Integrations Label for the Integrations team label Jan 27, 2020
@odacremolbap odacremolbap self-assigned this Jan 28, 2020
@odacremolbap odacremolbap requested a review from a team January 28, 2020 09:39
@andresrc andresrc added [zube]: Inbox Team:Platforms Label for the Integrations - Platforms team [zube]: In Review and removed [zube]: Inbox labels Jan 28, 2020
Copy link
Contributor

@exekias exekias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for starting this, great improvement!

if p.logger.IsDebug() {
bodyBytes, err := ioutil.ReadAll(resp.Body)
if err == nil {
p.logger.Debug("error received from prometheus endpoint: ", string(bodyBytes))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you expect this body to be large? as this error is fairly important we could consider showing it by default. With some trimming if needed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't expect large errors payloads there.
will change it to show description without debug enabled

@odacremolbap odacremolbap merged commit edc17b3 into elastic:master Jan 30, 2020
odacremolbap pushed a commit to odacremolbap/beats that referenced this pull request Jan 30, 2020
…c#15712)

* log prometheus errors instead of parsing

(cherry picked from commit edc17b3)
odacremolbap pushed a commit that referenced this pull request Jan 31, 2020
…of parsing families (#15961)

* [Metricbeat]Log prometheus errors instead of parsing families (#15712)
jsoriano pushed a commit to jsoriano/beats that referenced this pull request Feb 10, 2020
jsoriano pushed a commit to jsoriano/beats that referenced this pull request Feb 11, 2020
…c#15712)

* log prometheus errors instead of parsing

(cherry picked from commit edc17b3)
jsoriano added a commit that referenced this pull request Feb 12, 2020
…of parsing families (#16231)

(cherry picked from commit edc17b3)

Co-authored-by: Pablo Mercado <pablo.mercado@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Metricbeat Metricbeat review Team:Integrations Label for the Integrations team Team:Platforms Label for the Integrations - Platforms team v7.6.0 v7.7.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants