Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Filebeat Logstash Module can fail parsing the pipeline id in plain logs if it contains not alphanumeric characters #17242

Closed
lucabelluccini opened this issue Mar 25, 2020 · 2 comments · Fixed by #17243
Closed

Filebeat Logstash Module can fail parsing the pipeline id in plain logs if it contains not alphanumeric characters #17242

lucabelluccini opened this issue Mar 25, 2020 · 2 comments · Fixed by #17243
Labels
Feature:Stack Monitoring module Team:Services (Deprecated) Label for the former Integrations-Services team

Comments

@lucabelluccini
Copy link
Contributor

Filebeat Logstash Module can fail parsing the pipeline id in plain logs if it contains not alphanumeric characters.

  • Version: 7.6.0+
  • Steps to Reproduce:
    • Install Filebeat 7.6.0
    • Enable the Logstash module
    • Install Logstash 7.5 or 7.6 (which contains the pipeline id in the logs), ensure it is using plain logging
    • Create a pipeline with name .test which targets an invalid Elasticsearch instance or enable monitoring, targeting an invalid hostname
    • Start Logstash & Filebeat
    • You'll see rejections "Provided grok expressions do not match..."

This might occur also for .monitoring-logstash pipeline when Logstash has monitoring enabled and the destination cluster is not up and running.

A suggestion is to use DATA or NOTSPACE instead of WORD in the grok processor.
@elasticmachine
Copy link
Collaborator

Pinging @elastic/stack-monitoring (Stack monitoring)

lucabelluccini added a commit that referenced this issue Mar 25, 2020
@lucabelluccini
Allow a wider range of characters for logstash.log.pipeline_id
25b0b92 
Allow a wider range of characters for `logstash.log.pipeline_id`.
The `NOTSPACE` should be fine, but for covering any case, `DATA` would be safer.
Should solve #17242
@lucabelluccini lucabelluccini mentioned this issue Mar 25, 2020
Merged
3 tasks
@ycombinator ycombinator added the Team:Services (Deprecated) Label for the former Integrations-Services team label Mar 25, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

ycombinator pushed a commit that referenced this issue Mar 30, 2020
@lucabelluccini
Allow a wider range of characters for logstash.log.pipeline_id (#17243)
a59768f 
* Allow a wider range of characters for logstash.log.pipeline_id

Allow a wider range of characters for `logstash.log.pipeline_id`.
The `NOTSPACE` should be fine, but for covering any case, `DATA` would be safer.
Should solve #17242

* update tests for logstash plain 7.4 pipeline id

* Failed tests

* update changelog
ycombinator added a commit that referenced this issue Mar 30, 2020
@ycombinator @lucabelluccini
[7.x] Allow a wider range of characters for logstash.log.pipeline_id (#…
42578f6 
…17243) (#17338)

* Allow a wider range of characters for logstash.log.pipeline_id (#17243)

* Allow a wider range of characters for logstash.log.pipeline_id

Allow a wider range of characters for `logstash.log.pipeline_id`.
The `NOTSPACE` should be fine, but for covering any case, `DATA` would be safer.
Should solve #17242

* update tests for logstash plain 7.4 pipeline id

* Failed tests

* update changelog

* CHANGELOG grooming

Co-authored-by: Luca Belluccini <luca.belluccini@elastic.co>
ycombinator added a commit that referenced this issue Mar 31, 2020
@ycombinator @lucabelluccini
[7.7] Allow a wider range of characters for logstash.log.pipeline_id (#…
cdc6abf 
…17243) (#17357)

* Allow a wider range of characters for logstash.log.pipeline_id (#17243)

* Allow a wider range of characters for logstash.log.pipeline_id

Allow a wider range of characters for `logstash.log.pipeline_id`.
The `NOTSPACE` should be fine, but for covering any case, `DATA` would be safer.
Should solve #17242

* update tests for logstash plain 7.4 pipeline id

* Failed tests

* update changelog

* CHANGELOG grooming

Co-authored-by: Luca Belluccini <luca.belluccini@elastic.co>
@andresrc andresrc reopened this Apr 1, 2020
@andresrc andresrc closed this as completed Apr 1, 2020
ycombinator added a commit that referenced this issue Apr 2, 2020
@ycombinator @lucabelluccini
[7.6] Allow a wider range of characters for logstash.log.pipeline_id (#…
a0e182c 
…17243) (#17339)

* Allow a wider range of characters for logstash.log.pipeline_id (#17243)

* Allow a wider range of characters for logstash.log.pipeline_id

Allow a wider range of characters for `logstash.log.pipeline_id`.
The `NOTSPACE` should be fine, but for covering any case, `DATA` would be safer.
Should solve #17242

* update tests for logstash plain 7.4 pipeline id

* Failed tests

* update changelog

* CHANGELOG grooming

* Removing fields that are not available in Filebeat 7.6

Co-authored-by: Luca Belluccini <luca.belluccini@elastic.co>
leweafan pushed a commit to leweafan/beats that referenced this issue Apr 28, 2023
@ycombinator @lucabelluccini
[7.7] Allow a wider range of characters for logstash.log.pipeline_id (e…
be8f5a8 
…lastic#17243) (elastic#17357)

* Allow a wider range of characters for logstash.log.pipeline_id (elastic#17243)

* Allow a wider range of characters for logstash.log.pipeline_id

Allow a wider range of characters for `logstash.log.pipeline_id`.
The `NOTSPACE` should be fine, but for covering any case, `DATA` would be safer.
Should solve elastic#17242

* update tests for logstash plain 7.4 pipeline id

* Failed tests

* update changelog

* CHANGELOG grooming

Co-authored-by: Luca Belluccini <luca.belluccini@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Feature:Stack Monitoring module Team:Services (Deprecated) Label for the former Integrations-Services team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allow a wider range of characters for logstash.log.pipeline_id elastic/beats
4 participants
@ycombinator @lucabelluccini @andresrc @elasticmachine