Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Convert Filebeat logstash.* to ECS #9935

Merged
merged 13 commits into from
Jan 11, 2019
Merged

Convert Filebeat logstash.* to ECS #9935

merged 13 commits into from
Jan 11, 2019

Commits on Jan 11, 2019

  1. Fix funny pipeline descriptions

    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    dfc4d9e View commit details
    Browse the repository at this point in the history

  2. Perform simple field migrations only in pipelines with tests (plain l… 

    …ogs):

- logstash.log.level => log.level
- logstash.log.message => message
- logstash.slowlog.level => log.level
- logstash.slowlog.message => message
- logstash.slowlog.took_in_nanos => event.duration
- read_timestamp => event.created

Pipelines for JSON logs haven't been migrated.
    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    c1bb2d3 View commit details
    Browse the repository at this point in the history

  3. Add JSON logs to test those as well... 

    This fails the tests until `test_modules.py` has support for this.
    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    05e3a7b View commit details
    Browse the repository at this point in the history

  4. Fix event.dataset from rebase

    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    5cd2adb View commit details
    Browse the repository at this point in the history

  5. Move original ts to event.created for the JSON pipeline as well. 

    Note that this field is stripped out of the saved JSON, since it would differ from one test run to another
    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    62600ce View commit details
    Browse the repository at this point in the history

  6. Translate both JSON pipelines to ECS: 

    - log.level in both
- message only in the 'log' dataset. It's not kept in slowlog because it's always the same message
    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    3085b76 View commit details
    Browse the repository at this point in the history

  7. Bring plain parser for slow log in line with JSON one: 'message' isn'… 

    …t kept.
    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    f38d3a8 View commit details
    Browse the repository at this point in the history

  8. Strip out the trailing spaces after the logstash module name

    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    83cca37 View commit details
    Browse the repository at this point in the history

  9. Create field aliases for log and slowlog

    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    a521d72 View commit details
    Browse the repository at this point in the history

  10. Document ECS field migrations

    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    13035db View commit details
    Browse the repository at this point in the history

  11. Changelog

    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    6e1e539 View commit details
    Browse the repository at this point in the history

  12. Tweak to pipeline names so they mention slowlog specifically

    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    95b8cc5 View commit details
    Browse the repository at this point in the history

  13. Rename took_in_nanos to event.duration for the JSON logs as well.

    Mathieu Martin committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    d503db9 View commit details
    Browse the repository at this point in the history