Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Convert Filebeat nginx.access to ECS #9081

Merged
merged 16 commits into from
Nov 27, 2018
Merged

Convert Filebeat nginx.access to ECS #9081

merged 16 commits into from
Nov 27, 2018

Commits on Nov 26, 2018

  1. Use network.forwarded_ip instead of nginx.access.remote_ip_list for p… 

    …roxy IPs
    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    8dda0e6 View commit details
    Browse the repository at this point in the history

  2. nginx.access.user_name => user.name

    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    06f0f00 View commit details
    Browse the repository at this point in the history

  3. nginx.access.response_code => http.response.status_code

    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    33fa659 View commit details
    Browse the repository at this point in the history

  4. Rename 3 http fields and url: 

    - nginx.access.referrer => http.request.referrer
- nginx.access.method => http.request.method
- nginx.access.url => url.original
- nginx.access.http_version => http.version
    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    f010533 View commit details
    Browse the repository at this point in the history

  5. nginx.access.agent => user_agent.original

    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    6eb3473 View commit details
    Browse the repository at this point in the history

  6. Interpret remote host: 

    - Copy remote_ip to `source.ip` if it's an IP
- Copy remote_ip to `source.domain` otherwise
- Perform geoIP on `source.ip` when present
- Output geo information to `source.geo`
  - nginx.access.geoip.* => source.geo.*
    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    c35d745 View commit details
    Browse the repository at this point in the history

  7. Add two temporary definitions: 

    - `http.request.referrer`
- `url.original`

This is strange. Not sure how I pushed this PR with these failures unsolved...
    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    7298b75 View commit details
    Browse the repository at this point in the history

  8. Coerce status code and bytes into ints

    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    afb743d View commit details
    Browse the repository at this point in the history

  9. Shove all the renames so far in ecs-migration.yml

    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    3e15592 View commit details
    Browse the repository at this point in the history

  10. Changelog

    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    4957185 View commit details
    Browse the repository at this point in the history

  11. Perform the user_agent dance.

    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    76278fc View commit details
    Browse the repository at this point in the history

  12. Replace one of 3 Mac/FF user agent with Mac/Chrome, to spice things up

    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    70ba669 View commit details
    Browse the repository at this point in the history

  13. Remove grok to detect non-IPs in source.ip... 

    Adding a dumb `set` for source.ip, otherwise in the painless context, `source` itself doesn't exist yet. And adding this dictionary in painless sounded more painful than using `set`.
    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    0d25bbe View commit details
    Browse the repository at this point in the history

  14. Alias all the fields in the fileset

    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    00cc78e View commit details
    Browse the repository at this point in the history

  15. Move the last field not aliased to the top, to promote sanity

    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    72a22e4 View commit details
    Browse the repository at this point in the history

  16. Fix alias to http.request.referrer, not http.referrer

    Mathieu Martin committed Nov 26, 2018
    Configuration menu
    Copy the full SHA
    bb4c680 View commit details
    Browse the repository at this point in the history