fixes fluentd configuration params in fluentd-elasticsearch addon

[saravanan30erd]

What this PR does / why we need it:

• fixes the errors/warnings in fluentd configuration.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):

2018-08-28 11:40:50 +0000 [warn]: 'filters' parameter is deprecated: filters has been renamed as matches
2018-08-28 11:40:50 +0000 [warn]: 'filters' parameter is deprecated: filters has been renamed as matches
2018-08-28 11:40:50 +0000 [warn]: 'filters' parameter is deprecated: filters has been renamed as matches
2018-08-28 11:40:50 +0000 [warn]: 'filters' parameter is deprecated: filters has been renamed as matches
2018-08-28 11:40:50 +0000 [warn]: 'filters' parameter is deprecated: filters has been renamed as matches
2018-08-28 11:40:50 +0000 [warn]: parameter 'time_format' in <source>
  @id fluentd-containers.log
  @type tail
  path "/var/log/containers/*.log"
  pos_file "/var/log/es-containers.log.pos"
  time_format %Y-%m-%dT%H:%M:%S.%NZ
  tag "raw.kubernetes.*"
  read_from_head true
  <parse>
    @type "multi_format"
    <pattern>
      format json
      time_key "time"
      time_format "%Y-%m-%dT%H:%M:%S.%NZ"
      time_type string
    </pattern>
    <pattern>
      format /^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$/
      time_format "%Y-%m-%dT%H:%M:%S.%N%:z"
      expression ^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$
      ignorecase false
      multiline false
    </pattern>
  </parse>
</source> is not used.

Release note:

fixes the errors/warnings in fluentd configuration

[k8s-ci-robot]

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

---

• If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
• If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
• If you have done the above and are still having issues with the CLA being reported as unsigned, please email the CNCF helpdesk: helpdesk@rt.linuxfoundation.org

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[saravanan30erd]

finished the CLA

[saravanan30erd]

/assign @piosz

[neolit123]

/sig instrumentation
/remove-sig cluster-lifecycle
/ok-to-test
/kind bug

[coffeepac]

instead of removing this line, can you add time_type string? this will cause the specified time_format to be applied

[saravanan30erd]

@coffeepac
Thanks. Looks like time parameters (time_type, time_format,..) are only applicable to <parse> section of source plugin tail. If we use this params outside parse section in tail plugin then fluentd seems ignoring these params and we are getting the warnings.

I got the below errors/warnings when I tried to add time_type string , please correct me if I am wrong.

2018-08-28 17:28:59 +0000 [warn]: parameter 'time_format' in <source>
  @id fluentd-containers.log
  @type tail
  path "/var/log/containers/*.log"
  pos_file "/var/log/es-containers.log.pos"
  time_format %Y-%m-%dT%H:%M:%S.%NZ
  time_type string
  estimate_current_event false
  refresh_interval 20
  tag "raw.kubernetes.*"
  read_from_head true
  <parse>
    @type "multi_format"
    <pattern>
      format json
      time_key "time"
      time_format "%Y-%m-%dT%H:%M:%S.%NZ"
      time_type string
    </pattern>
    <pattern>
      format /^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$/
      time_format "%Y-%m-%dT%H:%M:%S.%N%:z"
      expression ^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$
      ignorecase false
      multiline false
    </pattern>
  </parse>
</source> is not used.
2018-08-28 17:28:59 +0000 [warn]: parameter 'time_type' in <source>
  @id fluentd-containers.log
  @type tail
  path "/var/log/containers/*.log"
  pos_file "/var/log/es-containers.log.pos"
  time_format %Y-%m-%dT%H:%M:%S.%NZ
  time_type string
  estimate_current_event false
  refresh_interval 20
  tag "raw.kubernetes.*"
  read_from_head true
  <parse>
    @type "multi_format"
    <pattern>
      format json
      time_key "time"
      time_format "%Y-%m-%dT%H:%M:%S.%NZ"
      time_type string
    </pattern>
    <pattern>
      format /^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$/
      time_format "%Y-%m-%dT%H:%M:%S.%N%:z"
      expression ^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$
      ignorecase false
      multiline false
    </pattern>
  </parse>
</source> is not used.

[coffeepac]

ahh, okay, I was misreading the docs then.

looks good!

[coffeepac]

/assign @coffeepac

[coffeepac]

/unassign @piosz

[coffeepac]

/test pull-kubernetes-e2e-gce

[coffeepac]

/approve
/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: coffeepac, saravanan30erd

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cluster/addons/fluentd-elasticsearch/OWNERS [coffeepac]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coffeepac]

thanks @saravanan30erd !

[k8s-github-robot]

Automatic merge from submit-queue (batch tested with PRs 67067, 67947). If you want to cherry-pick this change to another branch, please follow the instructions here.