Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dropEvent not commit offset #672

Open
kytool opened this issue Apr 11, 2024 · 0 comments
Open

dropEvent not commit offset #672

kytool opened this issue Apr 11, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@kytool
Copy link
Contributor

kytool commented Apr 11, 2024

In what area(s)?

/area interceptor

What version of Loggie?

main

Actual Behavior

dropEvent not commit offset

Steps to Reproduce the Problem

  • pipelines.yml
pipelines:
  - name: local
    sources:
      - type: file
        name: demo
        paths:
          - xxx.log
    interceptors:
      - type: transformer
        actions:
          - action: print()
          - if: NOT equal(level, DEBUG)
            then:
              - action: dropEvent()
    sink:
      type: dev
      printEvents: true
      codec:
        pretty: true

dropEvent后因为没有commit,导致offset不变
如果某个文件每一行都dropEvent,会导致

  1. 重启后会重复读 ,虽然不会到sink,但是还是会消耗资源
  2. 不知道文件实际上有没有读
@kytool kytool added the bug Something isn't working label Apr 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant