Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log flooding issue in ticdc check #7235

Closed
yiwen92 opened this issue Sep 27, 2022 · 1 comment · Fixed by #7245
Closed

Log flooding issue in ticdc check #7235

yiwen92 opened this issue Sep 27, 2022 · 1 comment · Fixed by #7245
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. severity/minor type/bug The issue is confirmed as a bug.

Comments

@yiwen92
Copy link

yiwen92 commented Sep 27, 2022

What did you do?

upgrade from 6.0.0 to 6.0.1, CDC running on 6.1.1, logs are quickly filled with tons of the entries similar to this one:
[2022/09/26 06:47:12.746 +00:00] [INFO] [mounter.go:265] ["column value is not found"] [table=xx] [column=xx]
[2022/09/26 06:47:12.746 +00:00] [INFO] [mounter.go:265] ["column value is not found"] [table=xx] [column=xx]
[2022/09/26 06:47:12.746 +00:00] [INFO] [mounter.go:265] ["column value is not found"] [table=xx] [column=xx]
...
it seems it reacts this way on null as column value.
Logs grow very fast (many Gb per day) on CDC nodes so I don't think it's very normal.
We don't see this issue on 6.1.0 clusters though.

What did you expect to see?

No response

What did you see instead?

Logs grow very fast (many Gb per day) on CDC nodes so I don't think it's very normal.

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

(paste TiDB cluster version here)

Upstream TiKV version (execute tikv-server --version):

(paste TiKV version here)

TiCDC version (execute cdc version):

(v6.1.1)
@yiwen92 yiwen92 added area/ticdc Issues or PRs related to TiCDC. type/bug The issue is confirmed as a bug. labels Sep 27, 2022
@yiwen92
Copy link
Author

yiwen92 commented Sep 27, 2022

This was introduced by https://github.com/pingcap/tiflow/pull/6416/files. Will change the log to debug level. @hi-rustin Could you please make sure it can be fixed in v6.1.2?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. severity/minor type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants