Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential bug in join when build has many duplicated keys #8796

Closed
windtalker opened this issue Feb 27, 2024 · 0 comments · Fixed by #8797
Closed

Potential bug in join when build has many duplicated keys #8796

windtalker opened this issue Feb 27, 2024 · 0 comments · Fixed by #8797
Labels
affects-6.6 affects-7.0 affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.2 affects-7.3 affects-7.4 affects-7.5 This bug affects the 7.5.x(LTS) versions. component/compute severity/major type/bug The issue is confirmed as a bug.

Comments

@windtalker
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!
In join probe, if build has many duplicated key, the intermedia result block could be large, in order to control the overall memory usage, on input block will be probed multiple times, each time only data in [probe_process_info.start_row, probe_process_info.end_row) will be processed.
But some data structures used in join assume whole block data is processed, for example:
https://github.com/pingcap/tiflash/blob/ef90ce478aff27a7499a1a55debac5957f326476/dbms/src/Interpreters/Join.cpp#L870
anti_filter and offsets_to_replicate in handleOtherConditions both assumes it contains all the data in block. In order to adopt this, before handleOtherConditions, the caller need to shrink anti_filter and offsets_to_replicate. Currently, it use assign. However, shrink actually should not use memcpy since there is a chance that dst and src have overlap, and it is undefined behavior for memcpy is dst and src have overlap

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiFlash version? (Required)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.6 affects-7.0 affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.2 affects-7.3 affects-7.4 affects-7.5 This bug affects the 7.5.x(LTS) versions. component/compute severity/major type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant