Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue with shift propagation from Table.reverse() on blink tables #3888

Merged
merged 2 commits into from
May 31, 2023

Conversation

rcaudy
Copy link
Member

@rcaudy rcaudy commented May 31, 2023

Fixes #3858

@rcaudy rcaudy added bug Something isn't working query engine core Core development tasks ReleaseNotesNeeded Release notes are needed labels May 31, 2023
@rcaudy rcaudy added this to the May 2023 milestone May 31, 2023
@rcaudy rcaudy requested a review from devinrsmith May 31, 2023 02:42
@rcaudy rcaudy self-assigned this May 31, 2023
@rcaudy
Copy link
Member Author

rcaudy commented May 31, 2023

Tested with unit test and the provided reproducer.

@rcaudy rcaudy enabled auto-merge (squash) May 31, 2023 02:44
Copy link
Member

@devinrsmith devinrsmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've verified this fixes the issue. I can't say I understand the nuances involved in ReverseOperation#onUpdate. I'm assuming the else branch is still technically "correct" for empty resultRowSets in the non-blink case; but this will be an improvement for non-blink cases regardless?

@rcaudy
Copy link
Member Author

rcaudy commented May 31, 2023

I've verified this fixes the issue. I can't say I understand the nuances involved in ReverseOperation#onUpdate. I'm assuming the else branch is still technically "correct" for empty resultRowSets in the non-blink case; but this will be an improvement for non-blink cases regardless?

Correct. There's no reason to communicate any shifts at all for rows that don't exist (in post-shift space) in the result table. In this case, we can assess that by looking at the result row set once all upstream removes have been removed. Not communicating unnecessary shifts simplifies the downstream update, and potentially saves a lot of work for downstream listeners.

@devinrsmith
Copy link
Member

devinrsmith commented May 31, 2023

communicating unnecessary shifts

Should we explicitly disallow unnecessary shifts? Or, have helpers that simplify them away for all cases?

@rcaudy rcaudy merged commit accb071 into deephaven:main May 31, 2023
@rcaudy rcaudy deleted the rwc-reverse-stream-fix branch May 31, 2023 04:32
@github-actions github-actions bot locked and limited conversation to collaborators May 31, 2023
@rcaudy
Copy link
Member Author

rcaudy commented May 31, 2023

#3891

if (nextShiftEnd < nextShiftStart) {
continue;
// Only compute downstream shifts if there are retained rows to shift
if (resultRowSet.isEmpty()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could also skip this if the prev rowset is empty (since shifts affect rows that exist in both key spaces)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working core Core development tasks NoDocumentationNeeded query engine ReleaseNotesNeeded Release notes are needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Stream table reverse assertion failure
3 participants