Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

r/consensus: fixed reusing follower sequence id #12990

Merged
merged 4 commits into from
Sep 4, 2023

Conversation

mmaslankaprv
Copy link
Member

@mmaslankaprv mmaslankaprv commented Aug 24, 2023

All the places where follower sequence id is updated use post incrementation. Using pre incrementation operator in heartbeat manager lead to a situation in which two requests were assigned the same sequence_id.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.2.x
  • v23.1.x
  • v22.3.x

Release Notes

  • none

graphcareful
graphcareful previously approved these changes Aug 24, 2023
Copy link
Contributor

@graphcareful graphcareful left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can confirm when testing locally this patch fixes the issue where it was initially observed in PR #12716 when testing the ducktape test in the last commit

@bharathv
Copy link
Contributor

nice find, looks like we are missing some test coverage for linearizable barrier here? worth adding one?

@dotnwat
Copy link
Member

dotnwat commented Aug 24, 2023

lead to a situation in which two requests were assigned the same sequence_id.

and what was the situation? presumably it was a duplicate of the value of the first instance of follower_metadata.last_sent_seq?

graphcareful
graphcareful previously approved these changes Aug 25, 2023
Copy link
Contributor

@bharathv bharathv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a clarification.

src/v/raft/consensus.cc Show resolved Hide resolved
bharathv
bharathv previously approved these changes Aug 28, 2023
@graphcareful
Copy link
Contributor

There was a unit test failure in

The following tests FAILED:
  | 34 - test_raft_rpunit (Failed)

@mmaslankaprv mmaslankaprv dismissed stale reviews from bharathv and graphcareful via b0dc9ba August 30, 2023 10:24
@mmaslankaprv mmaslankaprv force-pushed the fix-lb branch 2 times, most recently from b0dc9ba to f3f232c Compare August 30, 2023 10:25
src/v/raft/consensus.cc Outdated Show resolved Hide resolved
@bharathv
Copy link
Contributor

test failure seems related.

All the places where follower sequence id is updated use post
incrementation. Using pre incrementation operator in heartbeat manager
lead to a situation in which two requests were assigned the same
sequence_id.

Signed-off-by: Michal Maslanka <michal@redpanda.com>
Linerizable barrier should make the `_committed_offset` to advance to the
latest possible value. Added flushing of leader log as a part of
linearizable barrier to move the `_committed_offset` forward

Signed-off-by: Michal Maslanka <michal@redpanda.com>
Grab `_op_lock` in `consensus::lineraizable_barrier` to wait for
follower replies to take effect. This makes linearizable_barrier more
strict as before returning result leader waits for the follower replies
to be processed.

Signed-off-by: Michal Maslanka <michal@redpanda.com>
Signed-off-by: Michal Maslanka <michal@redpanda.com>
@mmaslankaprv mmaslankaprv merged commit f1cd0f0 into redpanda-data:dev Sep 4, 2023
21 checks passed
@mmaslankaprv mmaslankaprv deleted the fix-lb branch September 4, 2023 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants