Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sarama should implement KIP-320 #2365

Closed
dajac opened this issue Oct 17, 2022 · 1 comment
Closed

Sarama should implement KIP-320 #2365

dajac opened this issue Oct 17, 2022 · 1 comment

Comments

@dajac
Copy link

dajac commented Oct 17, 2022

We've recently found out that Sarama use the Fetch API >= version 11 but always set the leader epoch introduced in KIP-320 to -1. Fetching without a leader epoch could lead to spurious offset out of range errors if the client ends up fetching from a stale leader. With the Fetch API >= 11, this is even worst because the broker accept fetches from followers, even without a leader epoch. The client could get spurious out of range errors if it fetches from a follower by mistake as well.

We highly recommend to get KIP-320 implemented in Sarama in order to strengthen its guarantees.

We are discussing options to mitigate this on the server side as well here: https://issues.apache.org/jira/browse/KAFKA-14255.

Best,
David
dnwe added a commit that referenced this issue Nov 14, 2022
@dnwe
feat(proto): implement and use MetadataRequest v7
f40c313 
In order to receive the LeaderEpoch as part of the metadata response we
need to be sending and receiving version 7 of the
MetadataRequest+Response protocol. Luckily the first flexible version
wasn't until v8 so the changes to achieve this are minimal.

Contributes-to: #2365
dnwe added a commit that referenced this issue Nov 14, 2022
@dnwe
feat(proto): implement and use MetadataRequest v7
48d3cff 
In order to receive the LeaderEpoch as part of the metadata response we
need to be sending and receiving version 7 of the
MetadataRequest+Response protocol. Luckily the first flexible version
wasn't until v8 so the changes to achieve this are minimal.

Contributes-to: #2365
@dnwe dnwe mentioned this issue Nov 14, 2022
Merged
dnwe added a commit that referenced this issue Nov 14, 2022
@dnwe
feat(proto): implement and use MetadataRequest v7 (#2388)
72ea327 
In order to receive the LeaderEpoch as part of the metadata response we
need to be sending and receiving version 7 of the
MetadataRequest+Response protocol. Luckily the first flexible version
wasn't until v8 so the changes to achieve this are minimal.

Contributes-to: #2365
dnwe added a commit that referenced this issue Nov 14, 2022
@dnwe
fix: track and supply leader epoch to FetchRequest
609a3da 
Include the consumer's view of the leader in Fetch requests as per
KIP-320

Contributes-to: #2365
@dnwe dnwe mentioned this issue Nov 14, 2022
Merged
dnwe added a commit that referenced this issue Nov 14, 2022
@dnwe
fix: track and supply leader epoch to FetchRequest (#2389)
fcd0ea5 
Include the consumer's view of the leader in Fetch requests as per
KIP-320

Contributes-to: #2365
@dnwe
Copy link
Collaborator

dnwe commented Jan 22, 2023

Should be fixed now, thanks for reporting

@dnwe dnwe closed this as completed Jan 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dajac @dnwe