[feat] Support batch index acknowledgment #151
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes apache#87 ### Modifications - Add an consumer configuration `setBatchIndexAckEnabled` to enable the batch index ACK. When it's enabled, passing the original `MessageId` instead of the `MessageIdImpl` that trunks the batch index to the ACK grouping tracker. - Since now a `BatchedMessageIdImpl` could be accepted in the ACK grouping tracker, fix the compare logic. - Support passing a `BitSet` in `Commands::newAck` and get the internal `BitSet` from `MessageId` in `Commands::newMultiMessageAck`. - Skip the acknowledged batch indexes when receiving batched messages in `ConsumerImpl::receiveIndividualMessagesFromBatch`. ### Verifications Modify `BitSetTest.testSet` to verify the `BitSet::get` method added in this commit. Add `AcknowledgeTest.testBatchIndexAck` to test batch index ACK for all types of acknowledgment: - Individual ACK for a single message - Individual ACK for a list of messages - Cumulative ACK Add `AcknowledgeTest.testMixedCumulativeAck` to test the new compare logic between `BatchedMessageIdImpl` and `MessageIdImpl` works for cumulative ACK.
BewareMyPower
changed the title
shibd
approved these changes
Dec 26, 2022
|
@merlimat @RobertIndie @Demogorgon314 Could you take a second look? |
This was referenced Jan 3, 2023
|
@merlimat @RobertIndie @Demogorgon314 This PR has been pending for 3 weeks. Could any committer take a look? |
Demogorgon314
approved these changes
Jan 17, 2023
BewareMyPower
added a commit
to BewareMyPower/pulsar-client-cpp
that referenced
this pull request
Jan 17, 2023
### Motivation Currently the main branch is broken by the concurrent merge of apache#153 and apache#151. It's because when a batched message id is constructed from deserialization, there is no `getBitSet` implementation of the internal acker. ### Modifications Add a `bool` parameter to `MessageIdImpl::getBitSet` to indicate whether the message ID is batched. The logic is similar with https://github.com/apache/pulsar/blob/299bd70fdfa023768e94a8ee4347d39337b6cbd4/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PersistentAcknowledgmentsGroupingTracker.java#L325-L327 and https://github.com/apache/pulsar/blob/299bd70fdfa023768e94a8ee4347d39337b6cbd4/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PersistentAcknowledgmentsGroupingTracker.java#L345-L347 Add a `testMessageIdFromBuild` to test the acknowledgment for a message ID without an acker could succeed for a consumer that enables batch index ACK. ### TODO In future, apache/pulsar#19031 might be migrated into the C++ client to fix the consumer that disables batch index ACK.
4 tasks
BewareMyPower
added a commit
to BewareMyPower/pulsar-client-cpp
that referenced
this pull request
Jan 18, 2023
### Motivation Currently the main branch is broken by the concurrent merge of apache#153 and apache#151. ### Modifications Add a dummy `getBitSet` implementation to `BatchMessageAcker` and the correct implementation for `BatchMessageAckerImpl`.
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #87
Modifications
setBatchIndexAckEnabledto enable the batch index ACK. When it's enabled, passing the originalMessageIdinstead of theMessageIdImplthat trunks the batch index to the ACK grouping tracker.BatchedMessageIdImplcould be accepted in the ACK grouping tracker, fix the compare logic.BitSetinCommands::newAckand get the internalBitSetfromMessageIdinCommands::newMultiMessageAck.ConsumerImpl::receiveIndividualMessagesFromBatch.Verifications
Modify
BitSetTest.testSetto verify theBitSet::getmethod added in this commit.Add
AcknowledgeTest.testBatchIndexAckto test batch index ACK for all types of acknowledgment:Add
AcknowledgeTest.testMixedCumulativeAckto test the new compare logic betweenBatchedMessageIdImplandMessageIdImplworks for cumulative ACK.Documentation
doc-required(Your PR needs to update docs and you will update later)
doc-not-needed(Please explain why)
doc(Your PR contains doc changes)
doc-complete(Docs have been already added)