Use chunked_fifo to retrieve deltas from controller_backend
#11691
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ztlpn
reviewed
Jun 27, 2023
mmaslankaprv
force-pushed
the
fix-11673
branch
3 times, most recently
from
d1ea71c to
7928552
Compare
|
compilation failed. |
bharathv
previously approved these changes
Jul 3, 2023
| @@ -2758,9 +2758,14 @@ admin_server::get_reconfigurations_handler(std::unique_ptr<ss::http::request>) { | |||
| reconfiguration_states.error()), | |||
| ss::http::reply::status_type::service_unavailable); | |||
| } | |||
| // we are forced to use shared pointer as underlying chunked_fifo is not | |||
There was a problem hiding this comment.
I'm wondering what is forcing a copy in the first place, all arguments are moved in the underlying stream function. smells like a missing mutable somewhere and the const ness is forcing a copy. 🤔
Signed-off-by: Michal Maslanka <michal@redpanda.com>
Deltas were previously stored as a vector per `ntp`. Deltas access pattern (iteration, inserting and popping elements from back and from the end) makes it perfect candidate for `std::list` usage. The `std::deque` doesn't use large contiguous allocation so will not account for the memory fragmentation. Signed-off-by: Michal Maslanka <michal@redpanda.com>
Using `ss::chunked_fifo` to return deltas processed by controller backend. Previously used `std::vector` may lead to large allocations as it allocated large chunks of contiguous memory. Fixes: redpanda-data#11673 Signed-off-by: Michal Maslanka <michal@redpanda.com>
bharathv
approved these changes
Jul 7, 2023
|
/ci-repeat 1 |
|
/backport v23.1.x |
|
Failed to run cherry-pick command. I executed the commands below: |
BenPope
added a commit
to BenPope/redpanda
that referenced
this pull request
Oct 2, 2023
Operator added in redpanda-data#11691 Signed-off-by: Ben Pope <ben@redpanda.com>
7 tasks
vbotbuildovich
pushed a commit
to vbotbuildovich/redpanda
that referenced
this pull request
Oct 3, 2023
Operator added in redpanda-data#11691 Signed-off-by: Ben Pope <ben@redpanda.com> (cherry picked from commit a2b9966)
BenPope
added a commit
to BenPope/redpanda
that referenced
this pull request
Oct 3, 2023
Operator added in redpanda-data#11691 Signed-off-by: Ben Pope <ben@redpanda.com> (cherry picked from commit a2b9966)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using
ss::chunked_fifoto return deltas processed by controllerbackend. Previously used
std::vectormay lead to large allocations asit allocated large chunks of contiguous memory.
Fixes: #11673
Backports Required
Release Notes