fix: select cluster controller for certain actions #60
Conversation
crepererum
force-pushed
the
crepererum/controller_handling
branch
from
1f47ffa
to
2d70046
Compare
crepererum
force-pushed
the
crepererum/controller_handling
branch
from
2d70046
to
7ac1d80
Compare
src/client/controller.rs
Outdated
| match error { | ||
| Error::Request(RequestError::Poisoned(_) | RequestError::IO(_)) | ||
| | Error::Connection(_) => self.invalidate_cached_controller_broker().await, | ||
| Error::ServerError(ProtocolError::LeaderNotAvailable, _) => {} |
There was a problem hiding this comment.
Does it make sense to retry these variants here?
There was a problem hiding this comment.
Oh, you're right. Let me prune that list.
| )) | ||
| })?; | ||
|
|
||
| *current_broker = Some(Arc::clone(&broker)); |
There was a problem hiding this comment.
Does this have the same derp as partition leaders where brokers can be controller but not realise they are? I presume not??
There was a problem hiding this comment.
I think a desync could only lead NotController (I hope Kafka is checking this before checking the content of the operation). Let's see if we can some weird errors and then we can add another check.
|
|
||
| let topics = client.list_topics().await.unwrap(); | ||
| // might take a while to converge |
There was a problem hiding this comment.
We could avoid this by also performing this operation (list topics) on the control I guess, but to some extend you have to deal w/ eventual consistency because all read operations (cluster-wide like "list topics" and partition-wide like "get watermark") can always be answered by followers / non-controllers and why is leader/controller might change between our check / connection creation and the actual read request.
There was a problem hiding this comment.
Filed #63 so we can think about it.
We must not use some arbitrary broker for adding/removing topics. All brokers can read though (w/ eventual consistency).
1s is quite long, esp. when users expect rather low-latency operations. It's also a bit of a pain for our tests because a single error (like `NotAvailable`) can delay tests by a whole second easily leading to timeouts.
crepererum
force-pushed
the
crepererum/controller_handling
branch
from
b4e34b7
to
1948b1d
Compare
crepererum
force-pushed
the
crepererum/controller_handling
branch
from
1948b1d
to
48ced12
Compare
We must not use some arbitrary broker for adding/removing topics.
All brokers can read though (w/ eventual consistency).