Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k/protocol: Checking for tag value to be 0 #11443

Conversation

michael-redpanda
Copy link
Contributor

@michael-redpanda michael-redpanda commented Jun 15, 2023

Tag fields that are primitives, and don't have assigned default values, should not be encoded if their value is equal to 0.

Fixes: #11358

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.1.x
  • v22.3.x
  • v22.2.x

Release Notes

Improvements

  • Improved efficiency in encoding tag values in kafka wire protocol

Tag fields that are primitives should not be encoded if their value is
equal to 0.

Fixes: redpanda-data#11358

Signed-off-by: Michael Boquard <michael@redpanda.com>
@michael-redpanda michael-redpanda self-assigned this Jun 15, 2023
@michael-redpanda michael-redpanda marked this pull request as ready for review June 15, 2023 01:01
@michael-redpanda michael-redpanda requested review from graphcareful, dotnwat, NyaliaLui, BenPope and twmb and removed request for graphcareful June 15, 2023 01:01
@twmb
Copy link
Contributor

twmb commented Jun 15, 2023

Slight addendum -- they should not be encoded if they are equal to their default -- frequently, the default is 0, but there can be a non-zero default.

Copy link
Contributor

@NyaliaLui NyaliaLui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks OK to me but I lack the context from the discussion you, travis, and rob had so let's get a review from them.

@NyaliaLui NyaliaLui self-requested a review June 15, 2023 13:00
NyaliaLui
NyaliaLui previously approved these changes Jun 15, 2023
@dotnwat
Copy link
Member

dotnwat commented Jun 15, 2023

Is there a unit test we can add that would reliably reproduce it before this fix?

@michael-redpanda
Copy link
Contributor Author

Slight addendum -- they should not be encoded if they are equal to their default -- frequently, the default is 0, but there can be a non-zero default.

There already is a check for that (if default is set and is equal to default, then don't encode).

@michael-redpanda
Copy link
Contributor Author

Is there a unit test we can add that would reliably reproduce it before this fix?

Yeah, maybe something simple where I create two of these messages, one with error code = 1 and one with error code = 0. The error code = 0 one should be 4 bytes smaller. It's not a 'smart' test but it's quick.

We shouldn't crash if we parse an unknown error code.

Signed-off-by: Michael Boquard <michael@redpanda.com>
Signed-off-by: Michael Boquard <michael@redpanda.com>
@michael-redpanda
Copy link
Contributor Author

Is there a unit test we can add that would reliably reproduce it before this fix?

added

Copy link
Contributor

@graphcareful graphcareful left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job ! I just left some questions/nits

@@ -196,7 +196,7 @@ std::string_view error_code_to_str(error_code error) {
case error_code::transactional_id_not_found:
return "transactional_id_not_found";
default:
std::terminate(); // make gcc happy
return "unknown_error_code";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the logic here is we should crash because if we say we support a version of a certain kafka API and we observe an unknown error code, it means there's a bug in redpanda and the handler is implemented incorrectly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I guess that is debatable though, realistically perhaps crashing is too harsh.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I guess that is debatable though, realistically perhaps crashing is too harsh.

I feel like this is an easy DoS to run against RP

src/v/kafka/protocol/tests/protocol_test.cc Show resolved Hide resolved
src/v/kafka/protocol/tests/protocol_test.cc Show resolved Hide resolved
src/v/kafka/protocol/schemata/generator.py Show resolved Hide resolved
Copy link
Member

@dotnwat dotnwat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@piyushredpanda piyushredpanda merged commit 957eab6 into redpanda-data:dev Jun 23, 2023
@vbotbuildovich
Copy link
Collaborator

/backport v23.1.x

@vbotbuildovich
Copy link
Collaborator

/backport v22.3.x

@vbotbuildovich
Copy link
Collaborator

/backport v22.2.x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[v23.1.x] CI unit-test Failure (mismatched binary data) in test_kafka_protocol_compat
7 participants