Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Fixed key #123

Merged
merged 6 commits into from
Nov 7, 2017
Merged

Support Fixed key #123

merged 6 commits into from
Nov 7, 2017

Conversation

solsson
Copy link
Contributor

@solsson solsson commented Nov 6, 2017

I use kafkacat in replicated containers, where a useful key is for example the machine that the particular container runs on, which I can get from environment variables. It makes more sense to set this key value at command invocation, than sed ing the piped data.

I remember nothing of the C we wrote in school, but I gave this a shot. If you like the idea but - as is likely - not the implementaton or the choice of flag, I'm happy to see a rewrite.

I've built this as solsson/kafkacat-fixedkey and tested it briefly for my usecases, including those with per-message keys from stdin.

Copy link
Owner

@edenhill edenhill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good.
Cache the key_len on conf and change arg to -k

kafkacat.c Outdated
@@ -350,6 +351,11 @@ static void producer_run (FILE *fp, char **paths, int pathcnt) {
}
}

if (!key && conf.fixed_key) {
key = conf.fixed_key;
key_len = (size_t)(strlen(conf.fixed_key));
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cache key_len on conf

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. Will do.

kafkacat.c Outdated
@@ -936,6 +942,7 @@ static void RD_NORETURN usage (const char *argv0, int exitcode,
" -p -1 Use random partitioner\n"
" -D <delim> Delimiter to split input into messages\n"
" -K <delim> Delimiter to split input key and message\n"
" -F <str> Use a fixed key for all messages\n"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think -k would be more intuitive

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Funny I didn't think of that alternative.

solsson added a commit to Yolean/kubernetes-kafka that referenced this pull request Nov 6, 2017
Addresses #88 (comment).

I deemed it safe to assume that operational daemonset pods never co-exist on a node.

Tests edenhill/kcat#123,
as does 53f355a.
solsson added a commit to Yolean/kubernetes-kafka that referenced this pull request Nov 6, 2017
@solsson
Copy link
Contributor Author

solsson commented Nov 7, 2017

I've been running the build from after my review fixes, and see no regressions.

@edenhill edenhill merged commit 43d91e5 into edenhill:master Nov 7, 2017
@edenhill
Copy link
Owner

edenhill commented Nov 7, 2017

Thank you for this!

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants