-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding output with kafka has SSL security #1619
Conversation
Hi @mostafa. Thanks |
Thanks for making this pull request! I skimmed it and didn't spot any obvious issues, but a detailed review will probably have to wait until next week, sorry. This week we're focused on wrapping up the planned changes for k6 v0.28.0, which is going to be released early next week, if there are no issues. We'll review the PR after the release, so it should become a part of k6 v0.29.0, scheduled for mid-November. |
Hey @ducmeit1, Good to see you back. Could you please address linting issues related to your code in here before my test/review: |
Wow, you are fast! Please test your code against Go 1.15.1: https://app.circleci.com/pipelines/github/loadimpact/k6/723/workflows/05569c46-67ba-4992-adc2-591a2423962e/jobs/12897 BTW, I'm not sure if this is related to your changes. |
You're welcome! Hope this repository will be passed and releasing asap. |
I've used https://github.com/lensesio/fast-data-dev to setup a Kafka dev-env and used this guide to transform trust-store and client JKS files into PEM files. Yet, I still get an error while running this command: $ k6 run --out kafka=brokers=127.0.0.1:9093,topic=k6-output,format=json,tls_security=true,certificate=client.cer.pem,private_key=client.key.pem,certificate_authority=server.cer.pem,insecure_skip=false test.js The error: ERRO[0001] kafka: client has run out of available brokers to talk to (Is your cluster reachable?) Using kafkatool, I've confirmed that JKS files work as expected, so the $ openssl s_client -debug -connect localhost:9093 -tls1 Also, the unencrypted channel works well on Can you elaborate more on your testing environment? |
Could you set insecure_skip=true and retry |
Tried that, too. No luck! |
@ducmeit1 I would like to suggest not forcing to enable mTLS , in my case we are using Kafka with SSL only to cipher the communication not for authenticating the client, for this we are using ACLs. |
Thanks for all the work done here, but given that we are:
I will close this PR and ask that if you want to add this functionality to do so in the extension repo. |
In century, K6 has supported sending the raw output format to Kafka Clusters. However, mostly Kafka usually secured by an authentication method. The Kafka Cluster usually protected by SSL. It requires the client must provide a certificate that was signed by Kafka Cluster to produce the data or consume.
This PR would like to improve the Kafka producer client with Sarama and supports SSL authentication.
Sample usage:
Parameter requires: