Skip to content
This repository has been archived by the owner on Apr 26, 2023. It is now read-only.

[Merged by Bors] - Add SSL support to kafka-sink #404

Closed
wants to merge 15 commits into from

Conversation

simlay
Copy link
Contributor

@simlay simlay commented Nov 4, 2022

No description provided.

@simlay simlay changed the title Simlay/rdkafka ssl sasl 2nd try Add SSL support to kafka-sink Nov 4, 2022
@simlay simlay marked this pull request as ready for review November 4, 2022 20:23
@simlay simlay force-pushed the simlay/rdkafka-ssl-sasl-2nd-try branch from fa8103e to 77aa29f Compare November 8, 2022 17:59
@@ -30,7 +30,9 @@ fluvio = { git = "https://github.com/infinyon/fluvio", branch = "mas
fluvio-smartengine = { git = "https://github.com/infinyon/fluvio", branch = "master" }
fluvio-protocol = { git = "https://github.com/infinyon/fluvio", branch = "master" }
fluvio-spu-schema = { git = "https://github.com/infinyon/fluvio", branch = "master" }
sasl2-sys = { git = "https://github.com/infinyon/rust-sasl/", branch = "fix-cross-compiling" }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we fork instead of depend on branch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean have it point to master or a revision? Yeah, that should be fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you create a PR in upstreaming repo to see if they can merge this change

@simlay simlay requested a review from morenol November 11, 2022 18:15
@@ -30,6 +30,8 @@ fluvio = { git = "https://github.com/infinyon/fluvio", branch = "mas
fluvio-smartengine = { git = "https://github.com/infinyon/fluvio", branch = "master" }
fluvio-protocol = { git = "https://github.com/infinyon/fluvio", branch = "master" }
fluvio-spu-schema = { git = "https://github.com/infinyon/fluvio", branch = "master" }
sasl2-sys = { git = "https://github.com/infinyon/rust-sasl/", branch = "fix-cross-compiling" }
krb5-src = { git = "https://github.com/infinyon/rust-krb5-src", branch = "fix-cross-compiling" }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you create a PR in upstreaming repo to see if they can merge this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kafka = "0.9.0"
# There is some difference between rev v0.26.0 and the 0.26.0 on crates.io.
# Changing it to use the crates.io version breaks the musl cross compile.
rdkafka = { git="https://github.com/fede1024/rust-rdkafka", rev="v0.26.0", features = ["cmake-build", "libz-static", "tokio", "ssl-vendored" , "gssapi-vendored"], default-features = false }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this still required?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. This is still required. fede1024/rust-rdkafka#446 (comment) it's not ideal but has to do with the openssl runtime.

Makefile Outdated Show resolved Hide resolved
Copy link
Contributor

@morenol morenol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we need integration tests, maybe we can use docker-compose for a local kafka instance. WIll approve to see if we can merge to test using a docker hub image (unable to build with macos)

@simlay
Copy link
Contributor Author

simlay commented Nov 14, 2022

bors r+

bors bot pushed a commit that referenced this pull request Nov 14, 2022
@bors
Copy link

bors bot commented Nov 14, 2022

Pull request successfully merged into main.

Build succeeded:

  • Done

@bors bors bot changed the title Add SSL support to kafka-sink [Merged by Bors] - Add SSL support to kafka-sink Nov 14, 2022
@bors bors bot closed this Nov 14, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants