-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding Spark Streaming support #21
Comments
I played with this some today. My fork at https://github.com/jbtv/sparkling is working (locally) with Spark Streaming 1.3.1. I added support for direct Kafka streams, and implemented When I get around to writing tests for the streaming ns, I'll PR you. Thanks @chrisbetz |
Hi Blake, that's awesome news. As you have seen, I've (almost) no support for Spark Streaming so far. I'm just too busy with the rest. However, extending sparkling to support Spark Streaming (and Dataframes and ML) would be really good. So, I really appreciate your effort and I'm eagerly awaiting your PRs. BTW: Would you mind making PRs against the "develop" branch? This is making things a little cleaner in my opinion. Should be just a click away, anyhow ;) Cheers, Chris
|
Is there any update on bringing ML support to Sparkling? I'd love to switch Python for Clojure as my daily working environment! Keep the good work! |
Any follow-up on this? Would really enjoy a more Clojure-like interface for Spark Streaming. |
Hi @chrisbetz
Thanks for Sparkling, it's delicious. I just started working with it, and my employer Iris.tv is prepared to use it in production.
I am going to put some effort into Spark Streaming support, and I wonder if you have anything to share about the status of it. I see your note in the code that it is incomplete/experimental. I want to make use of it in production, if possible, so I'm going to try to get it to a point where we'd be comfortable doing that.
So, if you have a little time and you feel like explaining where things are (what works, what doesn't, what's left to be done), I would certainly appreciate it. If not, I will see what I can do on my own and send you PRs.
The text was updated successfully, but these errors were encountered: