[WIP] Allow usage of the topic name extractor in the to function

[Akeboshiwind]

Some approaches to this would be:

• Allow :topic-name to be either a string or a clojure function and dispatch based on that
• Add a new key (say :topic-extractor) to be added to the topic-config map and possible throw an exception when both :topic-name and :topic-extractor are supplied.
• Add a new arity to the to! function

The approach I've taken is the first option because I think the second and third options could be potentially confusing to use with two separate ways of specifying the topic name (although throwing an exception would probably help with that).

If you'd prefer another approach then I don't mind making the changes needed, I appreciate that this is mostly just a matter of taste :)
(FYI I'm Oliver Marshall from the slack channel)

[DaveWM]

hi @Akeboshiwind, thanks for the PR! I don't fully understand the need for a topic name extractor, what's your use case for this? In the example you've added to the tests, I think you can achieve the same result by:

1. Using branch to split the kstream into 2 kstreams, based on the record key
2. Calling to! on each of the resulting kstreams, sinking them to their respective topics

I may have missed something here though, is there any situation where that wouldn't work?

[Akeboshiwind]

Hey @DaveWM, to be honest I opened this ticket so long ago that it's not really useful to me anymore.

But even so, my original reason was mainly because of an undocumented feature of the to! function which uses the admin api to create a topic automatically if one doesn't exist.

One example use of this would be a 'message demultiplexer' of sorts where you have one topic with lots of messages to be sorted into other topics depending on their contents, and you don't have a finite list of topics these messages need to be sorted into.

[kopos]

Is it too late to request for this feature?

I have a KStream where events from all rooms are sent into. Now I want to log the events of a single room into it's own topic - which acts like a CQRS event stream for that room.

By having the TopicNameExtractor fn as the arg to to instead would fit perfectly for my case

[Dangercoder]

Hi, I also need this feature.

I have a KStream where I need to decide the output topic dynamically.

[dakra]

Friendly bump as development of jackdaw seems to have picked up again.

This is a feature I could use as well.