Allow to export configurable set of topic configuration keys

[amuraru]

Added a new configuration parameter to allow defining the
set of topic configuration keys that should be exported as label pairs
in kafka_topic_info

By default only cleanup.policy configuration is exported

See https://kafka.apache.org/documentation/#topicconfigs
for the list of topic level configs that can be defined

[amuraru]

this is done to ensure all configurations are returned

[weeco]

Hey Adi,
we are kinda abusing the prometheus labels for that. Someone else might be interested in other topic config options and we can definetely not add all config options. Hence I thought about that we could maybe make them configurable? I'm unsure whether (and how) it is possible to make dynamic labels work with the Prometheus lib.

Basically making it configurable like this (and use a sane default there):

  topics:
    # Granularity can be per topic or per partition. If you want to reduce the number of exported metric series and
    # you aren't interested in per partition metrics you could choose "topic".
    granularity: partition
    # AllowedTopics are regex strings of topic names whose topic metrics that shall be exported.
    # You can specify allowed topics by providing literals like "my-topic-name" or by providing regex expressions
    # like "/internal-.*/".
    allowedTopics: [ ".*" ]
    # IgnoredTopics are regex strings of topic names that shall be ignored/skipped when exporting metrics. Ignored topics
    # take precedence over allowed topics.
    ignoredTopics: [ ]
    # infoMetric is a configuration object for the kminion_kafka_topic_info metric
    infoMetric:
      # ConfigKeys are regex strings of Topic configs that you want to have exported as part of the metric
      configKeys: ["max_message_bytes", "min_insync_replicas"]

What do you think?

[amuraru]

we are kinda abusing the prometheus labels for that. Someone else might be interested in other topic config options and we can definetely not add all config options. Hence I thought about that we could maybe make them configurable? I'm unsure whether (and how) it is possible to make dynamic labels work with the Prometheus lib.

makes a lot of sense, Martin!

Will amend this PR to have these configurable.
Any preference for the defaults in configKeys ? Should I keep cleanup.policy only as this is what the tool supports?

[weeco]

I think cleanup.policy only should be fine for most users. If someone wants to inspect topic configs I'd hope they would use a UI like Kowl for it where users can also see what's inherited / default etc like that:

Thanks for tackling this :)

[amuraru]

Kowl is awesome indeed!
One particular use-case for exposing these as metrics is when we want for example to track the evolution of a particular topic configurations. Say, when was the topic max.message.size changed.

[amuraru]

be lenient to individual topic failures

[amuraru]

Updated the PR based on your suggestion @weeco
Let me know when you get a chance to review

[weeco]

Generally LGTM, added a few comments though 👍 .