Unable to scale for Kafka topics with mixed-case names

[shawnplace]

Report

KEDA is unable to create a healthy trigger object for Kafka topics whose names include uppercase letters. This has been tested with multiple services and alongside topic names composed solely of lowercase characters. The all-lowercase names are reporting as “Happy” but the topics whose names are composed of mixed casing are consistently failing to initialize the trigger.

The scaled object is also showing repeated occurrences of:
- error parsing kafka metadata: no username given

This is likely inaccurate though because the topic is using the same credentials, configured exactly the same as another topic (all lowercase naming) that is currently working as intended.

Expected Behavior

The expected behavior for this would be to see successful trigger creation and scaling for Kafka topics with names that include uppercase characters.

Actual Behavior

Using nearly identical configuration (only differences being trigger name and topic name), a trigger created for a topic with a name that includes uppercase characters fails to create/scale while triggers with all lowercase characters creates/scales successfully.

This was tested against two of our services, each consuming from at least two Kafka topics where one was named using all lowercase characters and the other consisted of mixed-casing. In both services, their scaled objects acknowledged all listed triggers for Kafka topics but only the all-lowercase topics were listed as "Happy"/healthy.

Steps to Reproduce the Problem

1. Create a Kafka topic whose name includes both uppercase and lowercase characters
2. Set up and configure KEDA to scale off this topic

• Our metadata section is setup as:
  metadata:
  bootstrapServers: bootstrap-server:portNumber
  consumerGroup: test-consumer-group
  topic: dev.domain.testKafkaTopicName.qry
  lagThreshold: '5'
  tls: enable
  sasl: plaintext
  allowIdleConsumers: 'true'

3. Observe ScaledObject's "Health" section - should not being showing mixed-case topic as "Happy"

Logs from KEDA operator

example

KEDA Version

2.11.2

Kubernetes Version

1.26

Platform

Microsoft Azure

Scaler Details

Kafka - https://keda.sh/docs/2.11/scalers/apache-kafka/

Anything else?

No response

[JorTurFer]

Definitively you should be able to call your topics as you wish.
There is another experimental kafka scaler, have you tried it?

Are you willing to contribute with the fix?

[dttung2905]

Hi @shawnplace , is there any log from your side for more information ?
From the KEDA code, we don't do any topic name transformation as the topic name is taken straight from TriggerMetadata config

keda/pkg/scalers/kafka_scaler.go

Lines 378 to 387 in d39c920

	switch {
	case config.TriggerMetadata["topicFromEnv"] != "":
	meta.topic = config.ResolvedEnv[config.TriggerMetadata["topicFromEnv"]]
	case config.TriggerMetadata["topic"] != "":
	meta.topic = config.TriggerMetadata["topic"]
	default:
	meta.topic = ""
	logger.V(1).Info(fmt.Sprintf("consumer group %q has no topic specified, "+
	"will use all topics subscribed by the consumer group for scaling", meta.group))
	}

Then the input is fed into Sarama DescribeTopics method

keda/pkg/scalers/kafka_scaler.go

Lines 593 to 596 in d39c920

	topicsMetadata, err := s.admin.DescribeTopics(topicsToDescribe)
	if err != nil {
	return nil, fmt.Errorf("error describing topics: %w", err)
	}

I also did a quick check on the restriction when creating kafka topic and there is no restriction on the mixed case. https://github.com/apache/kafka/blob/85737fe88aeca12a0070cd946016b6e384e78e36/clients/src/main/java/org/apache/kafka/common/internals/Topic.java#L29-L47

public class Topic {

    public static final String GROUP_METADATA_TOPIC_NAME = "__consumer_offsets";
    public static final String TRANSACTION_STATE_TOPIC_NAME = "__transaction_state";
    public static final String CLUSTER_METADATA_TOPIC_NAME = "__cluster_metadata";
    public static final TopicPartition CLUSTER_METADATA_TOPIC_PARTITION = new TopicPartition(
        CLUSTER_METADATA_TOPIC_NAME,
        0
    );
    public static final String LEGAL_CHARS = "[a-zA-Z0-9._-]";

    private static final Set<String> INTERNAL_TOPICS = Collections.unmodifiableSet(
            Utils.mkSet(GROUP_METADATA_TOPIC_NAME, TRANSACTION_STATE_TOPIC_NAME));

    private static final int MAX_NAME_LENGTH = 249;

    public static void validate(String topic) {
        validate(topic, "Topic name", message -> {
            throw new InvalidTopicException(message);
        });
    }

Hence, I suspect there might be something wrong with the Sarama DescribeTopics method (just a hypothesis only). Let me add more granular log level so that we have more information for debugging

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[dttung2905]

Hi @shawnplace, have you manage to solve this issue ?

I tried to replicate your issue with a mix case myTopic from my testing but was not able to reproduce your issue. This is my testing with terraform and strimzi kafka. Could you look through it if you have time? https://github.com/dttung2905/stream-processing-at-home/tree/main/strimzi-keda

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed due to inactivity.