[Bug] ConsistentHashingStickyKeyConsumerSelector's getConsumerKeyHashRanges doesn't return correct results

[lhotari]

Search before asking

• I searched in the issues and found nothing similar.

Read release policy

• I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

Version

all versions

Minimal reproduce step

After the last entry in the hash ring, it wraps around to the first entry:

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ConsistentHashingStickyKeyConsumerSelector.java

Lines 123 to 125 in 7e73967

	} else {
	consumerList = hashRing.firstEntry().getValue();
	}

This is ignored in the getConsumerKeyHashRanges method:

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ConsistentHashingStickyKeyConsumerSelector.java

Lines 138 to 145 in 7e73967

	int start = 0;
	for (Map.Entry<Integer, List<Consumer>> entry: hashRing.entrySet()) {
	for (Consumer consumer: entry.getValue()) {
	result.computeIfAbsent(consumer, key -> new ArrayList<>())
	.add(Range.of(start, entry.getKey()));
	}
	start = entry.getKey() + 1;
	}

Another problem is that the rule to pick one consumer when there are hash collisions is also ignored:

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ConsistentHashingStickyKeyConsumerSelector.java

Line 127 in 7e73967

return consumerList.get(hash % consumerList.size());

What did you expect to see?

the results returned by the getConsumerKeyHashRanges should be accurate and correct

What did you see instead?

the getConsumerKeyHashRanges logic

• ignores the "hash ring" wrap around after the last entry
• ignores the logic for choosing a single entry when there's a hash collision
  • this is a conflicting goal currently since for hash collisions, the target consumer is selected based on the input hash
    • for hash ranges with multiple consumers, it would be necessary to scan the complete range to find out the actual hash ranges for a consumer

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[lhotari]

This will be included in #23327 since the implementation was needed for tests in that PR.

[lhotari]

Issue #16654 is related.