Inconsistent match when percentage rollout and match all rules are used together

[poksme]

Describe the bug
When requesting a feature flag, with the same request entityId and payload and given two rules:

1. Percentage rollout over a specific segment -> 50% released 50% unreleased
2. Match all -> unreleased

We get inconsistent match and results over sequential requests.

Version Info
0.10.0

To Reproduce
On a fresh install

1. Create and enable a new feature flag
2. Create two variants released and unreleased
3. Create a segment subscribers that checks "premium_user": "true"
4. Create a segment all users that match all
5. In the previously created feature flag, setup two rules (in this order):
   1. If Matches segment subscribers then serve percentage rollout -> 50% released 50% unreleased
   2. If Matches segment all users then serve unreleased

Expected behavior
This is a legitimate use case where we want to progressively roll-out a feature to premium users while keeping it disabled for all others.

The expected behaviour when requesting this feature flag as a subscriber (i.e. "premium_user": "true") with the same payload is to have a consistent match on the first rule.

But currently, with the exact same payload (i.e. same entityId and context) we get randomly matched on first or second rule, effectively returning an inconsistent variant released or unreleased for one given premium user.

Screenshots

Request A -> Match Percentage Roll out (rule 1)

Request B -> Match all (rule 2)

Please note the inconsistent value of fields segmentKey and value in the response.

Additional context

• Database used: Postgres

[poksme]

When going through the source code looking for the possible fall-through condition I came upon this condition: https://github.com/markphelps/flipt/blob/b593d55f48d8a484e9913cfad4597df0c64fe370/storage/evaluator.go#L309-L312

I can try debugging more later this week.

[markphelps]

Thanks for finding and reporting this. I can also take a look more tomorrow and try to debug.

[markphelps]

@poksme FYI I was able to reproduce this with a failing test case: https://github.com/markphelps/flipt/pull/168/files#diff-b528e409c9ecda759b1e8d7d7dbd27feR454

Still working on debugging/fixing

[poksme]

Looks great!

I was able to re-produce similar issue easily with 2 match all rules setup. You will randomly match one or the other for the same entityId.

[markphelps]

@poksme this should be fixed in the latest release. Thanks again for finding it and filing the issue!