Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v0.33 ACLs with MATCH expressions are not properly applied with existing topics when using roles and TOPIC MATCH patterns #382

Closed
chrisoberle opened this issue Mar 8, 2024 · 2 comments
Closed

v0.33 ACLs with MATCH expressions are not properly applied with existing topics when using roles and TOPIC MATCH patterns #382

chrisoberle opened this issue Mar 8, 2024 · 2 comments

Comments

@chrisoberle
Copy link

chrisoberle commented Mar 8, 2024 •
edited
Loading

Describe the bug
v0.33 doesn't handle MATCH regex's correctly when using roles with existing users and topics.

To Reproduce

  1. Create test_topic.yaml:
---
apiVersion: core.jikkou.io/v1beta2
kind: ConfigMap
metadata:
  name: DefaultTopicConfig
data:
  retention.ms: 1209600000
  cleanup.policy: delete
---
apiVersion: kafka.jikkou.io/v1beta2
kind: KafkaTopicList
metadata:
  labels:
    environment: preprod
items:
  - metadata:
      name: topic.init.public
    spec:
      configMapRefs:
        - DefaultTopicConfig
  - metadata:
      name: topic.init.private
      annotations:
    spec:
      configMapRefs:
        - DefaultTopicConfig
  1. Apply test_topic.yaml and observe two topics created:
> jikkou apply -f test_topic.yaml 
TASK [CREATE] Create topic 'topic.init.private' (partitions=-1, replicas=-1, configs=[cleanup.policy=delete,retention.ms=1209600000]) - CHANGED 
{
  "end" : "2024-03-08T21:52:32.569809Z",
  "status" : "CHANGED",
  "description" : "Create topic 'topic.init.private' (partitions=-1, replicas=-1, configs=[cleanup.policy=delete,retention.ms=1209600000])",
  "changed" : true,
  "failed" : false,
  "change" : {
    "apiVersion" : "kafka.jikkou.io/v1beta2",
    "kind" : "KafkaTopicChange",
    "metadata" : {
      "name" : "topic.init.private",
      "labels" : {
        "environment" : "preprod"
      },
      "annotations" : {
        "jikkou.io/managed-by-location" : "test_topic.yaml",
        "jikkou.io/items-count" : 2
      }
    },
    "spec" : {
      "changes" : [ {
        "name" : "partitions",
        "op" : "CREATE",
        "after" : -1
      }, {
        "name" : "replicas",
        "op" : "CREATE",
        "after" : -1
      }, {
        "name" : "config.cleanup.policy",
        "op" : "CREATE",
        "after" : "delete"
      }, {
        "name" : "config.retention.ms",
        "op" : "CREATE",
        "after" : 1209600000
      } ],
      "op" : "CREATE",
      "data" : { }
    }
  }
}
TASK [CREATE] Create topic 'topic.init.public' (partitions=-1, replicas=-1, configs=[cleanup.policy=delete,retention.ms=1209600000]) - CHANGED 
{
  "end" : "2024-03-08T21:52:32.569861Z",
  "status" : "CHANGED",
  "description" : "Create topic 'topic.init.public' (partitions=-1, replicas=-1, configs=[cleanup.policy=delete,retention.ms=1209600000])",
  "changed" : true,
  "failed" : false,
  "change" : {
    "apiVersion" : "kafka.jikkou.io/v1beta2",
    "kind" : "KafkaTopicChange",
    "metadata" : {
      "name" : "topic.init.public",
      "labels" : {
        "environment" : "preprod"
      },
      "annotations" : {
        "jikkou.io/managed-by-location" : "test_topic.yaml",
        "jikkou.io/items-count" : 2
      }
    },
    "spec" : {
      "changes" : [ {
        "name" : "partitions",
        "op" : "CREATE",
        "after" : -1
      }, {
        "name" : "replicas",
        "op" : "CREATE",
        "after" : -1
      }, {
        "name" : "config.cleanup.policy",
        "op" : "CREATE",
        "after" : "delete"
      }, {
        "name" : "config.retention.ms",
        "op" : "CREATE",
        "after" : 1209600000
      } ],
      "op" : "CREATE",
      "data" : { }
    }
  }
}
EXECUTION in 4s 651ms 
ok : 0, created : 2, altered : 0, deleted : 0 failed : 0
  1. Create test_acl.yaml file
---
apiVersion: kafka.jikkou.io/v1beta2
kind: KafkaPrincipalRole
metadata:
  name: Reader
spec:
  acls:
    - resource:
        type: TOPIC
        # match everything that starts with "topic" and doesn't end in ".private"
        pattern: ^topic\..*(?<!\.private)$
        patternType: MATCH
      type: ALLOW
      operations:
        - READ
        - DESCRIBE
---
apiVersion: kafka.jikkou.io/v1beta2
kind: KafkaPrincipalAuthorization
metadata:
  name: User:testuser
spec:
  roles:
    - Reader
  1. Apply test_acl.yaml file and observe READ/DESCRIBE ACLs created for public topic
> jikkou apply -f test_acl.yaml 
TASK [CREATE] Create ACLs for principal 'User:testuser':
Create access control entry to ALLOW 'User:testuser' to execute operation(s) 'READ' on resource(s) 'TOPIC:LITERAL:topic.init.public'
Create access control entry to ALLOW 'User:testuser' to execute operation(s) 'DESCRIBE' on resource(s) 'TOPIC:LITERAL:topic.init.public' - CHANGED 
{
  "end" : "2024-03-08T21:52:54.321682Z",
  "status" : "CHANGED",
  "description" : "Create ACLs for principal 'User:testuser':\nCreate access control entry to ALLOW 'User:testuser' to execute operation(s) 'READ' on resource(s) 'TOPIC:LITERAL:topic.init.public'\nCreate access control entry to ALLOW 'User:testuser' to execute operation(s) 'DESCRIBE' on resource(s) 'TOPIC:LITERAL:topic.init.public'",
  "changed" : true,
  "failed" : false,
  "change" : {
    "apiVersion" : "kafka.jikkou.io/v1beta2",
    "kind" : "KafkaPrincipalAuthorizationChange",
    "metadata" : {
      "name" : "User:testuser",
      "labels" : { },
      "annotations" : {
        "jikkou.io/managed-by-location" : "test_acl.yaml"
      }
    },
    "spec" : {
      "changes" : [ {
        "name" : "acl",
        "op" : "CREATE",
        "after" : {
          "principal" : "User:testuser",
          "resourcePattern" : "topic.init.public",
          "patternType" : "LITERAL",
          "resourceType" : "TOPIC",
          "operation" : "READ",
          "type" : "ALLOW",
          "host" : "*"
        }
      }, {
        "name" : "acl",
        "op" : "CREATE",
        "after" : {
          "principal" : "User:testuser",
          "resourcePattern" : "topic.init.public",
          "patternType" : "LITERAL",
          "resourceType" : "TOPIC",
          "operation" : "DESCRIBE",
          "type" : "ALLOW",
          "host" : "*"
        }
      } ],
      "op" : "CREATE",
      "data" : { }
    }
  }
}
EXECUTION in 2s 826ms 
ok : 0, created : 1, altered : 0, deleted : 0 failed : 0
  1. Add a new public topic to test_topic.yaml
---
apiVersion: core.jikkou.io/v1beta2
kind: ConfigMap
metadata:
  name: DefaultTopicConfig
data:
  retention.ms: 1209600000
  cleanup.policy: delete
---
apiVersion: kafka.jikkou.io/v1beta2
kind: KafkaTopicList
metadata:
  labels:
    environment: preprod
items:
  - metadata:
      name: topic.init.public
    spec:
      configMapRefs:
        - DefaultTopicConfig
  - metadata:
      name: topic.init.private
    spec:
      configMapRefs:
        - DefaultTopicConfig
  - metadata:
      name: topic.new.public
    spec:
      configMapRefs:
        - DefaultTopicConfig
  1. Apply test_topic.yaml and observe new topic created
> jikkou apply -f test_topic.yaml
TASK [CREATE] Create topic 'topic.new.public' (partitions=-1, replicas=-1, configs=[cleanup.policy=delete,retention.ms=1209600000]) - CHANGED 
{
  "end" : "2024-03-08T21:53:32.362792Z",
  "status" : "CHANGED",
  "description" : "Create topic 'topic.new.public' (partitions=-1, replicas=-1, configs=[cleanup.policy=delete,retention.ms=1209600000])",
  "changed" : true,
  "failed" : false,
  "change" : {
    "apiVersion" : "kafka.jikkou.io/v1beta2",
    "kind" : "KafkaTopicChange",
    "metadata" : {
      "name" : "topic.new.public",
      "labels" : {
        "environment" : "preprod"
      },
      "annotations" : {
        "jikkou.io/managed-by-location" : "test_topic.yaml",
        "jikkou.io/items-count" : 3
      }
    },
    "spec" : {
      "changes" : [ {
        "name" : "partitions",
        "op" : "CREATE",
        "after" : -1
      }, {
        "name" : "replicas",
        "op" : "CREATE",
        "after" : -1
      }, {
        "name" : "config.cleanup.policy",
        "op" : "CREATE",
        "after" : "delete"
      }, {
        "name" : "config.retention.ms",
        "op" : "CREATE",
        "after" : 1209600000
      } ],
      "op" : "CREATE",
      "data" : { }
    }
  }
}
TASK [NONE] Unchanged topic 'topic.init.public' (partitions=-1, replicas=-1, configs=[cleanup.policy=delete,retention.ms=1209600000]) - OK 
{
  "end" : "2024-03-08T21:53:32.362831Z",
  "status" : "OK",
  "description" : "Unchanged topic 'topic.init.public' (partitions=-1, replicas=-1, configs=[cleanup.policy=delete,retention.ms=1209600000])",
  "changed" : false,
  "failed" : false,
  "change" : {
    "apiVersion" : "kafka.jikkou.io/v1beta2",
    "kind" : "KafkaTopicChange",
    "metadata" : {
      "name" : "topic.init.public",
      "labels" : {
        "kafka.jikkou.io/topic-id" : "W14PMzVCT7mpl1yaH5AivA"
      },
      "annotations" : {
        "kafka.jikkou.io/cluster-id" : "u20pmsXaSCmAa65MtwEPEw"
      }
    },
    "spec" : {
      "changes" : [ {
        "name" : "partitions",
        "op" : "NONE",
        "before" : -1,
        "after" : -1
      }, {
        "name" : "replicas",
        "op" : "NONE",
        "before" : -1,
        "after" : -1
      }, {
        "name" : "config.cleanup.policy",
        "op" : "NONE",
        "before" : "delete",
        "after" : "delete"
      }, {
        "name" : "config.retention.ms",
        "op" : "NONE",
        "before" : "1209600000",
        "after" : 1209600000
      } ],
      "op" : "NONE",
      "data" : { }
    }
  }
}
TASK [NONE] Unchanged topic 'topic.init.private' (partitions=-1, replicas=-1, configs=[cleanup.policy=delete,retention.ms=1209600000]) - OK 
{
  "end" : "2024-03-08T21:53:32.362833Z",
  "status" : "OK",
  "description" : "Unchanged topic 'topic.init.private' (partitions=-1, replicas=-1, configs=[cleanup.policy=delete,retention.ms=1209600000])",
  "changed" : false,
  "failed" : false,
  "change" : {
    "apiVersion" : "kafka.jikkou.io/v1beta2",
    "kind" : "KafkaTopicChange",
    "metadata" : {
      "name" : "topic.init.private",
      "labels" : {
        "kafka.jikkou.io/topic-id" : "y6PZ0FAuQFSfDNNXY1whIQ"
      },
      "annotations" : {
        "kafka.jikkou.io/cluster-id" : "u20pmsXaSCmAa65MtwEPEw"
      }
    },
    "spec" : {
      "changes" : [ {
        "name" : "partitions",
        "op" : "NONE",
        "before" : -1,
        "after" : -1
      }, {
        "name" : "replicas",
        "op" : "NONE",
        "before" : -1,
        "after" : -1
      }, {
        "name" : "config.cleanup.policy",
        "op" : "NONE",
        "before" : "delete",
        "after" : "delete"
      }, {
        "name" : "config.retention.ms",
        "op" : "NONE",
        "before" : "1209600000",
        "after" : 1209600000
      } ],
      "op" : "NONE",
      "data" : { }
    }
  }
}
EXECUTION in 4s 215ms 
ok : 2, created : 1, altered : 0, deleted : 0 failed : 0
  1. Apply same test_acl.yaml file and observe no ACLs created for new topic
> jikkou apply -f test_acl.yaml
EXECUTION in 3s 37ms 
ok : 0, created : 0, altered : 0, deleted : 0 failed : 0

In 0.32 ACLs for existing topics are applied, in 0.33 they are not.

Expected behavior
ACLs that match a MATCH pattern should be applied for existing topics.

Screenshots/Configs
If applicable, add screenshots to help explain your problem.

Runtime environment

  • OS: MacOS Sonoma 14.3
  • Jikkou: 0.33
  • Docker version: 20.10.24, build 297e128 (however, note that are using the native build locally and create our own container for CI)
  • Kafka Cluster Version: 2.8.2

Additional context
@fhussonnois
Copy link
Member

Hi @chrisoberle, thank you for reporting that issue. This should be fixed in the bugfix release v0.33.2.

@chrisoberle
Copy link
Author

Thanks @fhussonnois! We are in the process of updating now. Everything looks good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@chrisoberle @fhussonnois