Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ksqlDB] when using ACLs configure all internal topics with ALL permissions for the ksql server user #433

Merged
merged 7 commits into from
Dec 30, 2021

Conversation

solita-juusoma
Copy link
Contributor

  • Please check if the PR fulfills these requirements
  • The commit messages are descriptive
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • An issue has been created for the pull requests. Some issues might require previous discussion.
  • What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
    JulieOps is not giving enought ACLs to ksql user. Therefore ksql is not working.

Bug fix: Add missing KSQL ACLs (KSQL ACLs missing #432)

  • What is the current behavior? (You can also link to an open issue here)
    When we tried to use ksql we end up getting following error:
    [2021-12-23 13:44:54,963] INFO Principal = User:ksql_test is Denied Operation = Describe from host = 123.123.123.123 on resource = Topic:LITERAL:_confluent-ksql-hantti_db_configs for request = Metadata with resourceRefCount = 1 (kafka.authorizer.logger)

This means that JulieOps is not giving access to required internal topic Config Topic. See ksql documentation: https://github.com/confluentinc/ksql/blob/master/docs/operate-and-deploy/how-it-works.md#config-topic

'In headless mode, you supply SQL statements to each server in its SQL file. But ksqlDB still needs to store some internal metadata to ensure that it builds queries compatibly across restarts and upgrades. ksqlDB stores this metadata in an internal topic called the config topic. ksqlDB names the config topic _confluent-ksql-_configs, where is the value in the ksql.service.id property.'

  • What is the new behavior (if this is a feature change)?
    JulieOps will have ACLs that are said in ksqldb documentation.

ksqlDB always requires the following ACLs for its internal operations and data management:

  • The DESCRIBE_CONFIGS operation on the CLUSTER resource type.
  • The ALL operation on all internal TOPICS that are PREFIXED with _confluent-ksql-<ksql.service.id>.
  • The ALL operation on all internal GROUPS that are PREFIXED with _confluent-ksql-<ksql.service.id>.

See documentation: https://docs.ksqldb.io/en/latest/operate-and-deploy/installation/server-config/security/#required-acls

Therefore we need to change method that only introduces topic _confluent-ksql-%s_command_topic:

public String commandTopic() {
return String.format("_confluent-ksql-%s_command_topic", ksqlDbId);
}

to more common topic _confluent-ksql-%s method:

public String internalTopics() {
return String.format("_confluent-ksql-%s", ksqlDbId);
}

This way ksql user will have ACL ALL on all internal TOPICS that are PREFIXED with _confluent-ksql-<ksql.service.id>.

  • Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
    No.

  • Other information:

IMPORTANT: Please review the CONTRIBUTING.md file for detailed contributing guidelines.
IMPORTANT: Your pull request MUST target master.

PLEASE REMOVE THIS TEMPLATE BEFORE SUBMITTING

@solita-juusoma solita-juusoma changed the title [WiP] Fix/ksql internal topic acls Fix/ksql internal topic acls Dec 29, 2021
@solita-juusoma
Copy link
Contributor Author

solita-juusoma commented Dec 29, 2021

@purbon I am not that familiar with RBAC stuff since we are not using it but did changes to there too cause I think similar ACLs are needed.

Copy link
Collaborator

@purbon purbon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,
thanks a lot for your contribution. I have shared a few comments, let me know what do you think? honestly, I think the part that needs more work is the RBAC (as you mention you have no experience with it.

You can find the reference docs for RBAC https://docs.confluent.io/platform/current/security/rbac/ksql-rbac.html, happy to amend any error or improvement possible.

Thanks a lot for your contribution! this tools is great because of all of you!

@purbon purbon self-assigned this Dec 29, 2021
@purbon
Copy link
Collaborator

purbon commented Dec 30, 2021

LGTM, thanks a lot for your contribution!

@purbon purbon changed the title Fix/ksql internal topic acls [ksqlDB] when using ACLs configure all internal topics with ALL permissions for the ksql server user Dec 30, 2021
@purbon purbon merged commit 6668390 into kafka-ops:master Dec 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants