Externalizable source of truth for roles and attributes

[twobeeb]

Refactoring of the Security layer to accomodate for external security configuration

PR Checklist

• Technical design (already discussed with @tchiotludo)
• Implementation
• Exception Handling
• Cleanup dead code (most notably UserGroupUtils)
• Test coverage existing/default implementation (config based)
• Test coverage on new functionalities
• Documentation

PR Description

In my use case, the source of truth which grants AKHQ users read access to certain topics and connects is located in a fast moving database and because of that, AKHQ configuration application.yml must be regenerated more than once per day which is not ideal.

With this PR, I propose 3 implementations to map the user and users' groups into AKHQ roles and attributes.

Default configuration-based
This is the current implementation and the default one (doesn't break compatibility)

akhq:
  security:
    default-group: no-roles
    groups:
      reader:
        roles:
          - topic/read
        attributes:
          topics-filter-regexp: [".*"]
      no-roles:
        roles: []
    ldap: # LDAP users/groups to AKHQ groups mapping
    oidc: # OIDC users/groups to AKHQ groups mapping

REST API

akhq:
  security:
    default-group: no-roles
    rest:
      enabled: true
      url: https://external.service/get-roles-and-attributes
    groups: # anything set here will not be used

In this mode, AKHQ will send to the akhq.security.rest.url endpoint a POST request with the following JSON :

{
  "providerType": "LDAP or OIDC or BASIC_AUTH",
  "providerName": "OIDC provider name (OIDC only)",
  "username": "user",
  "groups": ["LDAP-GROUP-1", "LDAP-GROUP-2", "LDAP-GROUP-3"]
}

and expect the following JSON as response :

{
  "roles": ["topic/read", "topic/write", "..."],
  "attributes": 
  {
    "topics-filter-regexp": [".*"],
    "connects-filter-regexp": [".*"],
    "consumer-groups-filter-regexp": [".*"]
  }
}

Groovy API

akhq:
  security:
    default-group: no-roles
    groovy:
      enabled: true
      file: |
        package org.akhq.utils;
        class GroovyCustomClaimProvider implements ClaimProvider {
            @Override
            AKHQClaimResponse generateClaim(AKHQClaimRequest request) {
                AKHQClaimResponse a = new AKHQClaimResponse();
                a.roles = ["topic/read"]
                a.attributes = [
                        topicsFilterRegexp: [".*"],
                        connectsFilterRegexp: [".*"],
                        consumerGroupsFilterRegexp: [".*"]
                ]
                return a
            }
        }
    groups: # anything set here will not be used

akhq.security.groovy.file must be a groovy class that implements the interface ClaimProvider :

package org.akhq.utils;
public interface ClaimProvider {

    AKHQClaimResponse generateClaim(AKHQClaimRequest request);
    
    class AKHQClaimRequest{
        ProviderType providerType;
        String providerName;
        String username;
        List<String> groups;
    }
    class AKHQClaimResponse {
        private List<String> roles;
        private Map<String,Object> attributes;
    }
    enum ProviderType {
        BASIC_AUTH,
        LDAP,
        OIDC
    }
}

[twobeeb]

Failed tests seems unrelated to this PR

Some tests are never successful on my computer such as ConsumerGroupRepositoryTest list() (expected 6 was 11)

2021-04-21T13:34:49.8092851Z   org.akhq.controllers.SchemaControllerTest
2021-04-21T13:34:49.8094110Z     ✔ deleteNotExistApi()
2021-04-21T13:34:50.1087951Z     ✔ crud()
2021-04-21T13:34:50.5100508Z     ✘ listApi()
2021-04-21T13:34:50.5122285Z       org.opentest4j.AssertionFailedError: expected: <5> but was: <3>
....
....
2021-04-21T13:36:58.8092042Z   org.akhq.repositories.ConsumerGroupRepositoryTest
2021-04-21T13:36:58.8093412Z     ✔ search()
2021-04-21T13:36:58.8094154Z     ✔ listWithConsumerGroupRegex()
2021-04-21T13:36:58.9096872Z     ✘ list()
2021-04-21T13:36:58.9097971Z       org.opentest4j.AssertionFailedError: expected: <6> but was: <11>

@tchiotludo what do you think ?

[tchiotludo]

There is flakky test on connect part.
I restart them 🤞

[twobeeb]

So I switched from local WSL to Azure Ubuntu build :
dev branch

141 tests completed, 1 failed, 4 skipped
There were failing tests. See the report at: file:///home/foundation/akhq/build/reports/tests/test/index.html

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 5m 27s

this feature branch :

146 tests completed, 1 failed, 4 skipped
There were failing tests. See the report at: file:///home/foundation/akhq/build/reports/tests/test/index.html

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 4m 22s

In both branches the failing test is :

  org.akhq.controllers.SchemaControllerTest

    ✔ deleteNotExistApi() (1.9s)
    ✔ crud()
    ✘ listApi()

      org.opentest4j.AssertionFailedError: expected: <5> but was: <3>

But the build is successful (thanks to org.gradle.test-retry I assume)

[twobeeb]

@tchiotludo I believe it's ready for review.

Documentation is mostly a copy of this PR inside the README.md. I also added a small warning regarding the behavior of the attributes, since they only apply during read/list operation, and are not checked during insert/create. AKHQ users should know this.

Some notes on changes I've made :

• UserGroupUtils renamed to DefaultGroupsUtils since it is only ever used in the 3 Controller to obtain the roles and attributes of akhq.security.default-group
• UserGroupUtilsTest removed as it was there to show that attributes can be List<String> or String. Since all the code is now built around List<String> and some tests are included elsewhere, this UT is obsolete.
• RestApiClaimProvider doesn't have tests. I tried creating a dedicated Controller just for the test but without success.

[tchiotludo]

works perfectly !!!
Thanks for the hard work here 👍