root: Restructure broker / cache / channel / result configuration #7097

PKizzle · 2023-10-08T00:58:47Z

This is part of #5395.

It restructures the configuration to separate Redis configuration clearly from cache and broker settings.

Furthermore, it enables custom configuration of the broker with its respective transport options, cache, channel and result backend. With this change i.e. RabbitMQ or Amazon SQS may be used as a broker.
Broker transport options need to be Base64 encoded and are ignored if not formatted correctly.

Deprecation logic has been added that automatically converts the old configuration names into the new ones and outputs a log message to encourage changing the configuration accordingly.

This is handled in goauthentik#5395

# Conflicts: # authentik/lib/config.py

This enables usage of other brokers that require additional settings

This functionality is not part of this PR

netlify · 2023-10-08T00:58:51Z

✅ Deploy Preview for authentik-storybook canceled.

Name	Link
🔨 Latest commit	`d7740e3`
🔍 Latest deploy log	https://app.netlify.com/sites/authentik-storybook/deploys/6542a8df1b5f79000875be88

netlify · 2023-10-08T01:00:24Z

✅ Deploy Preview for authentik ready!

Name	Link
🔨 Latest commit	`fecf17a`
🔍 Latest deploy log	https://app.netlify.com/sites/authentik/deploys/6521fec893c1b4000828aed7
😎 Deploy Preview	https://deploy-preview-7097--authentik.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
Lighthouse	1 paths audited Performance: 94 (no change from production) Accessibility: 90 (no change from production) Best Practices: 100 (no change from production) SEO: 80 (no change from production) PWA: - View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify site configuration.

codecov · 2023-10-08T01:03:27Z

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (a10392e) 92.57% compared to head (d7740e3) 92.42%.
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7097      +/-   ##
==========================================
- Coverage   92.57%   92.42%   -0.15%     
==========================================
  Files         587      587              
  Lines       28895    28984      +89     
==========================================
+ Hits        26749    26788      +39     
- Misses       2146     2196      +50

Flag	Coverage Δ
e2e	`49.49% <18.09%> (-1.43%)`	⬇️
integration	`25.99% <18.09%> (-0.03%)`	⬇️
unit	`89.62% <96.19%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
authentik/api/v3/config.py	`94.54% <ø> (ø)`
authentik/flows/planner.py	`91.94% <100.00%> (-5.37%)`	⬇️
authentik/lib/tests/test_config.py	`100.00% <100.00%> (ø)`
authentik/outposts/consumer.py	`91.66% <100.00%> (ø)`
authentik/policies/process.py	`100.00% <100.00%> (ø)`
authentik/policies/reputation/signals.py	`93.93% <100.00%> (ø)`
authentik/root/settings.py	`87.30% <ø> (ø)`
authentik/lib/config.py	`96.89% <90.47%> (+0.05%)`	⬆️

... and 9 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

netlify · 2023-10-08T01:03:48Z

✅ Deploy Preview for authentik ready!

Name	Link
🔨 Latest commit	`ac78edc`
🔍 Latest deploy log	https://app.netlify.com/sites/authentik/deploys/65427dbf095f6d000787b5e6
😎 Deploy Preview	https://deploy-preview-7097--authentik.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
Lighthouse	1 paths audited Performance: 95 (no change from production) Accessibility: 90 (no change from production) Best Practices: 100 (no change from production) SEO: 80 (no change from production) PWA: - View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify site configuration.

Additionally add tests

PKizzle · 2023-10-09T15:49:47Z

@BeryJu This PR is ready to be reviewed in preparation for #7118.

authentik/root/settings.py

If more customization is desired users shall look at goauthentik.io/docs/installation/configuration#custom-python-settings

PKizzle · 2023-10-27T19:11:09Z

@BeryJu Could you take a look whether there are any more changes I should make?

# Conflicts: # authentik/root/settings.py

BeryJu

Only some small nits/questions, otherwise LGTM

authentik/lib/config.py

BeryJu · 2023-11-01T12:06:48Z

website/docs/installation/configuration.mdx

+    `AUTHENTIK_CACHE__TIMEOUT_REPUTATION` only applies to the cache expiry, see [`AUTHENTIK_REPUTATION__EXPIRY`](#authentik_reputation__expiry) to control how long reputation is persisted for.
+    :::
+
+## Channel Settings


We should probably explain for the docs what "Channel" means

Suggested change

## Channel Settings

## Websocket Settings

I would like to keep the reference to Django Channel Layers as it makes it much easier to lookup should any questions arise. I have added a little explanation.

BeryJu · 2023-11-01T12:08:01Z

authentik/lib/config.py

-            for dot_part in dot_parts[:-1]:
-                if dot_part not in current_obj:
-                    current_obj[dot_part] = {}
-                current_obj = current_obj[dot_part]
            # Check if the value is json, and try to load it
            try:
                value = loads(value)


Could we try to parse and load b64 here instead of explicitly having to call get_dict_from_b64_json? Maybe not such a good idea since some values might be b64 encoded that shouldn't be decoded?

Furthermore, b64 encoded values can then only be set in env vars and not in the yaml files.

authentik/lib/config.py

For channel layer settings

authentik/lib/config.py

Signed-off-by: Jens L. <jens@beryju.org>

…authentik into feature/reorganize-redis-config

PKizzle · 2023-11-06T16:26:33Z

@BeryJu Are there any other changes you would like to see?

* main: (24 commits) internal: remove special route for /outpost.goauthentik.io (#7539) providers/proxy: Fix duplicate cookies when using file system store. (#7541) web: bump API Client version (#7543) sources/ldap: add check command to verify ldap connectivity (#7263) internal: remove deprecated metrics (#7540) core: compile backend translations (#7538) web: bump prettier from 3.0.3 to 3.1.0 in /web (#7528) web: bump @trivago/prettier-plugin-sort-imports from 4.2.1 to 4.3.0 in /web (#7531) web: bump rollup from 4.3.0 to 4.4.0 in /web (#7529) core: bump celery from 5.3.4 to 5.3.5 (#7536) web: bump @formatjs/intl-listformat from 7.5.1 to 7.5.2 in /web (#7530) web: bump prettier from 3.0.3 to 3.1.0 in /tests/wdio (#7532) web: bump @trivago/prettier-plugin-sort-imports from 4.2.1 to 4.3.0 in /tests/wdio (#7533) website: bump prettier from 3.0.3 to 3.1.0 in /website (#7534) website: bump prism-react-renderer from 2.1.0 to 2.2.0 in /website (#7535) translate: Updates for file locale/en/LC_MESSAGES/django.po in zh_TW (#7537) root: Restructure broker / cache / channel / result configuration (#7097) core: bump twilio from 8.10.0 to 8.10.1 (#7474) web: bump axios from 1.5.0 to 1.6.1 in /web (#7518) web: bump wdio-wait-for from 3.0.7 to 3.0.8 in /tests/wdio (#7514) ...

PKizzle added 7 commits October 5, 2023 19:24

Initial commit

e108d3f

Merge branch 'main' into feature/reorganize-redis-config

ef7a908

Remove any remaining mentions of Redis URL

85562e9

This is handled in goauthentik#5395

Merge branch 'main' into feature/reorganize-redis-config

17f557b

# Conflicts: # authentik/lib/config.py

Allow setting broker transport options

7559cae

This enables usage of other brokers that require additional settings

Remove remaining reference to Redis URL

fecf17a

This functionality is not part of this PR

Reset default TLS requirements to none

284a082

PKizzle requested review from a team as code owners October 8, 2023 00:58

Fix linter errors

1b49078

PKizzle changed the title ~~Initial commit~~ Restructure broker / cache / result backend configuration Oct 8, 2023

PKizzle changed the title ~~Restructure broker / cache / result backend configuration~~ root: Restructure broker / cache / result backend configuration Oct 8, 2023

PKizzle changed the title ~~root: Restructure broker / cache / result backend configuration~~ root: Restructure broker / cache / channel / result backend configuration Oct 8, 2023

PKizzle mentioned this pull request Oct 8, 2023

root: Improve Redis configuration #5395

Closed

7 tasks

PKizzle added 2 commits October 9, 2023 16:58

Move dict from base64 encoded json to config.py

9390681

Additionally add tests

Merge branch 'main' into feature/reorganize-redis-config

b378278

PKizzle mentioned this pull request Oct 9, 2023

root: Improve Redis config and support #7118

Draft

8 tasks

BeryJu reviewed Oct 9, 2023

View reviewed changes

authentik/root/settings.py Outdated Show resolved Hide resolved

authentik/root/settings.py Outdated Show resolved Hide resolved

authentik/root/settings.py Outdated Show resolved Hide resolved

PKizzle added 3 commits October 9, 2023 18:37

Replace ast.literal_eval with json.loads

f8d9d9c

Use default channel and cache backend configuration

b5181df

If more customization is desired users shall look at goauthentik.io/docs/installation/configuration#custom-python-settings

Merge branch 'main' into feature/reorganize-redis-config

8961d83

PKizzle changed the title ~~root: Restructure broker / cache / channel / result backend configuration~~ root: Restructure broker / cache / channel / result configuration Oct 9, 2023

PKizzle requested a review from BeryJu October 10, 2023 15:11

Merge branch 'main' into feature/reorganize-redis-config

cc3eafc

PKizzle added 2 commits October 27, 2023 21:12

Merge branch 'main' into feature/reorganize-redis-config

4483d53

# Conflicts: # authentik/root/settings.py

Merge branch 'main' into feature/reorganize-redis-config

14b65a1

BeryJu requested changes Nov 1, 2023

View reviewed changes

rissson reviewed Nov 1, 2023

View reviewed changes

authentik/lib/config.py Show resolved Hide resolved

PKizzle added 7 commits November 1, 2023 17:25

Send config deprecation notification to all superusers

4b5ad1d

Remove duplicate method

37f2e75

Merge branch 'main' into feature/reorganize-redis-config

03db148

Add configuration explanation

ac78edc

For channel layer settings

Use Event for deprecation warning

cb0cb28

Fix remove duplicated method

535333f

Add missing comma

037bd1f

BeryJu reviewed Nov 1, 2023

View reviewed changes

authentik/lib/config.py Outdated Show resolved Hide resolved

BeryJu approved these changes Nov 1, 2023

View reviewed changes

BeryJu and others added 3 commits November 1, 2023 20:35

Update authentik/lib/config.py

e09f59b

Signed-off-by: Jens L. <jens@beryju.org>

Fix Event deprecation handling

04a86a1

Merge branch 'feature/reorganize-redis-config' of github.com:PKizzle/…

d7740e3

…authentik into feature/reorganize-redis-config

BeryJu merged commit 9db9ad3 into goauthentik:main Nov 10, 2023

PKizzle deleted the feature/reorganize-redis-config branch November 14, 2023 03:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

root: Restructure broker / cache / channel / result configuration #7097

root: Restructure broker / cache / channel / result configuration #7097

PKizzle commented Oct 8, 2023 •

edited

Loading

netlify bot commented Oct 8, 2023 •

edited

Loading

netlify bot commented Oct 8, 2023 •

edited

Loading

codecov bot commented Oct 8, 2023 •

edited

Loading

netlify bot commented Oct 8, 2023 •

edited

Loading

PKizzle commented Oct 9, 2023

PKizzle commented Oct 27, 2023

BeryJu left a comment

BeryJu Nov 1, 2023

PKizzle Nov 1, 2023 •

edited

Loading

BeryJu Nov 1, 2023

PKizzle Nov 1, 2023

PKizzle commented Nov 6, 2023

root: Restructure broker / cache / channel / result configuration #7097

root: Restructure broker / cache / channel / result configuration #7097

Conversation

PKizzle commented Oct 8, 2023 • edited Loading

netlify bot commented Oct 8, 2023 • edited Loading

✅ Deploy Preview for authentik-storybook canceled.

netlify bot commented Oct 8, 2023 • edited Loading

✅ Deploy Preview for authentik ready!

codecov bot commented Oct 8, 2023 • edited Loading

Codecov Report

netlify bot commented Oct 8, 2023 • edited Loading

✅ Deploy Preview for authentik ready!

PKizzle commented Oct 9, 2023

PKizzle commented Oct 27, 2023

BeryJu left a comment

Choose a reason for hiding this comment

BeryJu Nov 1, 2023

Choose a reason for hiding this comment

PKizzle Nov 1, 2023 • edited Loading

Choose a reason for hiding this comment

BeryJu Nov 1, 2023

Choose a reason for hiding this comment

PKizzle Nov 1, 2023

Choose a reason for hiding this comment

PKizzle commented Nov 6, 2023

PKizzle commented Oct 8, 2023 •

edited

Loading

netlify bot commented Oct 8, 2023 •

edited

Loading

netlify bot commented Oct 8, 2023 •

edited

Loading

codecov bot commented Oct 8, 2023 •

edited

Loading

netlify bot commented Oct 8, 2023 •

edited

Loading

PKizzle Nov 1, 2023 •

edited

Loading