Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use realm import script for faster startup #3499

Closed
wants to merge 5 commits into from

Conversation

shreddedbacon
Copy link
Member

@shreddedbacon shreddedbacon commented Aug 1, 2023

Checklist

  • Affected Issues have been mentioned in the Closing issues section
  • Documentation has been written/updated
  • PR title is ready for inclusion in changelog

This changes the startup of keycloak to use a realm import script that gets keycloak to a point of the latest changes. This is only called if the lagoon realm does not currently exist. This way a fresh install of Lagoon will be up and running much quicker than having to perform all the migrations from the dawn of time.

This significantly speeds up keycloak startup time.

This currently uses a realm export of the changes done in what would be Lagoon 2.17.0, but may need to be updated periodically if there are any changes. As such, whatever version this would be merged into of Lagoon would need a release note to say that users cannot upgrade to this version, and they must upgrade to the version prior before they can update to the version this is released in. As the whole point of this is to eliminate the dawn of time migrations that take place, as these run even on existing installations just in a no-op form, but still takes time before it gets to the more current migrations.

Future migrations can be added to this though that perform migrations the same way older migrations took place, but it would mean that it could get to the same position again where the migrations take too long.

I propose that every migration also gets added/updated into to the realm import. This way the initial realm import on brand new installs will always be up to date, and the migrations can periodically be cleaned up using the same method of you need to update to version X before you update to version Y. We should also track these upgrade paths better.

@shreddedbacon
Copy link
Member Author

shreddedbacon commented Aug 1, 2023

We can probably remove the KEYCLOAK_TEST_REALM_IMPORT variable and just have it only run if the realm doesn't already exist.

Edit: I removed it.

@shreddedbacon shreddedbacon changed the title feat: add basic realm import script for faster test startup feat: add basic realm import script for faster startup Dec 12, 2023
@shreddedbacon
Copy link
Member Author

Just from local testing from running make ui-logs-development to get a minimal core running, you're looking at about 30 seconds to get keycloak from 0 to ready. Which is considerably faster than the 4-5 minutes waiting for all the migrations to take place just to get a new keycloak to a desired state.

IMAGE_REPO=lagoon docker-compose -p lagoon --compatibility up -d api api-db actions-handler local-api-data-watcher-pusher ui keycloak keycloak-db broker api-redis logs2notifications local-minio mailhog
[+] Running 14/14
<snip container status>
Tue 12 Dec 2023 20:04:03 AEDT
keycloak_1                       | Config of Keycloak done. Log in via admin user 'admin' and password 'admin'
^Ccanceled
make: *** [Makefile:315: logs] Error 130
$ date
Tue 12 Dec 2023 20:04:39 AEDT

@shreddedbacon shreddedbacon changed the title feat: add basic realm import script for faster startup feat: use realm import script for faster startup Dec 12, 2023
@shreddedbacon
Copy link
Member Author

closing for #3624

@shreddedbacon shreddedbacon deleted the keycloak-bare-startup branch September 18, 2024 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant