-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Require UUIDv7 for serviceowner-provided ids #969
Comments
Refinement: There are serious concerns about index fragmentation and performance issues stemming from spamming the b-tree index with UUIDv4 values from Altinn 3/2 instances. This will probably negatively affect the index performance, even if we at a later stage introduce v7 requirements. We work to have Altinn 3 apps start using UUIDv7 as their PKs. In order to support historic instances, they will need to introduce as "dialogId" column containing the UUIDv7 value. This should be pre-generated and stored before transferring in order to ensure idempotence and avoid duplications. This change is to be considered breaking, and will have to be communicated with service providers. |
<!--- Provide a general summary of your changes in the Title above --> ## Description This adds a requirement for user-supplied IDs to be UUIDv7, and that the timestamp on these are in the past. While implementing this I stumbled over a bug where the old way of creating IDs meant they were sorted in the wrong order because GUIDs in .NET are little endian. (Sort by ID ascending would have the newest at the top, and oldest at the bottom) The idea with UUIDv7 is that the first 6 bytes is the timestamp of creation, and this helps us avoid a fragmented index There will only be appends on the index b-tree when creating new entities, and no inserts further up the index. If we were to allow old UUIDv4 for the legacy archive imports, the index would forever be fragmented and have a performance hit. We now require that all incoming IDs are V7 and big endian (this is the standard in most other languages) Sorting by ID ascending on f.ex. DialogID should now list out oldest at the top, and newest at the bottom. ## Related Issue(s) - #969 ## Verification - [ ] **Your** code builds clean without any errors or warnings - [ ] Manual testing done (required) - [ ] Relevant automated test added (if you find this hard, leave it and we'll help out) ## Documentation - [ ] Documentation is updated (either in `docs`-directory, Altinnpedia or a separate linked PR in [altinn-studio-docs.](https://github.com/Altinn/altinn-studio-docs), if applicable)
Introduction
This change introduces a UUIDv7 requirement for dialogIds, transmissionIds and activityIds
Description
To avoid fragmentation issues caused by the inherit randomness of UUIDv4 values in b-tree indexes, we will require all self-supplied IDs to be UUIDv7, which are stored sequentially and lets eg. joins be perfomed better.
Implementation
A valication routine will have to at a minimum check that the correct version bits are set. We need to consider if the timestamp part (the first 48 bits) should be validated somehow. Only accepting values within a small window relative to server time will not work, as items might be been generated on the server provider side much earlier. A more lax validation (eg. must be after 2010 and not in future) might be more suitable.
Tasks
Threat modelling
The text was updated successfully, but these errors were encountered: