Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Update streams even less often #373

Open
mqus opened this issue Jul 12, 2020 · 0 comments
Open

Feature: Update streams even less often #373

mqus opened this issue Jul 12, 2020 · 0 comments
Labels
refactoring Improvement without changing functionality

Comments

@mqus
Copy link
Collaborator

mqus commented Jul 12, 2020

#361 already reduces the stream updates for Views, but also introduces dependencies for foreign key relations (as they should), but treats all dependencies equally, even if some updates will not ever propagate through those dependencies. This mostly occurs in the following two cases:

Assuming two tables: Dog and Person, with Dog having a person_id (a foreign key). If I remove a person or make an update, then I will also expect Dog to change (sometimes), according to the ForeignKeyAction. On the other hand, I expect an insert into Person not to change Dog, because there is nothing to update since no foreign key relations could have been violated by this insertion. Also, if the foreignKeyAction was RESTRICT or NO ACTION then Dog will also not be updated. The last one has to be considered individually for DELETE and UPDATE since different actions can be set.

Currently this is not considered since all dependencies are treated equally and we assume each update to Person will also affect Dog, if at least one of the update/delete actions is one that could update Dog, to simplify things.

I propose to remove those normal dependencies for foreign key relations and just let the update/delete queries set multiple affected entities, depending on the kind of interaction (delete or update) and on the table they are made on. Since those foreignKeyActions can propagate multiple times and in multiple directions we will have to build up some second kind of dependency-management for those queries. This also plays nicely with #360 because this will profit from a changeListener which is streaming Set<String>.

So to summarize the proposal, we will keep resolving dependencies for DatabaseViews and queries into a set of entities, but will not resolve further. The dependencies for Entities on Entities will be resolved for each update/delete individually and will also result into a set of "changed Entities" for each query/update/delete operation, which will be inserted into the changeListener and Streams will then have to compute if an overlap exists.

Of course, this is a different beast of effort and can only be tackled once #361 is merged (#359 would be nice ergonomically), but I would like to hear opinions on this approach. I'm also pretty sure that I'm not the first one to come up with this model but I haven't researched on how room or other ORMs do it.

PS: If I think about it, we would have to compute if the set of dependencies of the query and the set of changes overlap. If this is a performance issue, we could speed this up by making a bitmap with each entity/table getting a position and if changes & dependencies > 0 then we have an overlap. those bitmaps can be computed beforehand and because I expect normal applications to not have more than 32 tables this would be pretty space-efficient, although it would have a bit of overhead in the generator. But this is just my inner CS nerd speaking :D

@mqus mqus added feature Request and implementation of a feature (release drafter) refactoring Improvement without changing functionality and removed feature Request and implementation of a feature (release drafter) labels Jul 12, 2020
@mqus mqus mentioned this issue Aug 12, 2020
23 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
refactoring Improvement without changing functionality
Development

No branches or pull requests

1 participant