[Bug]: Checked Memento "Original" Should Copy Collections #295
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Otherwise changes to the model can bleed through to the "original" dictionary
|
All tests pass in CI on P8 and manually on P10. Failure seems unrelated |
seandenigris
added a commit
to seandenigris/Magritte
that referenced
this pull request
Jan 14, 2024
Hopefully finally resolves a buffet of related issues (and PRs) mentioned below...
- The problem is that mementos were copying described values in two places:
- ```smalltalk
MACachedMemento>>cookRawPull: aDictionary
super cookRawPull: aDictionary.
aDictionary keysAndValuesDo: [ :key :value |
| isCollectionOfRelations |
isCollectionOfRelations := value isCollection and: [ key isKindOf: MAToManyRelationDescription ].
isCollectionOfRelations ifTrue: [
aDictionary at: key put: value copy ] ].```
- There are several problems with the above: Firstly, it is applied to cached mementos. This overly broad, since the motivating problem affects the `original` dictionary - something only checked mementos have. Even worse, it applies to all pulls, not just those to `original` dictionaries. This lead to a host of problems described in more detail below.
- ```smalltalk
MACheckedMemento>>reset
super reset.
self setOriginal: (self pullRawTransforming: [ :e | e copy ]).
```
- While this second copying might sometimes be what we want, it clearly does not work in some scenarios, creating the false impression that the model object has changed elsewhere, which prevents committing the memento.
- For example, {{gtMethod:MAMementoTest>>testSingletonValue}} was failing because the value was a class and the memento was storing a copying of it. Validation then failed because the installed class is not equivalent to a copy.
- Let's take a step back and review the domain model. For checked mementos, there are three versions of model state:
- 1. the model itself, which is the real *current* state
- 2. the cache, which is the desired state which the memento will attempt to push all at once on commit
- 3. the original, which is the memento's copy of the model state at the time the memento was created. This will be used before committing to make sure the model state hasn't changed elsewhere because otherwise committing might intentionally overwrite/destroy needed data.
- The motivating problem with the *original*, which inspired all this copying, is that, if a field references a collection, and we hold onto that actual collection, and the elements in the collection are changed from the outside, our "original" will also change and validation will never fail and so offers no protection.
- {{gtMethod:MACheckedMementoTest>>testValidateFailsOnReferencedCollectionChange}} tests for this. If we comment out the `#copy` in {{gtMethod:MACheckedMemento>>reset}} it will fail.
- Interestingly, outside changes also "bleed" into the cache, but it seems not to matter. If a user was worried about outside changes, they would use an {{gtClass:MACheckedMemento}} which would flag the problem during validation, not an {{gtClass:MACachedMemento}}. Choosing that memento type means we are specifically *not* checking for whether the model has changed elsewhere. Since the cache is changing with the model, there will be nothing to commit unless we explicitly change the field in the memento, which seems like the expected behavior.
- There are a bunch of seemingly related GH issues and PRs:
- 2019-09-04 - [Mementos Should Ignore Default values](magritte-metamodel#120) - claims to be fixed by the "...Copy Collections" fix below, but I don't see how
- 2022-05-31 - [Cached Memento Should Copy ToMany Collections](magritte-metamodel#281) - this first fix led to a lot of future pain. The primary mistake seems to have been that it was applied to all pulls by cached mementos, instead of just pulls to the original dictionary by checked mementos only. The issue talks about "the protection of the cache", but I think I meant "the protection of the original dictionary" because I don't see what damage could be caused by the cache changing underneath you. If you explicitly change a field, it will overwrite the cache, and if not it will be ignored because it is equivalent to the real live model state.
- 2022-08-22 - [Checked Memento "Original" Should Copy Collections](magritte-metamodel#295) - the description states "Otherwise changes to the model can bleed through to the "original" dictionary". However, the fix is applied to cached memento, not checked memento only
- 2022-08-23 - [Bloc Form To-One Tokens Pointing to Copy](magritte-metamodel#304) - tried to workaround ramifications of the overly broad copy problem fix
- 2022-08-23 - [Mementos - Only Copy Checked Original](https://github.com/magritte-metamodel/magritte/pull/305/files) attempted to improve the situation by replacing the original behavior - when pulling copy every value in every memento type - with a more targeted approach - copy only when setting the original of a checked memento. However, this still copied *all* values, not just the problematic referenced collections. It also missed the fact that cached mementos were copying collections during all pulls via `#cookPullRaw:`, not just when pulling to the original dictionary. It also seemingly undid some of the fix above
seandenigris
added a commit
that referenced
this pull request
Jan 14, 2024
* Cached Mementos - To Copy or Not To Copy?
- Hopefully finally resolves a buffet of related issues (and PRs) mentioned below
- Add Lepiter page describing the situation
- The problem is that mementos were copying described values in two places:
- ```smalltalk
MACachedMemento>>cookRawPull: aDictionary
super cookRawPull: aDictionary.
aDictionary keysAndValuesDo: [ :key :value |
| isCollectionOfRelations |
isCollectionOfRelations := value isCollection and: [ key isKindOf: MAToManyRelationDescription ].
isCollectionOfRelations ifTrue: [
aDictionary at: key put: value copy ] ].```
- There are several problems with the above: Firstly, it is applied to cached mementos. This overly broad, since the motivating problem affects the `original` dictionary - something only checked mementos have. Even worse, it applies to all pulls, not just those to `original` dictionaries. This lead to a host of problems described in more detail below.
- ```smalltalk
MACheckedMemento>>reset
super reset.
self setOriginal: (self pullRawTransforming: [ :e | e copy ]).
```
- While this second copying might sometimes be what we want, it clearly does not work in some scenarios, creating the false impression that the model object has changed elsewhere, which prevents committing the memento.
- For example, {{gtMethod:MAMementoTest>>testSingletonValue}} was failing because the value was a class and the memento was storing a copying of it. Validation then failed because the installed class is not equivalent to a copy.
- Let's take a step back and review the domain model. For checked mementos, there are three versions of model state:
- 1. the model itself, which is the real *current* state
- 2. the cache, which is the desired state which the memento will attempt to push all at once on commit
- 3. the original, which is the memento's copy of the model state at the time the memento was created. This will be used before committing to make sure the model state hasn't changed elsewhere because otherwise committing might intentionally overwrite/destroy needed data.
- The motivating problem with the *original*, which inspired all this copying, is that, if a field references a collection, and we hold onto that actual collection, and the elements in the collection are changed from the outside, our "original" will also change and validation will never fail and so offers no protection.
- {{gtMethod:MACheckedMementoTest>>testValidateFailsOnReferencedCollectionChange}} tests for this. If we comment out the `#copy` in {{gtMethod:MACheckedMemento>>reset}} it will fail.
- Interestingly, outside changes also "bleed" into the cache, but it seems not to matter. If a user was worried about outside changes, they would use an {{gtClass:MACheckedMemento}} which would flag the problem during validation, not an {{gtClass:MACachedMemento}}. Choosing that memento type means we are specifically *not* checking for whether the model has changed elsewhere. Since the cache is changing with the model, there will be nothing to commit unless we explicitly change the field in the memento, which seems like the expected behavior.
- There are a bunch of seemingly related GH issues and PRs:
- 2019-09-04 - [Mementos Should Ignore Default values](#120) - claims to be fixed by the "...Copy Collections" fix below, but I don't see how
- 2022-05-31 - [Cached Memento Should Copy ToMany Collections](#281) - this first fix led to a lot of future pain. The primary mistake seems to have been that it was applied to all pulls by cached mementos, instead of just pulls to the original dictionary by checked mementos only. The issue talks about "the protection of the cache", but I think I meant "the protection of the original dictionary" because I don't see what damage could be caused by the cache changing underneath you. If you explicitly change a field, it will overwrite the cache, and if not it will be ignored because it is equivalent to the real live model state.
- 2022-08-22 - [Checked Memento "Original" Should Copy Collections](#295) - the description states "Otherwise changes to the model can bleed through to the "original" dictionary". However, the fix is applied to cached memento, not checked memento only
- 2022-08-23 - [Bloc Form To-One Tokens Pointing to Copy](#304) - tried to workaround ramifications of the overly broad copy problem fix
- 2022-08-23 - [Mementos - Only Copy Checked Original](https://github.com/magritte-metamodel/magritte/pull/305/files) attempted to improve the situation by replacing the original behavior - when pulling copy every value in every memento type - with a more targeted approach - copy only when setting the original of a checked memento. However, this still copied *all* values, not just the problematic referenced collections. It also missed the fact that cached mementos were copying collections during all pulls via `#cookPullRaw:`, not just when pulling to the original dictionary. It also seemingly undid some of the fix above
- 2023-11-18 Checked Memento raises errors for Pier components - there is a now-passing test for just this scenario - MAMementoTest>>testSingletonValue
syrel
pushed a commit
to feenkcom/gtoolkit
that referenced
this pull request
Jan 23, 2024
Metacello new
baseline: 'GToolkitForPharo9';
repository: 'github://feenkcom/gtoolkit:v1.0.426/src';
load
All commits (including upstream repositories) since last build:
feenkcom/magritte@840ec0 by Juraj Kubelka
Merge branch 'master' into feenk
feenkcom/magritte@9077ac by Juraj Kubelka
Merge remote-tracking branch 'upstream/master'
feenkcom/magritte@a281ca by Sean DeNigris
Merge pull request #347 from seandenigris/enh_simplify-gt-column-view-api
[Enh]: GT Columned View from Description - Simplify API
feenkcom/magritte@55fdf7 by Sean DeNigris
[Enh]: GT Columned View from Description - Simplify API
Remove special method accepting a container
feenkcom/magritte@f1f43b by Sean DeNigris
Merge pull request #346 from seandenigris/bug_action-comparison2
[Bug]: action comparison
feenkcom/magritte@a45b45 by Sean DeNigris
[Bug]: action comparison
We were only looking at part of the state - species and action - which caused some actions to overshadow others. In fact, since these are mutable, we are better off sticking with identity comparison
feenkcom/magritte@7687c2 by Sean DeNigris
Cached Mementos - To Copy or Not To Copy? (#344)
* Cached Mementos - To Copy or Not To Copy?
- Hopefully finally resolves a buffet of related issues (and PRs) mentioned below
- Add Lepiter page describing the situation
- The problem is that mementos were copying described values in two places:
- ```smalltalk
MACachedMemento>>cookRawPull: aDictionary
super cookRawPull: aDictionary.
aDictionary keysAndValuesDo: [ :key :value |
| isCollectionOfRelations |
isCollectionOfRelations := value isCollection and: [ key isKindOf: MAToManyRelationDescription ].
isCollectionOfRelations ifTrue: [
aDictionary at: key put: value copy ] ].```
- There are several problems with the above: Firstly, it is applied to cached mementos. This overly broad, since the motivating problem affects the `original` dictionary - something only checked mementos have. Even worse, it applies to all pulls, not just those to `original` dictionaries. This lead to a host of problems described in more detail below.
- ```smalltalk
MACheckedMemento>>reset
super reset.
self setOriginal: (self pullRawTransforming: [ :e | e copy ]).
```
- While this second copying might sometimes be what we want, it clearly does not work in some scenarios, creating the false impression that the model object has changed elsewhere, which prevents committing the memento.
- For example, {{gtMethod:MAMementoTest>>testSingletonValue}} was failing because the value was a class and the memento was storing a copying of it. Validation then failed because the installed class is not equivalent to a copy.
- Let's take a step back and review the domain model. For checked mementos, there are three versions of model state:
- 1. the model itself, which is the real *current* state
- 2. the cache, which is the desired state which the memento will attempt to push all at once on commit
- 3. the original, which is the memento's copy of the model state at the time the memento was created. This will be used before committing to make sure the model state hasn't changed elsewhere because otherwise committing might intentionally overwrite/destroy needed data.
- The motivating problem with the *original*, which inspired all this copying, is that, if a field references a collection, and we hold onto that actual collection, and the elements in the collection are changed from the outside, our "original" will also change and validation will never fail and so offers no protection.
- {{gtMethod:MACheckedMementoTest>>testValidateFailsOnReferencedCollectionChange}} tests for this. If we comment out the `#copy` in {{gtMethod:MACheckedMemento>>reset}} it will fail.
- Interestingly, outside changes also "bleed" into the cache, but it seems not to matter. If a user was worried about outside changes, they would use an {{gtClass:MACheckedMemento}} which would flag the problem during validation, not an {{gtClass:MACachedMemento}}. Choosing that memento type means we are specifically *not* checking for whether the model has changed elsewhere. Since the cache is changing with the model, there will be nothing to commit unless we explicitly change the field in the memento, which seems like the expected behavior.
- There are a bunch of seemingly related GH issues and PRs:
- 2019-09-04 - [Mementos Should Ignore Default values](magritte-metamodel/magritte#120) - claims to be fixed by the "...Copy Collections" fix below, but I don't see how
- 2022-05-31 - [Cached Memento Should Copy ToMany Collections](magritte-metamodel/magritte#281) - this first fix led to a lot of future pain. The primary mistake seems to have been that it was applied to all pulls by cached mementos, instead of just pulls to the original dictionary by checked mementos only. The issue talks about "the protection of the cache", but I think I meant "the protection of the original dictionary" because I don't see what damage could be caused by the cache changing underneath you. If you explicitly change a field, it will overwrite the cache, and if not it will be ignored because it is equivalent to the real live model state.
- 2022-08-22 - [Checked Memento "Original" Should Copy Collections](magritte-metamodel/magritte#295) - the description states "Otherwise changes to the model can bleed through to the "original" dictionary". However, the fix is applied to cached memento, not checked memento only
- 2022-08-23 - [Bloc Form To-One Tokens Pointing to Copy](magritte-metamodel/magritte#304) - tried to workaround ramifications of the overly broad copy problem fix
- 2022-08-23 - [Mementos - Only Copy Checked Original](https://github.com/magritte-metamodel/magritte/pull/305/files) attempted to improve the situation by replacing the original behavior - when pulling copy every value in every memento type - with a more targeted approach - copy only when setting the original of a checked memento. However, this still copied *all* values, not just the problematic referenced collections. It also missed the fact that cached mementos were copying collections during all pulls via `#cookPullRaw:`, not just when pulling to the original dictionary. It also seemingly undid some of the fix above
- 2023-11-18 Checked Memento raises errors for Pier components - there is a now-passing test for just this scenario - MAMementoTest>>testSingletonValue
feenkcom/magritte@5bd298 by Sean DeNigris
Merge pull request #343 from seandenigris/issue_3478
[Bug]: GT Issue 3478 - ElementBuilder does not honor beHidden
feenkcom/magritte@145fc2 by Sean DeNigris
[Bug]: Issue 3478 - ElementBuilder does not honor beHidden
feenkcom/magritte@551825 by Sean DeNigris
Merge pull request #341 from seandenigris/gt-col-list-from-container
[Enh]: GT Columned List View from Container
feenkcom/magritte@debdd3 by Sean DeNigris
[Enh]: GT Columned List View from Container
Previously Container was auto-generated from a class and could not be modified.
feenkcom/magritte@d0eab4 by Sean DeNigris
Merge pull request #339 from seandenigris/enh_element-initial-answer
[Enh]: Element Descriptions - Initial Answer
feenkcom/magritte@23259d by Sean DeNigris
[Enh]: Element Descriptions - Initial Answer
Slightly different than `default`. Default is what the value of the field *is* if not explicitly set. The initial answer is what the value *might be* e.g. to make it easier to fill out a form.
feenkcom/magritte@ae3566 by Sean DeNigris
Merge pull request #338 from seandenigris/enh_gt-form-memento-inspect
[Enh]: GT Form - Add "Inspect Memento" Button
feenkcom/magritte@a022e0 by Sean DeNigris
[Enh]: GT Form - Add "Inspect Memento" Button
Like GT's own form
feenkcom/magritte@b24afd by Sean DeNigris
Merge pull request #336 from seandenigris/clean_remove-uid-props
[Clean]: Extract UID Property Support to Dynabook Foundation
feenkcom/magritte@2c6445 by Sean DeNigris
[Clean]: Extract UID Property Support to Dynabook Foundation
UID stuff moved out of `MATPropertyOwner`
feenkcom/magritte@dac718 by Sean DeNigris
Merge pull request #335 from seandenigris/enh_ma-print-string
[Enh]: Print via Desc - `#maFullPrintString` & Separator Config
feenkcom/magritte@bc7c2f by Sean DeNigris
[Enh]: Print via Description - `#maFullPrintString` and Separator Configuration
feenkcom/magritte@dd7f65 by Sean DeNigris
Merge pull request #333 from seandenigris/enh_property-uuid-setter
[Enh]: UUID Property Setter
feenkcom/magritte@eea847 by Sean DeNigris
[Enh]: UUID Property Setter
Trying to keep the API small, but needed this in real world use
feenkcom/magritte@96015b by Sean DeNigris
[Enh] GT columns from desc
Add columns to a gt list for each described field
feenkcom/magritte@6d67df by Jan Blizni�enko
Made loadable in Pharo 11 and 12 (#323)
* Baseline supporting Pharo 11 and 12
* Updated Grease version
* Mention Pharo 9+ in README
feenkcom/magritte@2201d8 by Juraj Kubelka
Feenk: Announce description property changes (#331)
* Add `blocStencil`to number description [#2543]
* Magritte descriptions announce property value changes [#3371]
- add `MAPropertyChangedAnnouncement`
- currently we support all properties and `MAActionDescription >> #label:` (which stored as an instance variable)
* add `MAElementDescription>>#autoAccept` [#3371]
* improve method comments [#3371]
* add `MATPropertyOwner >> #unsubscribel:` [#3378]
magritte-metamodel/magritte#331 (comment)
https://github.com/magritte-metamodel/magritte/pull/331/files/7bdfa5ea02083caf4b1257492f936bf1855f46b5#r1282553171
- magritte-metamodel/magritte#331 (comment)
- magritte-metamodel/magritte#331 (comment)
- magritte-metamodel/magritte#331 (comment)
---------
Co-authored-by: Veit Heller <veit@veitheller.de>
Co-authored-by: Andrei Chis <chisvasileandrei@gmail.com>
j-brant/SmaCC@d301a2 by John Brant
ts parser changes
b126cd by Tudor Girba
add page with editing python sources #3572
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Otherwise changes to the model can bleed through to the "original" dictionary