Fix to #31100 - Switch to storing enums as ints in JSON instead of strings #31317

maumar · 2023-07-20T09:07:30Z

Problem was that in 7.0 we stored enums inside JSON as strings by default (by applying EnumToStringConverter by convention), but in 8.0 we are changing this to int. This is a breaking change and it's extra problematic for databases that used EF JSON functionality in 7.0. This can easily create a scenario where there is a mix of string and int representation for an enum value within the same document. (some data was added in 7.0, and then some in 8.0 before customer realized that the breaking change has been made). To mitigate this we are adding a fallback mechanism when reading enum data that is part of JSON entity. We try to read as int and if that fails we try to read again as string. This way should minimize the disruption, moreover any data saved back to the database will be saved in the new format, so over time everything should normalize. We will still throw when projecting individual enum properties of a JSON entity (as opposed to the entire entity), because materialization for this goes through different path, indistinguishable from normal enum value read from column in relational table.

Fixes #31100

...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs

test/EFCore.Relational.Specification.Tests/Query/JsonQueryAdHocTestBase.cs

test/EFCore.SqlServer.FunctionalTests/Query/JsonQueryAdHocSqlServerTest.cs

ajcvickers · 2023-08-07T15:47:16Z

@AndriySvyryd @roji Took this over and updated to warn like we discussed.

src/EFCore.Relational/Metadata/Conventions/RelationalMapToJsonConvention.cs

AndriySvyryd · 2023-08-07T23:20:43Z

test/EFCore.SqlServer.FunctionalTests/Update/JsonUpdateSqlServerTest.cs

-            """
-@p0='[{"Date":"2101-01-01T00:00:00","Enum":"Two","Enums":[0,-1,1],"Fraction":10.1,"NullableEnum":"One","NullableEnums":[null,-1,1],"OwnedCollectionLeaf":[{"SomethingSomething":"e1_r_c1_c1"},{"SomethingSomething":"e1_r_c1_c2"}],"OwnedReferenceLeaf":{"SomethingSomething":"e1_r_c1_r"}},{"Date":"2102-01-01T00:00:00","Enum":"Three","Enums":[0,-1,1],"Fraction":10.2,"NullableEnum":"Two","NullableEnums":[null,-1,1],"OwnedCollectionLeaf":[{"SomethingSomething":"e1_r_c2_c1"},{"SomethingSomething":"e1_r_c2_c2"}],"OwnedReferenceLeaf":{"SomethingSomething":"e1_r_c2_r"}},{"Date":"2010-10-10T00:00:00","Enum":"Three","Enums":null,"Fraction":42.42,"NullableEnum":null,"NullableEnums":null,"OwnedCollectionLeaf":[{"SomethingSomething":"ss1"},{"SomethingSomething":"ss2"}],"OwnedReferenceLeaf":{"SomethingSomething":"ss3"}}]' (Nullable = false) (Size = 808)
+"""
+@p0='[{"Date":"2101-01-01T00:00:00","Enum":1,"Enums":[0,-1,1],"Fraction":10.1,"NullableEnum":0,"NullableEnums":[null,-1,1],"OwnedCollectionLeaf":[{"SomethingSomething":"e1_r_c1_c1"},{"SomethingSomething":"e1_r_c1_c2"}],"OwnedReferenceLeaf":{"SomethingSomething":"e1_r_c1_r"}},{"Date":"2102-01-01T00:00:00","Enum":2,"Enums":[0,-1,1],"Fraction":10.2,"NullableEnum":1,"NullableEnums":[null,-1,1],"OwnedCollectionLeaf":[{"SomethingSomething":"e1_r_c2_c1"},{"SomethingSomething":"e1_r_c2_c2"}],"OwnedReferenceLeaf":{"SomethingSomething":"e1_r_c2_r"}},{"Date":"2010-10-10T00:00:00","Enum":2,"Enums":null,"Fraction":42.42,"NullableEnum":null,"NullableEnums":null,"OwnedCollectionLeaf":[{"SomethingSomething":"ss1"},{"SomethingSomething":"ss2"}],"OwnedReferenceLeaf":{"SomethingSomething":"ss3"}}]' (Nullable = false) (Size = 784)


Do we have an issue tracking setting this size to something that doesn't pollute the query cache?

…rings Problem was that in 7.0 we stored enums inside JSON as strings by default (by applying EnumToStringConverter by convention), but in 8.0 we are changing this to int. This is a breaking change and it's extra problematic for databases that used EF JSON functionality in 7.0. This can easily create a scenario where there is a mix of string and int representation for an enum value within the same document. (some data was added in 7.0, and then some in 8.0 before customer realized that the breaking change has been made). To mitigate this we are adding a fallback mechanism when reading enum data that is part of JSON entity. We try to read as int and if that fails we try to read again as string. This way should minimize the disruption, moreover any data saved back to the database will be saved in the new format, so over time everything should normalize. We will still throw when projecting individual enum properties of a JSON entity (as opposed to the entire entity), because materialization for this goes through different path, indistinguishable from normal enum value read from column in relational table. Fixes #31100 Taking over the PR Tweaks from review

maumar requested review from ajcvickers and roji July 20, 2023 09:07