-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for custom converters and OnXXX callbacks #29177
Comments
What is a purpose of methods:
|
For reference, this would be a good test case to add once this feature comes online: |
My feedback on this as-written at 7:45 PST 😆
// Allows a surrogate pattern for strongly-typed scenarios to prevent boxing, such as for Enum.
public virtual JsonConverter CreateConverter(Type converterType); I don't recall us talking through this. What does this enable? public class JsonSerializationOptions
{
. . .
IList<JsonConverters> Converters { get; }
. . .
} I think what we have here is the best we came up with combining the desire to register This still bugs me to see We should get feedback from a wider group as well. Another approach would be add an interface that expresses an untyped json converter. public interface IJsonConverter
{
bool CanConvert(Type type);
bool TryRead(ref Utf8JsonReader reader, Type type, JsonSerializerOptions options, out object value);
void Write(Utf8JsonWriter writer, object value, JsonSerializerOptions options);
} Since this is an interface, This is cleaner for sure. The thing that I think is bothersome is defining an interface that nothing the framework will call. I would imagine the serializer would want to downcast to avoid boxing/casts where possible since the serializer is late-bound. |
@rynowak thanks for the feedback.
Converters will not be public for 3.0 (maybe never). However it may be desirable to extend or re-use these converters for convenience and performance when (de)serializing non-trivial POCO properties including Enums and collections. So I think it the converters have a good chance of being public at some point pending feedback. However, since we're not sure if the internal converters will be made public I'll move the base classes to
It is there for convenience and performance, but can be removed. Simple converters that derive from
This relates to the discussion around supporting generic converters polymorphically. The current approach works for all cases including the tricky Enum converter which needs to be strongly-typed to avoid boxing but also has generic constraints (
It allows re-ordering, remove and add which are the intended scenarios. We really didn't discuss making it "easy" to manually call read\write.
In most cases having |
As part of the However, we should consider adding the So, in conjunction with the APIs proposed in the original post, also consider adding: namespace System.Text.Json
{
public sealed partial class Utf8JsonWriter : System.IDisposable
{
public void WritePropertyName(JsonEncodedText propertyName) { }
public void WritePropertyName(System.ReadOnlySpan<byte> utf8PropertyName) { }
public void WritePropertyName(System.ReadOnlySpan<char> propertyName) { }
public void WritePropertyName(string propertyName) { }
}
} Additionally, we can then simplify the JsonSerializer/Writer interop API (from https://github.com/dotnet/corefx/issues/36714#issuecomment-497282656), and only add the WriteValue APIs: namespace System.Text.Json
{
public static partial class JsonSerializer
{
public static void WriteValue(Utf8JsonWriter writer, object value, Type type, JsonSerializerOptions options = null) { throw null; }
public static void WriteValue<TValue>(Utf8JsonWriter writer, TValue value, JsonSerializerOptions options = null) { throw null; }
//public static void WriteProperty(Utf8JsonWriter writer, ReadOnlySpan<byte> utf8PropertyName, object value, Type type, JsonSerializerOptions options = null) { throw null; }
//public static void WriteProperty(Utf8JsonWriter writer, ReadOnlySpan<char> propertyName, object value, Type type, JsonSerializerOptions options = null) { throw null; }
//public static void WriteProperty(Utf8JsonWriter writer, string propertyName, object value, Type type, JsonSerializerOptions options = null) { throw null; }
//public static void WriteProperty(Utf8JsonWriter writer, JsonEncodedText propertyName, object value, Type type, JsonSerializerOptions options = null) { throw null; }
//public static void WriteProperty<TValue>(Utf8JsonWriter writer, ReadOnlySpan<byte> utf8PropertyName, TValue value, JsonSerializerOptions options = null) { throw null; }
//public static void WriteProperty<TValue>(Utf8JsonWriter writer, ReadOnlySpan<char> propertyName, TValue value, JsonSerializerOptions options = null) { throw null; }
//public static void WriteProperty<TValue>(Utf8JsonWriter writer, string propertyName, TValue value, JsonSerializerOptions options = null) { throw null; }
//public static void WriteProperty<TValue>(Utf8JsonWriter writer, JsonEncodedText propertyName, TValue value, JsonSerializerOptions options = null) { throw null; }
}
} The performance benefit of adding the propertyname-value |
Notes from today: Callback attributes
Converters
We'll meet again tomorrow, attempting to close on these issues. |
Seemingly it is possible to use a converter implementation to do the before\after on read\write instead of OnXXX methods. However, calling back into the serializer for the same type will lead to infinite loop since the same converter will be called. We need to detect this somehow in general (as to not throw StackOverflow and instead throw JsonException or InvalidOperation), but using the converter in this way is not possible until we determine how to prevent re-calling the same converter and instead let the framework serializer. UPDATE: here are alternate locations for the OnXXX functionality using converters: When doing this, the converter cannot pass the same Below we don't pass the options in and instead use the "default" global options instance which will not have any custom converters registered, avoiding the infinite loop. If the default options are not feasible (because of settings) then a new instance of the options will need to be created and any settings applied manually; this will be slow since each new instance caches independently. public override void Read(ref Utf8JsonReader reader,
Type type, JsonSerializerOptions options,
out MyPOCO value)
{
// "before" code needs to go into the POCO constructor; code here doesn't have the POCO instance.
value = JsonSerializer.Read<MyPOCO>(ref reader); // note: "options" not passed in
// Place "after" code here (e.g. OnDeserialized)
}
public override void Write(Utf8JsonWriter writer,
MyPOCO value, JsonSerializerOptions options)
{
// Place "before" code here (e.g. OnSerializing)
JsonSerializer.Write(writer, value); // note: "options" not passed in
// Place "after" code here (e.g. OnSerialized)
} In addition to the workaround above, a future feature that fits with the current design would allow the notifications plus a materializer for an object. The read\write is handled normally by the framework. This works only for JSON objects (not value types) and assumes there is not already a converter for this type. Something like: public class ObjectMaterializer<T> : JsonConverter
{
public override sealed bool CanConvert(Type typeToConvert);
public virtual T Deserializing();
public virtual void Deserialized(T value);
public virtual void Serializing(T value);
public virtual void Serialized(T value);
} |
These are some of my raw notes from design discussions that we had when coming up with the proposal for converters. I'm including these because @ahsonkhan thought they might be useful. I've cut samples and API designs from my notes, because the top post is the authoritative place for that info. Some of these notes include details or decisions that have already been revised, so consider yourself warned Notes from RyanPrebuffering
?? open question ?? - Do we validate that a converter read/wrote a single object/value/array. edit: I don't think we made a decision on this, which means that we're probably not going to try and validate this. JSON.NET doesn't attempt to validate this invariant.
Like JsonDocument - JsonCommentHandling.Allow will throw since it has not supported meaning Reader/Writer API additionsSeperate reading/writing - a few options
We still think dotnet/corefx#3 is viable, but it needs more investigation
Serializer API additionsWhere we have a disagreement between the options used to create the reader/writer and the
JsonSerializer.Parse<TKey>(reader, options);
writer.WritePropertyName("Value");
JsonSerializer.Write(writer, value.Value, option); Notes from Steve• Scenarios to make sure we support: |
Raw notes from today:
|
I've just come across a scenario, porting some code away from Newtonsoft.Json, where I really need an equivalent to |
@khellang yes a |
Notes from toady:
|
#Comment comment #I12XVH 字典表 BootstrapDict 实体类 Define 为 int 类型 #Issue https://github.com/dotnet/corefx/issues/36639 https://github.com/dotnet/corefx/issues/39473 link #I12XVH
Is there any example for immutable classes? |
Primary scenarios for converters:
JsonElement
if the corresponding property isobject
.DateTime
, specific Enums, and supporting JSON strings for integers or floating point types.Update: currently OnXXX callback are out of scope for 3.0. See the comments section for a workaround using a converter.
OnXXX callbacks are methods that are invoked on the POCO object during (de)serialization. They are specified by adding an attribute to each such method. Scenarios for OnXXX callbacks mostly relate to defaulting and validation scenarios:- OnDeserializing: initialize getter-only collections before deserialization occurs.- OnDeserialized: initialize unassigned properties; set calculated properties in preparation for consumption; throw exception if missing or invalid state.- OnSerializing: assign default values; throw exception if missing or invalid state; set up any temporary variables used for serialization workarounds.- OnSerialized: clear any temporary variables.To create a new converter:
JsonValueConverter<T>
which closes the<T>
to the type to convert.JsonSerializerOptions
or by placing[JsonConverter]
on a property. Optionally, if the converter is for a custom data type, the data type itself can have a[JsonConverter]
which will self-register.API
Samples
Custom Enum converter
Polymorphic POCO converter
Passing additional state to converter using an attribute
List converter with additional state
Requirements
Int32
converter could be created to automatically convert a json string to anInt32
(not supported in the defaultInt32
converter).CanConvert
is used.null
andNullable<T>
(the converter is not called)[JsonConverter]
-derived attribute.Converters
collection on the options class from being changed once (de)serialization has occurred.InvalidOperationException
if there is more than one converter attribute on a given Type or property.Design for open generics (typically collection classes)
Consider a custom
List<T>
converter:The read method:
The approach taken here is have a separate type (
JsonConverterFactory
) with factory method (CreateConverter()
)Then the list converter would have both a non-generic
MyListConverter
class (that derives fromJsonConverterFactory
) and a genericMyListConverter<T>
(that derives fromJsonConverter<List<T>>
).Only the non-generic
MyListConverter
class is added to theoptions.Converters
property. Internally there is another list\dictionary that caches all created converters for a specificT
, meaningList<int>
is a different entry thanList<long>
.Priority of converter registration
Converters can be registered at design-time vs run-time and at a property-level vs. class-level. The basic rules for priority over which is selected:
Thus the priority from highest to lowest:
options.Converters.Add(converter, property)
options.Converters.Add(converter)
The text was updated successfully, but these errors were encountered: