Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Asp.NET Core 3.0 Json doesn't serialize Dictionary<Key,Value> #30524

Closed
willyt150 opened this issue Aug 7, 2019 · 51 comments · Fixed by #38056
Closed

New Asp.NET Core 3.0 Json doesn't serialize Dictionary<Key,Value> #30524

willyt150 opened this issue Aug 7, 2019 · 51 comments · Fixed by #38056
Assignees
Labels
area-System.Text.Json enhancement Product code improvement that does NOT require public API changes/additions
Milestone

Comments

@willyt150
Copy link

.NET Core 3.0 Preview 7

Asp.NET Web Apis, when returning a Dictionary it fails with a NotSupportedException. I've included the exception below.

Also, the ControllerBase.BadRequest method takes in a ModelStateDictionary, but when that's returned the serializer blows up as well with a NotSupportedException, but a slightly different message.

When will this support be added? Since this has been supported in Json.net and other serializers for a while I hope this is on the radar.

I do appreciate the fact that I can opt back in to using Json.net, so thank you very much for that!

Exception when returning a Dictionary
System.NotSupportedException: The collection type 'System.Collections.Generic.Dictionary`2[System.Int32,System.String]' is not supported.
at System.Text.Json.JsonClassInfo.GetElementType(Type propertyType, Type parentType, MemberInfo memberInfo, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.CreateProperty(Type declaredPropertyType, Type runtimePropertyType, PropertyInfo propertyInfo, Type parentClassType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.AddProperty(Type propertyType, PropertyInfo propertyInfo, Type classType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.AddPolicyProperty(Type propertyType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo..ctor(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializerOptions.GetOrAddClass(Type classType)
at System.Text.Json.WriteStackFrame.Initialize(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializer.WriteAsyncCore(Stream utf8Json, Object value, Type type, JsonSerializerOptions options, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|21_0(ResourceInvoker invoker, IActionResult result)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|29_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|17_1(ResourceInvoker invoker)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Exception when returning BadRequest
System.NotSupportedException: The collection type 'Microsoft.AspNetCore.Mvc.SerializableError' is not supported.
at System.Text.Json.JsonClassInfo.GetElementType(Type propertyType, Type parentType, MemberInfo memberInfo, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.CreateProperty(Type declaredPropertyType, Type runtimePropertyType, PropertyInfo propertyInfo, Type parentClassType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.AddProperty(Type propertyType, PropertyInfo propertyInfo, Type classType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.AddPolicyProperty(Type propertyType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo..ctor(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializerOptions.GetOrAddClass(Type classType)
at System.Text.Json.WriteStackFrame.Initialize(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializer.WriteAsyncCore(Stream utf8Json, Object value, Type type, JsonSerializerOptions options, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|21_0(ResourceInvoker invoker, IActionResult result)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|29_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|17_1(ResourceInvoker invoker)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

@ahsonkhan
Copy link
Member

ahsonkhan commented Aug 8, 2019

Both "not supported exception" errors are limitations within the built-in serializer and is by design (at least for what is shipping in 3.0).

When will this support be added? Since this has been supported in Json.net and other serializers for a while I hope this is on the radar.
I do appreciate the fact that I can opt back in to using Json.net, so thank you very much for that!

There a bunch of serializer capabilities that are on our radar to support in vNext (so 5.0) and beyond, with custom dictionary support being one of them.

Asp.NET Web Apis, when returning a Dictionary it fails with a NotSupportedException

When serializing, only Dictionary<string, TValue> is supported today (i.e. TKeys that are string). Your dictionary is of <int, string> which isn't supported.
https://github.com/dotnet/corefx/blob/93d7aa1c1737b6da29d04b78557215e18eb786d4/src/System.Text.Json/tests/Serialization/DictionaryTests.cs#L385-L390

@steveharter, @layomia - is there a potential workaround here in the meantime? What would it take to add support for non-string key'd dictionary within the serializer for 5.0?

System.NotSupportedException: The collection type 'Microsoft.AspNetCore.Mvc.SerializableError' is not supported.

Adding support for a type like SerializableError was not on my radar. @pranavkm, @rynowak - what's the context here? I am not familiar with ModelStateDictionary, could this be supported within mvc itself with a custom converter?

Edit: Nevermind, that's already fixed.

@ahsonkhan
Copy link
Member

ahsonkhan commented Aug 8, 2019

System.NotSupportedException: The collection type 'Microsoft.AspNetCore.Mvc.SerializableError' is not supported.

This was a known issue dotnet/aspnetcore#11459 that was recently fixed (as part of preview 8): dotnet/corefx#39001

@willyt150
Copy link
Author

Thank you very much for your quick replies @ahsonkhan!

The "limitation" of the key being a string actually makes sense when I think about it. I see now that Json.net actually generates json with the key being a string, when deserializing it would just get me an int back. It would definitely be nice having the support for non-string keys in the future, but not a show stopper.

Ok, glad to hear that the Mvc.SerializableError not being supported has been fixed. Any ideas on if there's a hoped for release date of Preview 8? Tried to search and find something, but not seeing anything about that.

Once preview 8 comes out we'll try giving the .net core 3 json serialization library a try again, but for now we're needing to stick with Json.net

@layomia
Copy link
Contributor

layomia commented Aug 8, 2019

@steveharter, @layomia - is there a potential workaround here in the meantime? What would it take to add support for non-string key'd dictionary within the serializer for 5.0?

@ahsonkhan @willyt150 the workaround for this is to use a custom converter that implements JsonConverter<T> where T is Dictionary<int, string>.
See https://github.com/dotnet/corefx/issues/36639#issue-429928740 for some examples.

@ahsonkhan
Copy link
Member

ahsonkhan commented Aug 8, 2019

Any ideas on if there's a hoped for release date of Preview 8?

Some time later this month.

Thinking about this some more, removing the up-for-grabs for now since it may be something we don't want to support by default.

@willyt150
Copy link
Author

Thank you @layomia I'll take a look into that.

Thank you @ahsonkhan, looking forward to the fix coming through!

@ahsonkhan
Copy link
Member

ahsonkhan commented Aug 19, 2019

From @namse (from https://github.com/dotnet/corefx/issues/40404):

Hello. When I try to serialize Dictionary with integer key, it throw System.NotSupportedException.

I think it makes sense to support Json serialization which Dictionary has ToString-able key. for example, when we run ToString for int or boolean, it return "123" or "true". I think that key is ToString-able key.

Verison

System.Text.Json Nuget Version : 4.6.0-preview8.19405.3

Code

var dictionary = new Dictionary<int, string>
{
  [5] = "five"
};
JsonSerializer.Serialize(dictionary);

Expected

"{"5": "five"}"

But what happen

Error System.NotSupportedException Thrown

Actually, there is compatibility problem when I change Newtonsoft.Json to System.Text.Json. They return string as I expected. I think System.Text.Json don't have to be compatible but... you know.

@hez2010
Copy link
Contributor

hez2010 commented Aug 25, 2019

I have implemented a converter which support both Serialization and Deserialization for IDictionary<TKey, TValue> where TKey has a static method TKey Parse(string):

using System;
using System.Collections;
using System.Collections.Generic;
using System.Globalization;
using System.Reflection;
using System.Text.Json;
using System.Text.Json.Serialization;

namespace JsonDictionaryConverter
{
    sealed class JsonNonStringKeyDictionaryConverter<TKey, TValue> : JsonConverter<IDictionary<TKey, TValue>>
    {
        public override IDictionary<TKey, TValue> Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            var convertedType = typeof(Dictionary<,>)
                .MakeGenericType(typeof(string), typeToConvert.GenericTypeArguments[1]);
            var value = JsonSerializer.Deserialize(ref reader, convertedType, options);
            var instance = (Dictionary<TKey, TValue>)Activator.CreateInstance(
                typeToConvert,
                BindingFlags.Instance | BindingFlags.Public,
                null,
                null,
                CultureInfo.CurrentCulture);
            var enumerator = (IEnumerator)convertedType.GetMethod("GetEnumerator")!.Invoke(value, null);
            var parse = typeof(TKey).GetMethod("Parse", 0, BindingFlags.Public | BindingFlags.Static, null, CallingConventions.Any, new[] { typeof(string) }, null);
            if (parse == null) throw new NotSupportedException($"{typeof(TKey)} as TKey in IDictionary<TKey, TValue> is not supported.");
            while (enumerator.MoveNext())
            {
                var element = (KeyValuePair<string?, TValue>)enumerator.Current;
                instance.Add((TKey)parse.Invoke(null, new[] { element.Key }), element.Value);
            }
            return instance;
        }

        public override void Write(Utf8JsonWriter writer, IDictionary<TKey, TValue> value, JsonSerializerOptions options)
        {
            var convertedDictionary = new Dictionary<string?, TValue>(value.Count);
            foreach (var (k, v) in value) convertedDictionary[k?.ToString()] = v;
            JsonSerializer.Serialize(writer, convertedDictionary, options);
            convertedDictionary.Clear();
        }
    }

    sealed class JsonNonStringKeyDictionaryConverterFactory : JsonConverterFactory
    {
        public override bool CanConvert(Type typeToConvert)
        {
            if (!typeToConvert.IsGenericType) return false;
            if (typeToConvert.GenericTypeArguments[0] == typeof(string)) return false;
            return typeToConvert.GetInterface("IDictionary") != null;
        }

        public override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options)
        {
            var converterType = typeof(JsonNonStringKeyDictionaryConverter<,>)
                .MakeGenericType(typeToConvert.GenericTypeArguments[0], typeToConvert.GenericTypeArguments[1]);
            var converter = (JsonConverter)Activator.CreateInstance(
                converterType,
                BindingFlags.Instance | BindingFlags.Public,
                null,
                null,
                CultureInfo.CurrentCulture);
            return converter;
        }
    }
}

Test:

class Entity
{
    public string Value { get; set; }
}
class TestClass
{
    public Dictionary<int, Entity> IntKey { get; set; }
    public Dictionary<float, Entity> FloatKey { get; set; }
    public Dictionary<double, Entity> DoubleKey { get; set; }
    public Dictionary<DateTime, Entity> DateTimeKey { get; set; }
    public Dictionary<string, Entity> StringKey { get; set; }
}
class Program
{
    static void Main(string[] args)
    {
        var options = new JsonSerializerOptions();
        options.Converters.Add(new JsonNonStringKeyDictionaryConverterFactory());
        var x = new TestClass
        {
            IntKey = new Dictionary<int, Entity> { [1] = new Entity { Value = "test" } },
            FloatKey = new Dictionary<float, Entity> { [1.3f] = new Entity { Value = "test" } },
            DoubleKey = new Dictionary<double, Entity> { [1.35] = new Entity { Value = "test" } },
            DateTimeKey = new Dictionary<DateTime, Entity> { [DateTime.Now] = new Entity { Value = "test" } },
            StringKey = new Dictionary<string, Entity> { ["test"] = new Entity { Value = "test" } }
        };

        var value = JsonSerializer.Serialize(x, options);
        Console.WriteLine(value);
        var obj = JsonSerializer.Deserialize<TestClass>(value, options);
        Console.WriteLine(JsonSerializer.Serialize(obj, options));
    }
}

Result:

{"IntKey":{"1":{"Value":"test"}},"FloatKey":{"1.3":{"Value":"test"}},"DoubleKey":{"1.35":{"Value":"test"}},"DateTimeKey":{"8/25/2019 6:47:48 PM":{"Value":"test"}},"StringKey":{"test":{"Value":"test"}}}
{"IntKey":{"1":{"Value":"test"}},"FloatKey":{"1.3":{"Value":"test"}},"DoubleKey":{"1.35":{"Value":"test"}},"DateTimeKey":{"8/25/2019 6:47:48 PM":{"Value":"test"}},"StringKey":{"test":{"Value":"test"}}}

However it still cannot serialize a nested Dictionary such as Dictionary<int, Dictionary<int, int>> because System.Text.Json doesn't accept the inner type Dictionary<int, int>. I think it's a bug.

@steveharter
Copy link
Member

However it still cannot serialize a nested Dictionary such as Dictionary<int, Dictionary<int, int>> because System.Text.Json doesn't accept the inner type Dictionary<int, int>. I think it's a bug.

Supporting only <string, x> is by design due to scope-cutting in order to ship for v3.0. The 3.0 release is intended to be a minimal viable product with the most common scenarios supported.

@hez2010
Copy link
Contributor

hez2010 commented Aug 26, 2019

@steveharter At least you shouldn't throw a notsupportexception when there's a useable converter.

@thorgeirk11
Copy link

Are there any plans to support this in .net core 3.1?

@buraktamturk
Copy link

For the newcomers, the temporary solution is to revert back to Newtonsoft.Json.

  1. Add package reference to Microsoft.AspNetCore.Mvc.NewtonsoftJson.
  2. Add .AddNewtonsoftJson() just after .AddControllers() / .AddMvc() or any other combination.

@steveharter
Copy link
Member

steveharter commented Sep 24, 2019

@steveharter At least you shouldn't throw a notsupportexception when there's a useable converter.

Yes that is a fair point. Perhaps we can remove this restriction for 3.1. cc @layomia

Also just to clarify that today dictionary elements are serialized like properties which is possible because the key is a string. Supporting non-string keys means elements will be serialized as a KeyValuePair.

@unruledboy
Copy link

unruledboy commented Sep 25, 2019

Oh boy, had this problem right after I upgraded to 3.0.

Had to install the newton package with AddNewtonsoftJson.

@ahsonkhan
Copy link
Member

From @israellot in https://github.com/dotnet/corefx/issues/41345

var dictionary = new Dictionary<int, int>()
            {
                [0] = 1
            };

 var serialized = System.Text.Json.JsonSerializer.Serialize(dictionary);

This simple serialization is handled well by the former default Newtonsoft json library by serializing the int key as string. On System.Text.Json is throws a not supported exception.

@ahsonkhan
Copy link
Member

ahsonkhan commented Sep 26, 2019

@israellot, @unruledboy, and others on the thread, can you provide details on why your object model requires Dictionaries with integer keys in your scenarios and why changing it to be Dictionary<string, TValue> wouldn't work? I would love to see some usages for gathering requirements and to help motivate the fix.

They would get serialized as strings anyway, so I don't understand in what scenarios you'd want to your underlying dictionary to have int32 keys instead.

@israellot
Copy link

israellot commented Sep 26, 2019

@ahsonkhan I believe the key motivation is compatibility.
The previous default serializer was Newtonsoft, so users may have written entire models relying on its ability to serialize and deserialize arbitrary classes. Migrating from 2.X to 3.0 will now cause a silent breaking change since we'll only know the incompatibility at runtime.

I believe many scenarios involve using json just as transport across the wire, and in this case, the domain model might be Dictionary<int,TValue>. Your suggestion boils down to creating a separate DTO object Dictionary<string,TValue> and converting between the two, seems rather inefficient since now we need to allocate another object just to be compliant with the serializer.
Looking strictly to the serializer, restricting dictionary to having string keys is logical as that's the only possible json representation. But considering that the serializer plays a role in applications I believe it's best to remove as much friction as possible.

@Kieranties
Copy link

Kieranties commented Sep 27, 2019

I've ran into this issue with a small program I'm writing where parts of a version label are provided through a json file.
The label parts have a key that specifies the index where the label part may be inserted. This means the keys are numeric values, e.g

{
  "parts" : {
    "1" : "alpha",
    "3" : "beta"
  }
}

Using Newtonsoft, the json can be deserialized without issue to a Dictionary<int, string>. After converting to System.Text.Json serialization failed.

I've resolved this issue by creating my own DictionaryConverter, and DictionaryConverter<T,K>.
I also created a simple converter that allows integers to be deserialized from a string

These are then registered through the serializer options: https://github.com/Kieranties/SimpleVersion/blob/master/src/SimpleVersion.Core/Serialization/Serializer.cs#L22

These changes allow the keys of a dictionary to be deserialized instead of directly read as a string. This further opens up support for keys to be arbitrary types that could have their own converters registered for serialization (e.g. enums/type/ types that may be serialized as unique keys etc)

I've not formally tested things but within the current development this seems to have resolved the issue.

@steveharter
Copy link
Member

Setting milestone for 3.1 to remove any restrictions that prevents a custom converter from being created that can handle any TKey for Dictionary<TKey,TValue>.

@steveharter steveharter self-assigned this Sep 27, 2019
@steveharter
Copy link
Member

steveharter commented Oct 3, 2019

Update: I've added samples that work with 3.0. I didn't notice any issues such as with nested dictionaries as reported above.

There are two JSON formats being used in the samples:

  • JSON object with string properties: {"1":"val1","2":"val2"} even though TKey is not a string.
    • These rely on a TryParse method on the corresponding supported key type. For v3.0 it is not feasible to provide support for a generalized TKey since TryParse methods can be different for any TKey and because there are culture settings that need to be provided (typically Invariant). So the samples below are for a single TKey type.
    • Sample Dictionary<int, string>. This is a simple example for just TValue == string.
    • Sample Dictionary<Guid, TValue>. This can handle any TValue.
    • Sample Dictionary<TKey, TValue> where TKey is Enum. For Enums; this can handle any TValue.
  • JSON array with KeyValuePair entries: [{"Key":1,"Value":"val1"},{"Key":2,"Value":"val2"}]

If these samples are satisfactory, I will change this issue to 5.0 in order to discuss whether we provide built-in support that don't require custom converters.

@steveharter
Copy link
Member

Setting milestone to 5.0 for consideration (what if any of the above examples should work by default).

@RandomGHUser
Copy link

Deserialization also seems to map generic object types to JsonDocument rather than their normal (primitive?) types.

Example:

string test = "[{\"id\":86,\"name\":\"test\"}]";
var SystemTextJson = System.Text.Json.JsonSerializer.Deserialize<List<Dictionary<string, object>>>(test);
var NewtonSoftJson = Newtonsoft.Json.JsonConvert.DeserializeObject<List<Dictionary<string, object>>>(test);

SystemTextJson[0]["id"] shows as: ValueKind = Number : "86"
NewtonSoftJson[0]["id"] shows as: 86

@roguecode
Copy link

@steveharter An enum keyed dictionary with that converter serializes as:
{ "Stuff": [ { "Key": 1, "Value": "String" }, { "Key": 3, "Value": "String" }, { "Key": 2, "Value": "String" } ] }

While JSON.NET gives what I'd assume most people would expect:
{ "Stuff": { "Item1": "String", "Item2": "String", "Item3": "String" }

@steveharter
Copy link
Member

Deserialization also seems to map generic object types to JsonDocument rather than their normal (primitive?) types.

Yes that is by design. See https://github.com/dotnet/corefx/issues/38713

@steveharter
Copy link
Member

@steveharter An enum keyed dictionary with that converter serializes as:
{ "Stuff": [ { "Key": 1, "Value": "String" }, { "Key": 3, "Value": "String" }, { "Key": 2, "Value": "String" } ] }
While JSON.NET gives what I'd assume most people would expect:
{ "Stuff": { "Item1": "String", "Item2": "String", "Item3": "String" }

I assume you're using the "Sample Dictionary<TKey, TValue>"? If so, yes that used KeyValuePair which has "Key" and "Value" properties. I did not provide a sample for TKey-based enum dictionaries that serialize as property names, but I will work on adding that.

@roguecode
Copy link

Yep, that one. And ah ok, I thought you had meant that as a generic dictionary serializer.
Interested to see your new sample when available, as the one we're currently using doesn't seem as fast as I'd want it to be.

@steveharter
Copy link
Member

@roguecode here's a Enum sample for Dictionary<TKey, TValue> where TKey is an Enum and uses the "property" JSON syntax instead of KeyValuePair. I also updated the list of samples above to include this new sample.

@ts46235

This comment has been minimized.

@layomia

This comment has been minimized.

@jozkee
Copy link
Member

jozkee commented Feb 23, 2020

Presumably anything you can currently serialize as a standalone object?

Yes.

@kstreichergb
Copy link

kstreichergb commented May 6, 2020

Deserialization also seems to map generic object types to JsonDocument rather than their normal (primitive?) types.

Example:

string test = "[{\"id\":86,\"name\":\"test\"}]";
var SystemTextJson = System.Text.Json.JsonSerializer.Deserialize<List<Dictionary<string, object>>>(test);
var NewtonSoftJson = Newtonsoft.Json.JsonConvert.DeserializeObject<List<Dictionary<string, object>>>(test);

SystemTextJson[0]["id"] shows as: ValueKind = Number : "86"
NewtonSoftJson[0]["id"] shows as: 86

Of all the mentioned issues, this bothers me the most. List or T[] or Dictionary<string,object> should deserialize properly for any type that can be mapped directly from the few types Json has to CLR types.

The 3.0 release is intended to be a minimal viable product with the most common scenarios supported

I wonder how List<Dictionary<string,object>> isn't one of the most common scenarios:

[// N objects
{"a":4},
{"b","Bla"},
]

As this does also deserialize into System.Text.JsonElement, where I would expect double (Number) and string (String)

@Voochichichi
Copy link

I've ran into this issue with a small program I'm writing where parts of a version label are provided through a json file.
The label parts have a key that specifies the index where the label part may be inserted. This means the keys are numeric values, e.g

{
  "parts" : {
    "1" : "alpha",
    "3" : "beta"
  }
}

Using Newtonsoft, the json can be deserialized without issue to a Dictionary<int, string>. After converting to System.Text.Json serialization failed.

I've resolved this issue by creating my own DictionaryConverter, and DictionaryConverter<T,K>.
I also created a simple converter that allows integers to be deserialized from a string

These are then registered through the serializer options: https://github.com/Kieranties/SimpleVersion/blob/feature/netcore3/src/SimpleVersion.Core/Serialization/Serializer.cs#L20

These changes allow the keys of a dictionary to be deserialized instead of directly read as a string. This further opens up support for keys to be arbitrary types that could have their own converters registered for serialization (e.g. enums/type/ types that may be serialized as unique keys etc)

I've not formally tested things but within the current development this seems to have resolved the issue.

hey @Kieranties , the github links 404 for me

@Kieranties
Copy link

hey @Kieranties , the github links 404 for me

@AirEssY I've corrected the links in my original comment, thinks are now in master at: https://github.com/Kieranties/SimpleVersion/tree/master/src/SimpleVersion.Core/Serialization

Trolldemorted added a commit to enowars/EnoEngine that referenced this issue Jun 3, 2020
Savallator pushed a commit to enowars/EnoEngine that referenced this issue Jun 3, 2020
* wip

* Fix enocore.models.database namespace

* Honor EnoELK logging definitions

* long keys are bad (dotnet/runtime#30524)

* More longs

* Use correct tostring method

* Fix flagshooter

* Move over all general types

* Fix SubmittedFlag Flag FK
@derekjwilliams
Copy link

derekjwilliams commented Jun 12, 2020

If the Dictionary is equivalent to a JavaScript Map, then any (JS type represented in C#) should be acceptable,

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map

A Map's keys can be any value (including functions, objects, or any primitive).

An example of the standard approach to deserialize a Map in JS is:

const countries = new Map(JSON.parse('[[1,"Bahamas (the)"],[2,"Bolivia (Plurinational State of)"]]'))

console.log(countries)

Which produces:

Map(2) {
  1 => 'Bahamas (the)',
  2 => 'Bolivia (Plurinational State of)'
}

TL;DR: restricting keys to strings does not play well with JS

@onionhammer
Copy link

@jozkee so is this coming in .NET 5 only or will it be go into 3.*?

@jozkee
Copy link
Member

jozkee commented Jul 7, 2020

@onionhammer .NET 5.0, you can also try out the feature in the next preview (5.0 preview8).
There are no plans of porting this into 3.x.

@verloka
Copy link

verloka commented Sep 2, 2020

Solution for asp net core 3.x:

var dic1 = new Dictionary<TKey, TValue>(); 
return Json(new { dic1 }); // does not work

var dic2 = from i in new Dictionary<TKey, TValue>() select new { i.Key, i.Value }
return Json(new { dic2 });  //works prety well

@onionhammer
Copy link

@verloka that is not the desired output

@layomia
Copy link
Contributor

layomia commented Sep 3, 2020

@jozkee so is this coming in .NET 5 only or will it be go into 3.*?

This won't be backported to 3.x but you can add use the System.Text.Json NuGet package in your project to get all the new features in .NET 5.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Text.Json enhancement Product code improvement that does NOT require public API changes/additions
Projects
None yet