[release/5.0-rc2] Fix default handling for value types when converter based on interface (#42319)

[layomia]

Backport of #42319 to release/5.0-rc2.

Customer Impact

• Fixes Custom struct not ignored during serialization when default #42237 in .NET 5
• Edge case scenario that has workaround
• Fixes limitation in new .NET 5 JSON feature: (ignore value-type default System.Text.Json option to ignore default values during serialization #779)
• Project that discovered this issue, Schema.NET is migrating from Newtonsoft.Json to System.Text.Json
  • ongoing effort that started in Nov 2019 - Switch to System.Text.Json RehanSaeed/Schema.NET#100
  • migration enabled by new 5.0 issues
  • project has ~175K downloads: https://www.nuget.org/packages/Schema.NET/
• Project which discovered the issue has been a good test case for .NET 5 JSON features/architecture changes, and uncovered issues relating to InvalidOperationException and AccessViolationException (System.Text.Json Access Violation Exception (Attempted to read or write protected memory) #40878)
• Fix unblocks this project in a convenient way (no need for workaround), and helps migration make forward progress which may help discover other issues.

---

JsonIgnoreCondition.WhenWritingDefault is not honored when a property is a struct, the property is the default value, a custom converter is provided, and the converter's TypeToConvert is typeof(object) or an interface. The expected behavior is for the property to be ignored and to not call the converter.

This is not a common usage pattern (polymorphic converter implementations). The workaround is to specialize the converter to handle to specific type of the property, or to not use the converter if unnecessary.

Example highlighting the issue:

public class MyClass
{
    [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
    [JsonConverter(typeof(MyPolymorphicConverter))] // polymorphic converter causes issue
    public MyStruct MyStruct { get; set; }
}

public interface IInterface { }

public struct MyStruct : IInterface { }

// workaround is for this to be specialized to JsonConverter<MyStruct> or converter should not be used if not needed.
public class MyPolymorphicConverter : JsonConverter<IInterface>
{
    // Converter implementation
}

string json = JsonSerializer.Serialize(new MyClass()); // custom converter will be called
Console.WriteLine(json);
// Expected {}
// Actual (assuming converter writes this) {"MyStruct":{}}

Testing

Extensive tests have been added. Fix has been verified in reporting project. We don't expect a theme around issues polymorphic converter usages to develop, but will this will be an area to focus on in an oncoming fuzz testing and validation effort.

Risk

The fix is targeted to the scenario it fixes. It is unlikely to break other areas. There is no perf regression to the happy path.

[layomia]

Marking as servicing-consider but should not be merged until #42359 is for proper layering. This commit as-is contains changes from that PR - this was done to fix a merge conflict with cherry-picking #42319 to release/5.0-rc2.

[layomia]

Edge case scenario that has workaround

Closing for this reason given the stage of the release. We don't expect many apps to hit this and can provide guidance on how to workaround it in 5.0.