System.Text.Json: Add TimeSpanConverter #54186

CodeBlanch · 2021-06-14T22:27:59Z

Description

Adds TimeSpan support to System.Text.Json JsonSerializer.

Uses Utf8Parser.TryParse & Utf8Formatter.TryFormat for the heavy lifting.

Public API Changes

    public static partial class JsonMetadataServices
    {
       public static System.Text.Json.Serialization.JsonConverter<System.TimeSpan> TimeSpanConverter { get { throw null; } }
    }

Considerations

This is using the constant ("c") format. I think the general short format ("g") could compact things a bit, but not sure if that would round-trip with Newtonsoft and it is culture-sensitive.

/cc @layomia @eiriktsarpalis @JamesNK

dotnet-issue-labeler · 2021-06-14T22:28:03Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

ghost · 2021-06-14T22:28:07Z

Tagging subscribers to this area: @eiriktsarpalis, @layomia
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes #29932

Description

Adds TimeSpan support to System.Text.Json JsonSerializer.

Uses Utf8Parser.TryParse & Utf8Formatter.TryFormat for the heavy lifting.

Public API Changes

    public static partial class JsonMetadataServices
    {
       public static System.Text.Json.Serialization.JsonConverter<System.TimeSpan> TimeSpanConverter { get { throw null; } }
    }

Considerations

This is using the constant ("c") format. I think the general short format ("g") could compact things a bit, but not sure if that would round-trip with Newtonsoft and it is culture-sensitive.

/cc @layomia @eiriktsarpalis @JamesNK

Author:	CodeBlanch
Assignees:	-
Labels:	`area-System.Text.Json`, `new-api-needs-documentation`
Milestone:	-

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Value.ReadTests.cs

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Value.WriteTests.cs

...es/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/TimeSpanConverter.cs

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Value.ReadTests.cs

...es/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/TimeSpanConverter.cs

eiriktsarpalis · 2021-06-28T12:19:36Z

CI test failures are unrelated. @layomia has your feedback been addressed?

layomia

LGTM, but we should remove the 'g' format fallback, keeping in line with the implementation for other primitives that only support one popular format (e.g. ISO 8601 for DateTime). This way there's no confusion about what formats are supported/unsupported. If we get feedback later requesting other formats, we can re-add the fallback(s).

CodeBlanch · 2021-06-28T18:52:46Z

Updated to remove the 'g' fallback.

...tem.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonMetadataServices.Converters.cs

...es/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/TimeSpanConverter.cs

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Value.ReadTests.cs

src/libraries/System.Text.Json/src/System/Text/Json/ThrowHelper.cs

layomia

Thanks for removing the fallback. Just a couple minor comments left to address, then this PR is good to merge.

layomia · 2021-07-01T19:36:54Z

...es/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/TimeSpanConverter.cs

+            bool result = Utf8Formatter.TryFormat(value, output, out int bytesWritten, 'c');
+            Debug.Assert(result);
+
+            writer.WriteStringValue(output.Slice(0, bytesWritten));


[Non-blocking for this PR] When #54254 goes in, we should consider updating this to use the new Utf8JsonWriter.WriteRawValue API so we can avoid logic that checks whether the input payload needs to be escaped. We'd first need to quote the formatted value.

layomia

Thanks, @CodeBlanch! LGTM, with a few suggestions for a follow up PR.

Test failures are unrelated to the changes in this PR.

layomia · 2021-07-07T16:35:29Z

src/libraries/System.Text.Json/src/System/Text/Json/JsonHelpers.Date.cs

@@ -117,12 +117,6 @@ public static bool IsValidDateTimeOffsetParseLength(int length)
            return IsInRangeInclusive(length, JsonConstants.MinimumDateTimeParseLength, JsonConstants.MaximumEscapedDateTimeOffsetParseLength);
        }

-        [MethodImpl(MethodImplOptions.AggressiveInlining)]
-        public static bool IsValidDateTimeOffsetParseLength(long length)


Why delete this helper?

It wasn't being used anymore so I cleaned it up. The code that was calling it (in Utf8JsonReader.TryGet) is now calling IsInRangeInclusive directly so it can pass in the max length based on encoded string or not. You want me to put it back with the other feedback items for a follow-up?

layomia · 2021-07-07T16:42:16Z

src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.TryGet.cs

                {
                    value = default;
                    return false;
                }

                Debug.Assert(sequenceLength <= JsonConstants.MaximumEscapedDateTimeOffsetParseLength);
-                Span<byte> stackSpan = stackalloc byte[(int)sequenceLength];
-
+                Span<byte> stackSpan = stackalloc byte[_stringHasEscaping ? JsonConstants.MaximumEscapedDateTimeOffsetParseLength : JsonConstants.MaximumDateTimeOffsetParseLength];


Thanks for these changes. Searching for "stackalloc byte" in the STJ.csproj shows there might be a few more places that would benefit from this pattern, particularly in JsonReaderHelper.*.cs. We'd take a follow up PR to address this if you're up for it!

layomia · 2021-07-07T16:49:53Z

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/JsonDateTimeTestData.cs

@@ -215,6 +214,7 @@ public static IEnumerable<object[]> InvalidISO8601Tests()
            yield return new object[] { "\"1997-07-16T19:20:30.45555555550000000\"" };
            yield return new object[] { "\"1997-07-16T19:20:30.45555555555555555\"" };
            yield return new object[] { "\"1997-07-16T19:20:30.45555555555555555555\"" };
+            yield return new object[] { "\"1997-07-16T19:20:30.4555555555555555+01:300\"" };


I don't think we have DateTimeOffset failure tests for leading whitespace. We'd take a follow up to add these.

layomia · 2021-07-07T16:52:59Z

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Value.ReadTests.cs

@@ -303,6 +306,9 @@ public static void ValueFail()
            Assert.Throws<JsonException>(() => JsonSerializer.Deserialize<DateTimeOffset>(unexpectedString));
            Assert.Throws<JsonException>(() => JsonSerializer.Deserialize<DateTimeOffset?>(unexpectedString));

+            Assert.Throws<JsonException>(() => JsonSerializer.Deserialize<TimeSpan>(unexpectedString));
+            Assert.Throws<JsonException>(() => JsonSerializer.Deserialize<TimeSpan?>(unexpectedString));


Due to other logic, we should be fine, but a few more nullable TimeSpan tests would be good. E.g. deserializing null into a non-nullable TimeSpan should fail, but should be fine for nullable TimeSpan?.

layomia · 2021-07-07T16:55:41Z

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Utf8JsonReaderTests.TryGet.Date.cs

+        [MemberData(nameof(JsonDateTimeTestData.InvalidISO8601Tests), MemberType = typeof(JsonDateTimeTestData))]
+        public static void TryGetDateTime_HasValueSequence_False(string testString)
+        {
+            static void test(string testString, bool isFinalBlock)


nit

Suggested change

static void test(string testString, bool isFinalBlock)

static void Test(string testString, bool isFinalBlock)

* Follow-up feedback from #54186. * Code review. * Code review. * Dedicated constant for the max number of chars permitted to be allocated on the stack.

dotnet/runtime#54186 (comment)

* Constant `stackalloc` can be optimized by the JIT. dotnet/runtime#54186 (comment) * Fix ESXXX KID computation cause by incorrect canonicalized JWK size ('crv' was computed as base64-encoded) * `EllipticalCurve` is now a class instead of a readonly struct. This reduce the cost of struct copy.

Added TimeSpanConverter.

9fa6f46

CodeBlanch requested review from eiriktsarpalis, jozkee, layomia and steveharter as code owners June 14, 2021 22:28

dotnet-issue-labeler bot added area-System.Text.Json new-api-needs-documentation labels Jun 14, 2021

CodeBlanch mentioned this pull request Jun 14, 2021

System.Text.Json: TimeSpan support #53908

Closed

eiriktsarpalis reviewed Jun 15, 2021

View reviewed changes

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Value.ReadTests.cs Show resolved Hide resolved

eiriktsarpalis reviewed Jun 15, 2021

View reviewed changes

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Value.ReadTests.cs Show resolved Hide resolved

eiriktsarpalis reviewed Jun 15, 2021

View reviewed changes

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Value.WriteTests.cs Show resolved Hide resolved

CodeBlanch added 3 commits June 15, 2021 12:48

Code review.

03b10a3

Test tweak.

9c5c1f6

Added invalid cases.

bce438f

layomia reviewed Jun 20, 2021

View reviewed changes

...es/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/TimeSpanConverter.cs Outdated Show resolved Hide resolved

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Value.ReadTests.cs Show resolved Hide resolved

Remove the ToArray call in the case of ValueSequence.

0293935

layomia reviewed Jun 23, 2021

View reviewed changes

...es/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/TimeSpanConverter.cs Outdated Show resolved Hide resolved

Support escaped strings in TimeSpanConverter.

cf8d104

lewing mentioned this pull request Jun 28, 2021

[wasm] NRE in System.Runtime.InteropServices.JavaScript.Runtime.SetupJSContinuation, for System.Xml.Linq.xNodeBuilder.Tests #53614

Closed

layomia suggested changes Jun 28, 2021

View reviewed changes

CodeBlanch added 3 commits June 28, 2021 10:26

Merge remote-tracking branch 'upstream/main' into json-timespan2

6041e5c

Removed 'g' format fallback.

2838c26

Fixed 'h:mm:ss' being accepted by TimeSpanConverter.

0248a15

eiriktsarpalis reviewed Jun 29, 2021

View reviewed changes

...tem.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonMetadataServices.Converters.cs Show resolved Hide resolved

eiriktsarpalis reviewed Jun 29, 2021

View reviewed changes

...es/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/TimeSpanConverter.cs Outdated Show resolved Hide resolved

eiriktsarpalis approved these changes Jun 29, 2021

View reviewed changes

GrabYourPitchforks reviewed Jun 30, 2021

View reviewed changes

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Value.ReadTests.cs Show resolved Hide resolved

layomia self-requested a review July 1, 2021 19:16

layomia reviewed Jul 1, 2021

View reviewed changes

src/libraries/System.Text.Json/src/System/Text/Json/ThrowHelper.cs Show resolved Hide resolved

layomia reviewed Jul 1, 2021

View reviewed changes

CodeBlanch added 2 commits July 6, 2021 16:24

Code review.

f875e6f

Code review.

f3e49ff

layomia approved these changes Jul 7, 2021

View reviewed changes

layomia added this to the 6.0.0 milestone Jul 7, 2021

layomia assigned CodeBlanch Jul 7, 2021

layomia merged commit d5f9544 into dotnet:main Jul 7, 2021

CodeBlanch deleted the json-timespan2 branch July 7, 2021 17:26

CodeBlanch mentioned this pull request Jul 8, 2021

System.Text.Json: stackalloc constants + misc PR feedback #55350

Merged

eiriktsarpalis pushed a commit to CodeBlanch/runtime that referenced this pull request Jul 16, 2021

Follow-up feedback from dotnet#54186.

5c4d4fd

maliming mentioned this pull request Jul 19, 2021

TimeSpan cannot be used for caching. abpframework/abp#9612

Closed

layomia mentioned this pull request Jul 19, 2021

Reduce allocations in VersionConverter #55564

Merged

ycrumeyrolle added a commit to uruk-project/Jwt that referenced this pull request Jul 22, 2021

Constant stackalloc can be optimized by the JIT.

3263e61

dotnet/runtime#54186 (comment)

ycrumeyrolle mentioned this pull request Jul 22, 2021

Constant stackalloc can be optimized by the JIT. uruk-project/Jwt#569

Merged

This was referenced Jul 28, 2021

System.Net.Security test run killed on Mono Linux #56478

Closed

[WinHttpHandler] Long running test: ResponseHeadersRead_SynchronizationContextNotUsedByHandler on Win7/Win8 #54034

Closed

ghost locked as resolved and limited conversation to collaborators Aug 6, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

System.Text.Json: Add TimeSpanConverter #54186

System.Text.Json: Add TimeSpanConverter #54186

CodeBlanch commented Jun 14, 2021

dotnet-issue-labeler bot commented Jun 14, 2021

ghost commented Jun 14, 2021

Description

Public API Changes

Considerations

eiriktsarpalis commented Jun 28, 2021

layomia left a comment

CodeBlanch commented Jun 28, 2021

layomia left a comment

layomia Jul 1, 2021 •

edited

Loading

layomia left a comment

layomia Jul 7, 2021

CodeBlanch Jul 7, 2021

layomia Jul 7, 2021

layomia Jul 7, 2021

layomia Jul 7, 2021

layomia Jul 7, 2021

	static void test(string testString, bool isFinalBlock)
	static void Test(string testString, bool isFinalBlock)

System.Text.Json: Add TimeSpanConverter #54186

System.Text.Json: Add TimeSpanConverter #54186

Conversation

CodeBlanch commented Jun 14, 2021

Description

Public API Changes

Considerations

dotnet-issue-labeler bot commented Jun 14, 2021

ghost commented Jun 14, 2021

Description

Public API Changes

Considerations

eiriktsarpalis commented Jun 28, 2021

layomia left a comment

Choose a reason for hiding this comment

CodeBlanch commented Jun 28, 2021

layomia left a comment

Choose a reason for hiding this comment

layomia Jul 1, 2021 • edited Loading

Choose a reason for hiding this comment

layomia left a comment

Choose a reason for hiding this comment

layomia Jul 7, 2021

Choose a reason for hiding this comment

CodeBlanch Jul 7, 2021

Choose a reason for hiding this comment

layomia Jul 7, 2021

Choose a reason for hiding this comment

layomia Jul 7, 2021

Choose a reason for hiding this comment

layomia Jul 7, 2021

Choose a reason for hiding this comment

layomia Jul 7, 2021

Choose a reason for hiding this comment

layomia Jul 1, 2021 •

edited

Loading