React to latest Roslyn nullability changes #36104

GrabYourPitchforks · 2020-05-08T07:46:15Z

Follow-up to #35782, which addressed the corelib side of things. This PR addresses the libraries side.

Due to #34417 (comment) I can't remove the override in Versions.props just yet. But the latest nightly compiler toolchain fixes that blocking issue, so this PR updates the explicit override to point to the latest nightly to demonstrate that we do compile cleanly against it.

I plan on reverting my changes to Versions.props as part of checkin, assuming the updated code also compiles cleanly on the older toolset. If the updated code doesn't compile cleanly on the older toolset, then the best course of action might be to wait until a new drop of Arcade comes out which picks up the updated compiler, then update Arcade, merge the contents of this PR, and remove the override from Versions.props all in one large PR.

Contributes to #34417.

eng/Versions.props

GrabYourPitchforks · 2020-05-08T07:47:45Z

...raries/System.Collections.Concurrent/src/System/Collections/Concurrent/BlockingCollection.cs

+
+            if (waitForSemaphoreWasSuccessful)
+            {
+                Debug.Assert(item != null);


Open question: I'm not sure if this assertion is actually correct. Is it possible to have - say - BlockingCollection<string?>? If that's legal, we should remove this assert.

Is it possible to have - say - BlockingCollection<string?>?

Yes, that's valid. Please revert the added code. It's a little disheartening this didn't cause any existing tests to fail. Sigh.

I'll remove the assertion and add an explicit unit test.

GrabYourPitchforks · 2020-05-08T07:50:51Z

src/libraries/System.Runtime/ref/System.Runtime.cs

@@ -354,7 +354,7 @@ public abstract partial class Array : System.Collections.ICollection, System.Col
        public static int LastIndexOf<T>(T[] array, T value) { throw null; }
        public static int LastIndexOf<T>(T[] array, T value, int startIndex) { throw null; }
        public static int LastIndexOf<T>(T[] array, T value, int startIndex, int count) { throw null; }
-        public static void Resize<T>([System.Diagnostics.CodeAnalysis.NotNullAttribute] ref T[]? array, int newSize) { }
+        public static void Resize<T>([System.Diagnostics.CodeAnalysis.NotNullAttribute] ref T[]? array, int newSize) { throw null; }


Interesting consequence of the recent [NotNull] change is that the compiler now treats it as an out-like parameter: you must assign a non-null value to it before method exit. So this means our ref assemblies' dummy method bodies need to throw, just as they would for any other method which has an out parameter.

I didn't see any other cases where this needed to be changed, but might have implications for our "generate ref asms automatically" tool.

GrabYourPitchforks · 2020-05-08T08:45:51Z

@steveharter @layomia @jozkee Can I get your eyes specifically on the commit 29a837b in this PR? There was a bit to untangle and I was trying to follow other established patterns within System.Text.Json.

stephentoub · 2020-05-08T13:26:44Z

...raries/System.Collections.Concurrent/src/System/Collections/Concurrent/BlockingCollection.cs

            return waitForSemaphoreWasSuccessful;
+#pragma warning restore CS8762


There's no way to suppress this warning with a well-placed ! somewhere? The clutter caused by ! is bad enough, but adding a bunch of pragmas to suppress false positive nullable warnings just makes me sad.
cc: @jcouv

It's still not great, but possibly a bit better: add a return true; below the Debug.Assert(item != null); at line 761.

From the language/compiler perspective, it may be useful to allow return waitForSemaphoreWasSuccessful!; to solve this.

In reply to: 422141700 [](ancestors = 422141700)

From the language/compiler perspective, it may be useful to allow return waitForSemaphoreWasSuccessful!; to solve this.

@jcouv, yeah, let's do that 😄

I did consider using the if (condition) { assert(); return true; } else { return false; } pattern, but it produces different IL and codegen than a simple return condition;. I tried to avoid making changes that affected the codegen in any way.

Argh, I missed that the Debug.Assert(item != null) doesn't work here. An alternative would be _ = item!; to pretend that it's not null, but that smells in this case, and also that's not yet implemented.
I'll push on allowing suppression on return someBoolValue!;.

Another option (also smells) is to suppress on places that assign to item (pretend like we're never assigning maybe-null values)

In reply to: 422242826 [](ancestors = 422242826)

Yeah, using the ! operator inappropriately early in the method strikes me as undesirable. I'll leave the pragma for now and add a code comment directing to #36132. Once the language team settles on guidance for this we can react to it here.

Much appreciate your responsiveness on this! :)

...raries/System.Collections.Concurrent/src/System/Collections/Concurrent/BlockingCollection.cs

src/libraries/System.Collections.Concurrent/src/System/Collections/Concurrent/ConcurrentBag.cs

src/libraries/System.Net.Mail/src/System/Net/Mime/HeaderCollection.cs

....Json/src/System/Text/Json/Serialization/Converters/Collection/DictionaryDefaultConverter.cs

stephentoub

Thanks for fixing these up.

layomia · 2020-05-08T14:25:11Z

The System.Text.Json changes have been addressed in #35826, in reaction to JsonConverter<T>.Read being annotated with [return: MaybeNull]. There are a couple differences though, e.g. in my change, the out T value of JsonConverter<T>.TryRead is annotated with [MaybeNull], rather than [MaybeNullWhen(false)].

All things considered, after that PR is merged (which should be very soon), there shouldn't be any changes to System.Text.Json needed here.

FWIW the System.Text.Json changes LGTM, so I'm also fine with this PR going in first, with #35826 reacting to the change.

danmoseley · 2020-05-08T14:28:02Z

src/libraries/System.Net.WebHeaderCollection/ref/System.Net.WebHeaderCollection.cs

@@ -97,7 +97,9 @@ public partial class WebHeaderCollection : System.Collections.Specialized.NameVa
        public void Add(System.Net.HttpRequestHeader header, string? value) { }
        public void Add(System.Net.HttpResponseHeader header, string? value) { }
        public void Add(string header) { }


All these presumably complicate things next time we run the tool to update the refs.

GrabYourPitchforks · 2020-05-08T16:23:03Z

Thanks @layomia for the tip! If your stuff is going in today I'll back my JSON changes out and rebase on top of yours when ready.

layomia · 2020-05-08T17:59:14Z

@GrabYourPitchforks #35826 is in now (sorry for conflicts ;)

GrabYourPitchforks · 2020-05-08T22:28:28Z

Conflicts are fine. I appreciate your efforts @layomia over on the JSON + nullable PR and for driving it through API review! :)

GrabYourPitchforks · 2020-05-09T00:32:55Z

PR feedback-inspired changes since last iteration:

Explicitly pinned the new compiler version
Backed out the System.Text.Json changes and merged in Layomi's PR which addresses that project's nullability annotations
Removed a stray Debug.Assert and cleaned up some comment text around pragmas

React to latest Roslyn nullability changes

a8446cb

GrabYourPitchforks added the area-Meta label May 8, 2020

GrabYourPitchforks requested review from stephentoub and buyaa-n May 8, 2020 07:46

GrabYourPitchforks commented May 8, 2020

View reviewed changes

eng/Versions.props Outdated Show resolved Hide resolved

GrabYourPitchforks commented May 8, 2020

View reviewed changes

GrabYourPitchforks added the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label May 8, 2020

GrabYourPitchforks requested review from jozkee, layomia and steveharter as code owners May 8, 2020 08:44

stephentoub reviewed May 8, 2020

View reviewed changes

...raries/System.Collections.Concurrent/src/System/Collections/Concurrent/BlockingCollection.cs Outdated Show resolved Hide resolved

stephentoub reviewed May 8, 2020

View reviewed changes

src/libraries/System.Collections.Concurrent/src/System/Collections/Concurrent/ConcurrentBag.cs Show resolved Hide resolved

stephentoub reviewed May 8, 2020

View reviewed changes

src/libraries/System.Net.Mail/src/System/Net/Mime/HeaderCollection.cs Show resolved Hide resolved

stephentoub reviewed May 8, 2020

View reviewed changes

....Json/src/System/Text/Json/Serialization/Converters/Collection/DictionaryDefaultConverter.cs Outdated Show resolved Hide resolved

stephentoub approved these changes May 8, 2020

View reviewed changes

danmoseley reviewed May 8, 2020

View reviewed changes

jaredpar mentioned this pull request May 8, 2020

OSX machines are de-provisioned during CI / PR runs leading to failures #34472

Closed

jcouv mentioned this pull request May 8, 2020

Relax enforcement of conditional attributes dotnet/roslyn#44080

Closed

safern mentioned this pull request May 8, 2020

ServerAsyncAuthenticateTest.ServerAsyncAuthenticate_MismatchProtocols_Fails test failure #29642

Closed

GrabYourPitchforks added 2 commits May 8, 2020 15:30

Merge remote-tracking branch 'origin/master' into fix_nullability_2

73003ac

PR feedback

9a70a2b

GrabYourPitchforks force-pushed the fix_nullability branch from 29a837b to 9a70a2b Compare May 9, 2020 00:29

GrabYourPitchforks merged commit edbb262 into dotnet:master May 9, 2020

GrabYourPitchforks deleted the fix_nullability branch May 9, 2020 03:15

ghost locked as resolved and limited conversation to collaborators Dec 9, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

React to latest Roslyn nullability changes #36104

React to latest Roslyn nullability changes #36104

GrabYourPitchforks commented May 8, 2020

GrabYourPitchforks May 8, 2020 •

edited

Loading

stephentoub May 8, 2020

GrabYourPitchforks May 9, 2020

GrabYourPitchforks May 8, 2020

GrabYourPitchforks commented May 8, 2020

stephentoub May 8, 2020

jcouv May 8, 2020

stephentoub May 8, 2020

GrabYourPitchforks May 8, 2020

jcouv May 8, 2020

jcouv May 8, 2020

GrabYourPitchforks May 8, 2020

stephentoub left a comment

layomia commented May 8, 2020 •

edited

Loading

danmoseley May 8, 2020

GrabYourPitchforks commented May 8, 2020

layomia commented May 8, 2020

GrabYourPitchforks commented May 8, 2020

GrabYourPitchforks commented May 9, 2020

		return waitForSemaphoreWasSuccessful;
		#pragma warning restore CS8762

React to latest Roslyn nullability changes #36104

React to latest Roslyn nullability changes #36104

Conversation

GrabYourPitchforks commented May 8, 2020

GrabYourPitchforks May 8, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GrabYourPitchforks commented May 8, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stephentoub left a comment

Choose a reason for hiding this comment

layomia commented May 8, 2020 • edited Loading

Choose a reason for hiding this comment

GrabYourPitchforks commented May 8, 2020

layomia commented May 8, 2020

GrabYourPitchforks commented May 8, 2020

GrabYourPitchforks commented May 9, 2020

GrabYourPitchforks May 8, 2020 •

edited

Loading

layomia commented May 8, 2020 •

edited

Loading