Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bot upstream main merge 2021 11 30 #10

Closed
wants to merge 342 commits into from

Conversation

joshpeterson
Copy link

PeterSolMS and others added 30 commits November 11, 2021 16:02
…net#60966)

We reference entry #0 in the pinned plug queue even if there are no pinned plugs at all and thus the pinned plug queue contains left-over data from the mark phase.

The fix is to initialize saved_pinned_plug_index to a value that is invalid as a pinned plug queue index, and only use saved_pinned_plug_index as an index if  is valid.
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Don't run if the JIT-EE GUID has changed, since there won't be any collections
to download, so the downloads will fail.

Also update the superpmi-asmdiffs trigger comment.
… annotate individual methods as requiring AVX (dotnet#61259)

* Updating src/tests/Interop/PInvoke/Generics/GenericsNative.Vector* to annotate individual methods as requiring AVX

* Always use __m256i on XARCH
* XARCH: Remove redudant tests for GT_LT/GT_GE relops.

We can now optimize cases such as `(x + y < 0)` or `for (int x = v; x >= 0; x--)`
using the flag tracking logic during the emit stage. Notably, cases that
would generate...

```
add     reg0, reg1
test    reg0, reg0
jge     LABEL
```

now transform to

```
add     reg0, reg1
jns     LABEL
```

This transform is valid for signed GE and signed LT only.

* Add a few asserts related to flag reuse optimizations.
* Moving Narrow to implemented using SIMDAsHWIntrinsic

* Moving Widen to implemented using SIMDAsHWIntrinsic

* Fix some handling of Narrow/Widen hwintrinsics

* Ensure that Vector.Widen is still treated as an intrinsic

* Fixing NI_VectorT128_WidenUpper on ARM64 to actually call gtNewSimdWidenUpper
Measure browser app start times, 2 measurements implemented.

First to measure till the JS window.pageshow event, second to measure
time when we reach managed C# code.

Example ouput:

    | measurement | time |
    |-:|-:|
    |                    AppStart, Page show |   108.1400ms |
    |                AppStart, Reach managed |   240.2174ms |
* Environment.SystemPageSize returns cached value

* we are no longer shipping MS.IO.Redist, so we can use Array.MaxLength directly

* we are no longer shipping MS.IO.Redist, there is no need for File to be partial

* we are no longer shipping MS.IO.Redist, there is no need for FileInfo to be partial

* there is no need for .Win32.cs and .Windows.cs file anymore
* Remove all DLLIMPORTGENERATOR_ENABLED usage.

* Explicitly set EnableDllImportGenerator to true in test project.
… the receiver is a gshared type constrained to a primitive type/enum. (dotnet#61513)
…trained types. (dotnet#61514)

These are used for example in Span<T>:.ctor ().
This was preventing block compaction with loop entry blocks in loops
that had been previously optimized away (and thus removed from the
loop table).

There are a few cases where we now delete dead code that was previously
left in the function. There are a number of spurious local weighting
and IG textual asm diffs changes, possibly due to how PerfScore is implemented
(there are some surprisingly large PerfScore changes in a few cases,
despite no change in (most) generated code).
Implement method name wildcard matching for method descriptions

Globbing doesn't work because we don't have g_pattern_match_simple in eglib.
But a plain '*' wildcard does work.
When both OSR and PGO are enabled:
* Enable instrumenting OSR methods, so that the combined profile data from
Tier0 plus any OSR variants provide a full picture for subsequent Tier1
optimization.
* Use block profiles for both Tier0 methods that are likely to have patchpoints
and OSR methods.
* Fix phase ordering so partially jitted methods don't lose probes.
* A few more fixes for partial compilation, because the number of things
we think we might instrument and the number of things we end up instrumenting
can differ.
* Also improve the DumpJittedMethod output for OSR, and allow selective dumping
of a particular OSR variant by specifying its IL offset.

The updates on the runtime side are to pass BBINSTR to OSR methods, and to
handle the (typical) case where the OSR method instrumentation schema is a subset
of the Tier0 method schema.

We are still allowing OSR methods to read the profile data. So they are both
profile instrumented and profile optimized. Not clear if this is going to work
well as the Tier0 data will be incomplete and optimization quality may be poor.
Something to revisit down the road.
Emit it in the interpreter when a method is inlined or replaced with
an intrinsic. This is needed so the AOT profiler can track these
methods.
…traints. (dotnet#59437)

If a generic argument is a primitive type, and it has an interface constraint
that enums don't implement, then partial sharing for that instance is not
useful, since only the specific primitive type instance will be able
to use the shared version.

Fixes dotnet#54910.
* Enable new analyzers in global configs

* Address PR feedback
For now there's no functional change to the behavior of the tests,
I have just copied the bits to inject from Jeremy's example in his
pending PR.

Thanks

Tomas
* embedded size

* next array

* read consistency while rehashing

* comments

* remove CRST_UNSAFE_ANYMODE and COOP mode in the callers

* typo

* fix 32bit builds

* couple changes from review.

* Walk the buckets in resize.

* remove a `REVIEW:` comment.

* Update src/coreclr/vm/dacenumerablehash.inl

PR review suggestion

Co-authored-by: Jan Kotas <jkotas@microsoft.com>

* remove use of `auto`

* DAC stuff

* Constructor and GrowTable are not called by DAC, no need for DPTR, added a comment about a cast.

* SKIP_SPECIAL_SLOTS

* More compact dac_cast in GetNext

Co-authored-by: Jan Kotas <jkotas@microsoft.com>
For now there's no functional change to the behavior of the tests,
I have just copied the bits to inject from Jeremy's example in his
pending PR.

Thanks

Tomas

Contributes to: dotnet#54512
For now there's no functional change to the behavior of the tests,
I have just copied the bits to inject from Jeremy's example in his
pending PR.

I have spot-checked that some of the tests use Main with the
command-line args argument. I'm not changing them in this PR, the
signature only becomes important once we start actually merging
the IL tests and I presume we'll clean that up at that point.

Thanks

Tomas

Contributes to: dotnet#54512
am11 and others added 27 commits November 26, 2021 16:32
* Fix miscellaneous typos

* Cleanup trailing whitespaces
```sh
# git remote add dotnet https://github.com/dotnet/runtime && git pull --rebase dotnet main

if uname 2>/devnull | grep -q Darwin; then
    space=" "
fi

git show --name-only --pretty="" HEAD...dotnet/main |\
    xargs -I{} sh -c "test -f {} && sed -i$space'' 's/[[:space:]]*$//' {}"
```
…62003)

* mono_runtime_delegate_invoke wrapper with MONO_ENTER_GC_UNSAFE/MONO_EXIT_GC_UNSAFE;

mono_runtime_delegate_invoke wrapper with MONO_ENTER_GC_UNSAFE/MONO_EXIT_GC_UNSAFE;

* Update object.c
The CSE logic maintains a custom hashtable implementation.

It triggers a resize at the same time as adding a new CSE,
but forgets the hash of the new CSE needs to be updated from
its pre-resize value. Failing to do so can lead to losing some
CSEs, though not in a correctness-impacting way.
Otherwise we can end up not seeing the loop has memory havoc.

Also added an assert that will prevent this issue from arising in the future.
…et#60054)

* CI matrix change: add Windows.Server.Core.20H2 (dotnet#59572)

* Add Windows.Server.Core.20H2 to CI matrix

* add windows server 20H2

* Remove 20H2 from PR

* Remove 20H2 from PR run

* change helix queue for Windows.Server.Core.20H2
* Remove active issue.
* Use --web-server-use-cop.
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
* Allow zero byte reads on raw HTTP/1.1 response streams

* Allow zero byte reads on the rest of HTTP/1.1 response streams

* Allow zero byte reads on HTTP/2 response streams

* Allow zero byte reads on HTTP/3 response streams

* Enable sync zero-byte reads

* Add zero-byte read tests

* Fully enable zero-byte reads on HTTP/2 and 3

* Add zero-byte read tests for HTTP/2 and HTTP/3

* Remove unsafe-ish code from PeekChunkFromConnectionBuffer

* Add comments when we do extra zero-byte reads

* Update MockQuicStreamConformanceTests to allow zero-byte reads

* Update ConnectedStream tests to allow zero-byte reads

* Skip zero-byte read tests on Browser

* Update comment on explicit zero-byte reads in ChunkedEncodingReadStream
…tnet#61853)

This PR does two things:
1. Reorganizes how we keep track of deltas, and how we do metadata lookups after updates
2. Adds an ifdefed-out initial draft of support for adding methods.  (It worked for a simple test but it's not ready for general use yet).

Details of the delta changes.

* Previously Mono used a fully immutable model of metadata changes.  Logically each delta brings in modifications to existing table rows and addition of new rows.  Previously mono never formed the fully mutated tables - instead, each lookup was essentially responsible for playing all the changes forward every time it needed to lookup a row.  This was fine when we primarily supported only row additions (because we could just skip past all the updates that hadn't added the row we wanted yet, and just look in a single delta).  But as we started to support more and more modifications (of custom attributes, of property getters & setters, of parameters, etc) we now had to look through the whole list of deltas to make sure we found the latest modification of every row.
* The new approach is for each `DeltaInfo` (representing a summary of a particular update to a single assembly) to contain a collection of `mutants` - the fully modified tables with all the newly added rows and all the new modifications.  So lookups are fast now - we just go to the latest generation that is visible to a particular thread and look at its mutants tables.  The downside is increased memory use.  So we allocate the mutants from memory pools owned by each `DeltaInfo`.  Right now we never dealloc the pools, but in the future we could.  We already iterate over all the threads during our stop-the-world phase - we can record the earliest generation still needed by every thread and delete the pools for all earlier copies.  In practice this means we only keep 3 sets of tables: the mmaped baseline tables, the newest mutants, and the mutants from the prior generation for any still-executing methods.
* Additionally instead of storing a list of delta images in the `BaselineInfo`, we now store a list of `DeltaInfo` structs which each `DeltaInfo` pointing to a delta image.  This means that we can avoid repeated hash table lookups to map from a delta image to its `DeltaInfo` every time the runtime queries us for a table row or for a heap element.

---

* checkpoint: allow adding methods to existing classes.

Doesn't do anything yet.

And needs more sanity checking (roslyn won't give us virtual or abstract
methods, but we should check).

* fixme in loader

* [metadata] Add mono_metadata_table_num_rows

Returns the number of rows in a metadata table, taking into account metadata
update deltas

* add table to ptr table helper

* Param attr lookups for deltas can have param_index == 0

The params are added with a subsequent enclog "add param" function.

* WIP: start adding support for parameter additions.

It "works" in that calling methods appears to work (ie direct token references
seem to do the right thing).

Primarily this is because the param table additions are not that interesting.
All the good stuff is in the method signature (which is just in the blob heap).

Presumably anything that actually needs parameter attributes, or anything that
uses reflection to look at the parameters, will break.

* WIP: add MethodDef -> TypeDef lookup

Allows calling non-public methods, which are now assigned the correct parent

* Add hot reload test for lambda capturing this

Lambdas that only capture `this` (and not local variables or arguments) compile
to a private instance method in the enclosing class.  So it is enough to
support EnC deltas that add methods.

* clarify comments about MONO_METHOD_PARAMLIST

* [hot_reload] Store debug info of updated methods

* [hot_reload] Allocate modifiable tables in DeltaInfo

This is the foundation for a new approach for metadata lookups.
Instead of using an immutable model (each lookup traverses every delta to find
the most up to date version of each table row), we are going to create a
complete updated table for each generation and only do the lookup in the latest
exposed generation directly.  (This is essentially the CoreCLR model).

This commit is just the first foundations: we allocate the tables and copy over
the previous generations' rows and zero out any rows that will be inserted.
Delta applications and lookups have not been updated yet.

As a slight optimization, tables that don't have modified or added rows are not
copied from the base image.  If a generation modifies or adds rows, from that
point forward, each subsequent generation will copy the table.

We could be a bit more thrifty with copying, but it will complicate lookups.
Also eventually we will try to deallocate the pools for generations that are
older than no thread needs anymore.

Metadata heaps are still looked up in each delta directly - heap combining does
not seem necessary yet.

* [mini] Allow MONO_VERBOSE_METHOD='*:*'

Implement method name wildcard matching for method descriptions

Globbing doesn't work because we don't have g_pattern_match_simple in eglib.
But a plain '*' wildcard does work.

* populate mutated table rows

leave suppressed columns unchanged

* [hot_reload] Switch lookups to the mutant tables

* cleanup: remove of effective_table calculation

The row index doesn't change in the new scheme, pass it by value

* cleanup: Remove relative_delta_index from component API

It's only used internally by the update logic

* cleanup: Pass DeltaInfo to relative_delta_index

don't compute it from the dmeta image

* cleanup: Store a list of DeltaInfo in the BaselineInfo

Instead of storing a list of delta images and then relying on delta_info_lookup
to find the DeltaInfo, just store them directly on the baseline info.

This changes the cleanup responsibilities a bit.  Now we destroy the DeltaInfo
when we are iterate through the delta infos when we close the baseline image,
instead of when we remove the delta_image_to_info hashtable entry.

* Turn off method addition support, for now

Just want to get the cleanups enabled for now

* Fix null ptr when checking for updated ppdb info
The codespaces prebuild action can only run in the dotnet/runtime repo and not forks. The feature needs to be enabled for each repo and will fail if it isn't enabled. In order to minimize failure notifications, add a check to the action so it only runs against dotnet/runtime.
…tnet#62015)

* A draft of doc for describing debugger behaviour.

* Removing spaces.
The clang 13 optimizer started to assume that "this" pointer is always
properly aligned. That lead to elimination of some code that was actually
needed.
It also takes pointer aliasing rules more strictly in one place in jit.
That caused the optimizer to falsely assume that a callee with an argument
passed by reference is not modifying that argument and used a stale
copy of the original value at the caller site.

This change fixes both of the issues. With this fix, runtime compiled
using clang 13 seems to be fully functional.
In particular do not attach it to INS_align that do not end up producing
any code.
* fix TimeSpan support in sourcegen

* address feedback
* remove MonoManagedAssemblyLoadContext

* fix mono_alc_from_gchandle use field

* change by vargaz review

* formatting

* fixed compiler error
…2020)

* Fix System.Object serialization with custom number handling

* fix typos
…et#62132)

The collection definitions must be in the same assembly as the test that uses them
* No-op ConcurrentStack.PushRange(T[]) if empty

Resolves dotnet#62121.

* Add more bounds checks

Assert that ArgumentOutOfRangeException is thrown if specifying indexes for an empty array with PushRange(T[], int, int).

* Review feedback

Allow out-of-bounds index on empty array if count is zero.

* Add extra test case

Add a further test case for a non-empty array but with a count parameter value of zero.
…et#62150)

We have two non-public methods for getting a HeaderDescriptor from a string name, one static on HeaderDescriptor and one instance on HttpHeaders.  HttpHeadersNonValidated was using the former, but this then doesn't take into account custom logic used by HttpHeaders in other methods, like TryAddWithoutValidation, which means some methods on HttpHeadersNonValidated can't find the corresponding headers.
…otnet#62180)

Past issues/PRs were re-tagged as area-System.Runtime.InteropServices and the issue labeler bot was retrained.
@joshpeterson joshpeterson deleted the bot-upstream-main-merge-2021-11-30 branch November 30, 2021 20:26
yuc434 pushed a commit that referenced this pull request Aug 8, 2022
* Initial implementation for contract customization

fix build errors

Move converter rooting to DefaultJsonTypeInfoResolver so that it can be used standalone

Fix ConfigurationList.IsReadOnly

Minor refactorings (#1)

* Makes the following changes:

* Move singleton initialization for DefaultTypeInfoResolver behind a static property.
* Consolidate JsonSerializerContext & IJsonTypeInfoResolver values to a single field.
* Move reflection fallback logic away from JsonSerializerContext and into JsonSerializerOptions

* Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializerOptions.cs

* remove testing of removed field

Simplify the JsonTypeInfo.CreateObject implemenetation (#2)

* Simplify the JsonTypeInfo.CreateObject implemenetation

* Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.cs

* Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.cs

Co-authored-by: Krzysztof Wicher <mordotymoja@gmail.com>

Co-authored-by: Krzysztof Wicher <mordotymoja@gmail.com>

Tests and fixes for JsonTypeInfoKind.None

TypeInfo type mismatch tests

Allow setting NumberHandling on JsonTypeInfoKind.None

test resolver returning wrong type of options

JsonTypeInfo/JsonPropertyInfo mutability tests

rename test file

Move default converter rooting responsibility behind DefaultJsonTypeInfoResolver (#3)

* Move default converter rooting responsibility behind DefaultJsonTypeInfoResolver

* address feedback

Add simple test for using JsonTypeInfo<T> with APIs directly taking it

fix and tests for untyped/typed CreateObject

uncomment test cases, remove todo

More tests and tiny fixes

Add a JsonTypeInfoResolver.Combine test for JsonSerializerContext (#4)

* Fix JsonTypeInfoResolver.Combine for JsonSerializerContext

* Break up failing test

Fix simple scenarios for combining contexts (#6)

* Fix simple scenarios for combining contexts

* feedback

JsonSerializerContext combine test with different camel casing

Remove unneeded virtual calls & branching when accessing Get & Set delegates (#7)

JsonPropertyInfo tests everything minus ShouldSerialize & NumberHandling

Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs

Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs

throw InvalidOperationException rather than ArgumentNullException for source gen when PropertyInfo.Name is assigned through JsonPropertyInfoValues

tests for duplicated property names and JsonPropertyInfo.NumberHandling

Add tests for NumberHandling and failing tests for ShouldSerialize

disable the failing test and add extra checks

disable remainder of the failing ShouldSerialize tests, fix working one

Fix ShouldSerialize and IgnoreCondition interop

Add failing tests for CreateObject + parametrized constructors

Fix CreateObject support for JsonConstructor types (#10)

* Fix CreateObject support for JsonConstructor types

* address feedback

Make contexts more combinator friendly (#9)

* Make contexts more combinator friendly

* remove converter cache

* redesign test to account for JsonConstructorAttribute

* Combine unit tests

* address feedback

* Add acceptance tests for DataContract attributes & Specified pattern (#11)

* Add private field serialization acceptance test (#13)

* tests, PR feedback (#14)

* PR feedback and extra tests

* Shorten class name, remove incorrect check (not true for polimorphic cases)

* Make parameter matching for custom properties map property Name with parameter (#16)

* Test static initialization with JsonTypeInfo (#17)

* Fix test failures and proper fix this time (#18)

* Fix test failures and proper fix this time

* reinstate ActiveIssueAttribute

* PR feedback and adjust couple of tests which don't set TypeInfoResolver

* fix IAsyncEnumerable tests

* Lock JsonSerializerOptions in JsonTypeInfo.EnsureConfigured()

Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>

Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.