Merge main to tokenizer branch #10872
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ence-packages build 20240805.2 Microsoft.SourceBuild.Intermediate.source-build-reference-packages From Version 9.0.0-alpha.1.24374.1 -> To Version 9.0.0-alpha.1.24405.2
…812.1 Microsoft.SourceBuild.Intermediate.arcade , Microsoft.DotNet.Arcade.Sdk From Version 8.0.0-beta.24376.1 -> To Version 8.0.0-beta.24412.1
…814.3 Microsoft.SourceBuild.Intermediate.arcade , Microsoft.DotNet.Arcade.Sdk From Version 9.0.0-beta.24408.2 -> To Version 9.0.0-beta.24414.3
…813.2 Microsoft.SourceBuild.Intermediate.arcade , Microsoft.DotNet.Arcade.Sdk From Version 8.0.0-beta.24412.1 -> To Version 8.0.0-beta.24413.2
fixup semantic tokens tweask
…816.2 Microsoft.SourceBuild.Intermediate.arcade , Microsoft.DotNet.Arcade.Sdk From Version 9.0.0-beta.24414.3 -> To Version 9.0.0-beta.24416.2
…ence-packages build 20240815.3 Microsoft.SourceBuild.Intermediate.source-build-reference-packages From Version 9.0.0-alpha.1.24405.2 -> To Version 10.0.0-alpha.1.24415.3
Most places already passed an ImmutableArray here. I left the method itself being an iterator, as 50% of callers need an ImmutableArray result, but the other 50% need an array, so no clear winner.
All callers did the conversion anyway
A clean error list is a good error list.
…ence-packages build 20240905.1 Microsoft.SourceBuild.Intermediate.source-build-reference-packages From Version 10.0.0-alpha.1.24428.1 -> To Version 10.0.0-alpha.1.24455.1
…tension method on RazorCodeDocument (dotnet#10851) For a long while, the `GetLanguageKind(...)` method that determines whether an index into a document falls within Razor, C# or HTML has been a bit of a wart on the `IDocumentMappingService`. It really isn't part of document mapping, and its implementation is completely distinct. In fact, making the actual change is quite simple, so why hadn't it been done yet? The answer is mocking. There are several tests that mock `IDocumentMappingService.GetLanguageKind(...)` to lie about test inputs. In my not-so-humble opinion, this represents an abuse of mocking. Instead of setting up tests to have the necessary inputs that ensure `GetLanguageKind(...)` would return a real and correct result, the inputs would often be garbage and an `IDocumentMappingService` mock would lie about the `GetLanguageKind(...)` result at a particular location. This makes moving `GetLanguageKind(...)` off of `IDocumentMappingService` a much larger change than it needs to be. This is why there are substantial test changes in this PR. Don't misunderstand me as a mocking hater! Mocking libraries are definitely useful! In fact, there are new mocks used in this very PR! However, mocks should be used judiciously and thoughtfully, and in this case, a mock was used to write lazy tests. Fixes dotnet#8774
dotnet#10858) This pull request updates the following dependencies [marker]: <> (Begin:011df26a-fbd1-45b0-94b9-08db3601dcca) ## From https://github.com/dotnet/source-build-reference-packages - **Subscription**: 011df26a-fbd1-45b0-94b9-08db3601dcca - **Build**: 20240905.1 - **Date Produced**: September 5, 2024 10:13:43 PM UTC - **Commit**: ad3c9aa85596f42c6a483233c50fab8cee8c412a - **Branch**: refs/heads/main [DependencyUpdate]: <> (Begin) - **Updates**: - **Microsoft.SourceBuild.Intermediate.source-build-reference-packages**: [from 10.0.0-alpha.1.24428.1 to 10.0.0-alpha.1.24455.1][1] [1]: dotnet/source-build-reference-packages@6bcf90f...ad3c9aa [DependencyUpdate]: <> (End) [marker]: <> (End:011df26a-fbd1-45b0-94b9-08db3601dcca)
…dit` (dotnet#10855) Fixes dotnet#10842 The formatting self-nerd-sniping continues. The formatting engine was written to use the LSP `TextEdit` class, which makes some sense, but also uses Roslyn APIs like `SourceText` a lot, which uses the `TextChange` struct instead. This meant lots of code to convert to and from the two types. Changing the whole formatting engine over to `TextChange`, and using more `TextSpan`, `LinePositionSpan` etc. removes a lot of this code. It also makes a lot more sense in cohosting, to boot. I wouldn't claim that I've gone through and improved the perf of the formatting engine, but rather I've use the changes to lead me to things that need fixing. ie, I started out moving from `TextEdit[]` to `ImmutableArray<TextChange>`, and this let me to places where pooled array builders could be used, and places where `Range` and `Position` were used which didn't make much sense, and then the constructor for `LinePosition` threw at one point because it turns out we were only using the `Line` property from the `Position` that used to be used, and so never validated the characters, so that API moved to `int`, etc. TL;DR the commits tell the story, and there could well be something I missed, if it never came across my plate for another reason.
…enizer * upstream/main: (270 commits) Fix after merge PR Feedback Update dependencies from https://github.com/dotnet/source-build-reference-packages build 20240905.1 Fix auto insert service after merge from main PR Feedback Use ImmutableArray in SourceTextDiffer too Create a helper method and revert change to shared code, just in case Convert HtmlFormatter to ImmutableArray<TextChange> IEnumarable to ImmutableArray Remove some more usage of LSP types, and simplify ranges to line numbers Use pooled collections in a few more spots Extract common code to helper method Rename some methods to Try... pattern Rename some variables etc. Fix broken tests Get all consuming code compiling again Convert TextEdit to TextChange Remove flaky test OnAutoInsert Cohosting Tests (dotnet#10829) Update GetLanguageKind(...) tests and move to Workspaces.Test ...
333fred
commented
Sep 10, 2024
Comment on lines
+66
to
+68
| public CSharpParseOptions CSharpParseOptions => _options.CSharpParseOptions; | ||
|
|
||
| public bool StartOfLine { get; set; } = true; | ||
| public bool ParseLeadingDirectives => _options.ParseLeadingDirectives; |
There was a problem hiding this comment.
Moved these to pull from _options, rather than being auto props.
|
|
||
| var context = new ParserContext(source, Options); | ||
| using var context = new ParserContext(source, Options); |
There was a problem hiding this comment.
Adjacent line changes required merging here.
|
@dotnet/razor-compiler for review |
You could tokenize them. |
jjonescz
approved these changes
Sep 11, 2024
| @@ -203,9 +203,10 @@ public TestHtmlMarkupParser(ParserContext context) | |||
| EnsureCurrent(); | |||
| } | |||
|
|
|||
| public void Dispose() | |||
| void IDisposable.Dispose() | |||
There was a problem hiding this comment.
Because I implemented IDisposable on the base class, which conflicts with this derived implementation.
This was referenced Nov 4, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Will call out the merge locations.