Adding cohost completion handler (#11048)

* Basic infrastructure and HTML completion

* Moving CompletionListCache into common code

* Moved RazorCompletionListProvider and supporting infrustructure to common code

* Adding OOPRazorCompletionFactsService and moving MarkupTransitionCompletionItemProvider to common layer.

* Moving LspTagHelperCompletionService to common layer

* Move RazorCompletionListProvider to the Workspaces layer

* Add OOP MEF exports for completion services from Workspaces layer needed by RemoteCompletionService

* Hook up RazorComplelistListProvider in the RemoteCompletionService

Switch passed in and returned types from Roslyn to VS Platform LSP types since that's what all of the common completion code in the Workspaces layer uses. We will need to convert returned Roslyn completion items to VS platform LSP completion items.

* Bump Roslyn version to get new CSharp completion APIs

* Hooking up C# completion API

* Common code for completion trigger characters and correct character set used in cohosting

* Respect AutoShowCompletion setting in cohosting

* Move IsValidTrigger method to CompletionTriggerCharacters class in the workspaces layer (to be used in cohosting later)

* Call HTML completion only if we are in HTML and pass a set of existing HTML completion item labels to RazorCompletionListProvider

* Pass C# existing completion item labels to RazorCompletionListProvider and minor cleanup

* Final completion list merge logic

* Move delegated completion helper RewriteContext method into Workspace layer and use it in cohost completion request

* Provisional completion support

* Moving delegated response rewriters to the common layer

* Consuming delegated completion response re-writers in C#

Also simplifying parameters passed to the response re-writers to only what's needed.

* Switch to Roslyn CompletionParams as request input so converters are hooked up and we are getting VSInternalCompletionContext in CompletionParams

* Splitting delegated response rewriters into C# and HTML and simplifying them
They all already checked (or should've checked) for language and were operating on either C# or HTML, never on both. HTML re-writer will get called from the client and can be much simpler. In cohosting it doesn't make sense to have them all in one list since C# will get called in OOP and HTML on the client (in VS).

* Moving ShouldIncludeSnippets helper into the common layer and hooking it up in cohosting

* Adding snippets completion support to cohosting

* First part of completion options clean-up

Renamed some fields and variables dealing with "add snippets" options and added comments. We currently have two options that mean "add snippets" - one for the delegated completion, and one for Razor completion. The values of those don't correlate. The Razor one is always true in LSP and Cohost, always false for legacy editor. The delegation one actually depends on the position.

* Second part of completion options clean-up - fixing options values in cohost

* First set of tests

* Adding directives completion provider and test to cohost

* Adding directive and directive attribute completion providers and tests

Also adding a snippet completion provider test and markup transition test

* Adding test for CommitElementsWithSpace option

* Adding test for AutoInsertAttributeQuotes

* Moved most of the tests for moved code from LanguageServer to Workspaces test projects

The tests were left behind (some in this PR, some in prior PRs) when the code was moved to Workspaces layer. This commit addresses most of them other than in Delegation subworkspace

* Fixing delegated response re-writer tests.

We had inconsistent handling of null completion item labels between our response re-writers. Some handled null labels, others would through. Since label shouldn't be null (non-nullable), I adjusted the tests not to use null labels.

Also the tests previously passed because they created DelegatedCompletionListProvider with only a selected DelegatedResponseRewriter. Now the DelegatedCompletionHelper will apply all response re-writers for the correct language (either C# or HTML), which is what the product actually does, so I feel that's fine. It exposed these test failures due to inconsistent null label handling

* Add required cancellation token argument to GetGeneratedDocumentAsync in RemoteCompletionService

* Simplifying trigger character data

Switching AllTriggerCharacters to string[] since we only use it for registration/capability data, which needs string[]. and we never do look ups via Contains. Also removing rendundant property and calculations in CompletionListProvider

* Serialization attribute clean-up

* Converting DelegatedCSharpCompletionResponseRewriter to interface per CR suggestion

* Changing ShouldIncludeSnippets to take in RazorCodeDocument per CR suggestion

* Removed unnecessary null checks per CR suggestion

* PR feedback - ConfigureAwait(false), shared code for commit characters, better collection check

* Simplify applying provisional text edit

* PR test suggestions

* Adding a comment clarifying the reasoning about the return value of OOP code

* Removing item count verification and retries per PR feedback

* Switching to hardcoded directive list in tests per PR feedback

* Moving TagHelperServiceTestBase back to LanguageServer.Test project

This should probably move to the common test project eventually, but not in this PR. Moving this to the workspaces layer caused many changes in unrelated test files and changed test project references. Per PR feedback leaving this in the LanguageServer.Test project for now. We can refactor later as appropriate.

* Cleanup per PR feedback

* Fixing build post-merge

* Pass argument for supportsVsExtensions in RemoteDocumentSymbolService

Hardcoding true for now until we plumb this through Initialize. Fixes osolete API build break

* Add Debug.Fail to make the intent of returning null vs empy completion set more obvious

* Cleanup and comments per PR feedback

* One more comment per PR feedback

* Don't avertise Resolve until we provide it

* Removing unnecessary async

* C# override test

* Use remote IClientCapabilitiesService

Instead of passing client capabilities with each call to RemoteCompletionService we can now use remote IClientCapabilitiesService that's initialized with client capabilities on initial connection.

* Misc cleanup per PR feedback

eng/targets/Services.props

@@ -33,5 +33,6 @@
     <ServiceHubService Include="Microsoft.VisualStudio.Razor.GoToImplementation" ClassName="Microsoft.CodeAnalysis.Remote.Razor.RemoteGoToImplementationService+Factory" />
     <ServiceHubService Include="Microsoft.VisualStudio.Razor.SpellCheck" ClassName="Microsoft.CodeAnalysis.Remote.Razor.RemoteSpellCheckService+Factory" />
     <ServiceHubService Include="Microsoft.VisualStudio.Razor.Diagnostics" ClassName="Microsoft.CodeAnalysis.Remote.Razor.RemoteDiagnosticsService+Factory" />
+    <ServiceHubService Include="Microsoft.VisualStudio.Razor.Completion" ClassName="Microsoft.CodeAnalysis.Remote.Razor.RemoteCompletionService+Factory" />
   </ItemGroup>
 </Project>

src/Razor/benchmarks/Microsoft.AspNetCore.Razor.Microbenchmarks/LanguageServer/RazorCompletionBenchmark.cs

@@ -2,7 +2,6 @@
 // Licensed under the MIT license. See License.txt in the project root for license information.
 
 using System;
-using System.Collections.Generic;
 using System.IO;
 using System.Text;
 using System.Threading;
@@ -13,6 +12,7 @@
 using Microsoft.AspNetCore.Razor.LanguageServer.Completion.Delegation;
 using Microsoft.AspNetCore.Razor.LanguageServer.EndpointContracts;
 using Microsoft.AspNetCore.Razor.LanguageServer.Hosting;
+using Microsoft.CodeAnalysis.Razor.Completion;
 using Microsoft.CodeAnalysis.Razor.DocumentMapping;
 using Microsoft.CodeAnalysis.Razor.Logging;
 using Microsoft.CodeAnalysis.Razor.ProjectSystem;
@@ -37,13 +37,12 @@ public async Task SetupAsync()
     {
         var razorCompletionListProvider = RazorLanguageServerHost.GetRequiredService<RazorCompletionListProvider>();
         var lspServices = RazorLanguageServerHost.GetRequiredService<ILspServices>();
-        var responseRewriters = lspServices.GetRequiredServices<DelegatedCompletionResponseRewriter>();
         var documentMappingService = lspServices.GetRequiredService<IDocumentMappingService>();
         var clientConnection = lspServices.GetRequiredService<IClientConnection>();
         var completionListCache = lspServices.GetRequiredService<CompletionListCache>();
         var loggerFactory = lspServices.GetRequiredService<ILoggerFactory>();
 
-        var delegatedCompletionListProvider = new TestDelegatedCompletionListProvider(responseRewriters, documentMappingService, clientConnection, completionListCache);
+        var delegatedCompletionListProvider = new TestDelegatedCompletionListProvider(documentMappingService, clientConnection, completionListCache);
         var completionListProvider = new CompletionListProvider(razorCompletionListProvider, delegatedCompletionListProvider);
         var configurationService = new DefaultRazorConfigurationService(clientConnection, loggerFactory);
         var optionsMonitor = new RazorLSPOptionsMonitor(configurationService, RazorLSPOptions.Default);
@@ -141,12 +140,19 @@ public async Task RazorCompletionAsync()
 
     private class TestDelegatedCompletionListProvider : DelegatedCompletionListProvider
     {
-        public TestDelegatedCompletionListProvider(IEnumerable<DelegatedCompletionResponseRewriter> responseRewriters, IDocumentMappingService documentMappingService, IClientConnection clientConnection, CompletionListCache completionListCache)
-            : base(responseRewriters, documentMappingService, clientConnection, completionListCache)
+        public TestDelegatedCompletionListProvider(IDocumentMappingService documentMappingService, IClientConnection clientConnection, CompletionListCache completionListCache)
+            : base(documentMappingService, clientConnection, completionListCache)
         {
         }
 
-        public override Task<VSInternalCompletionList?> GetCompletionListAsync(int absoluteIndex, VSInternalCompletionContext completionContext, DocumentContext documentContext, VSInternalClientCapabilities clientCapabilities, Guid correlationId, CancellationToken cancellationToken)
+        public override Task<VSInternalCompletionList?> GetCompletionListAsync(
+            int absoluteIndex,
+            VSInternalCompletionContext completionContext,
+            DocumentContext documentContext,
+            VSInternalClientCapabilities clientCapabilities,
+            RazorCompletionOptions completionOptions,
+            Guid correlationId,
+            CancellationToken cancellationToken)
         {
             return Task.FromResult<VSInternalCompletionList?>(
                 new VSInternalCompletionList

src/Razor/benchmarks/Microsoft.AspNetCore.Razor.Microbenchmarks/LanguageServer/TagHelperCompletionBenchmark.cs

@@ -29,7 +29,7 @@ public class TagHelperCompletionBenchmark
     [Benchmark]
     public object GetAttributeCompletions()
     {
-        var tagHelperCompletionService = new LspTagHelperCompletionService();
+        var tagHelperCompletionService = new TagHelperCompletionService();
         var context = new AttributeCompletionContext(
             TagHelperDocumentContext.Create(prefix: null, CommonResources.TelerikTagHelpers),
             existingCompletions: [],
@@ -46,7 +46,7 @@ public object GetAttributeCompletions()
     [Benchmark]
     public object GetElementCompletions()
     {
-        var tagHelperCompletionService = new LspTagHelperCompletionService();
+        var tagHelperCompletionService = new TagHelperCompletionService();
         var context = new ElementCompletionContext(
             TagHelperDocumentContext.Create(prefix: null, CommonResources.TelerikTagHelpers),
             existingCompletions: s_existingElementCompletions,

src/Razor/benchmarks/Microsoft.AspNetCore.Razor.Microbenchmarks/Serialization/CompletionListSerializationBenchmark.cs

@@ -22,10 +22,8 @@ public class CompletionListSerializationBenchmark
 
     public CompletionListSerializationBenchmark()
     {
-        var completionService = new LspTagHelperCompletionService();
-        var configurationService = new BenchmarkConfigurationSyncService();
-        var optionsMonitor = new RazorLSPOptionsMonitor(configurationService, RazorLSPOptions.Default);
-        var tagHelperCompletionProvider = new TagHelperCompletionProvider(completionService, optionsMonitor);
+        var completionService = new TagHelperCompletionService();
+        var tagHelperCompletionProvider = new TagHelperCompletionProvider(completionService);
 
         var documentContent = "<";
         var queryIndex = 1;

src/Razor/src/Microsoft.AspNetCore.Razor.LanguageServer/Completion/CompletionListProvider.cs

@@ -3,11 +3,11 @@
 
 using System;
 using System.Collections.Generic;
-using System.Collections.Immutable;
 using System.Linq;
 using System.Threading;
 using System.Threading.Tasks;
 using Microsoft.AspNetCore.Razor.LanguageServer.Completion.Delegation;
+using Microsoft.CodeAnalysis.Razor.Completion;
 using Microsoft.CodeAnalysis.Razor.ProjectSystem;
 using Microsoft.VisualStudio.LanguageServer.Protocol;
 
@@ -22,25 +22,27 @@ public CompletionListProvider(RazorCompletionListProvider razorCompletionListPro
     {
         _razorCompletionListProvider = razorCompletionListProvider;
         _delegatedCompletionListProvider = delegatedCompletionListProvider;
-
-        var allTriggerCharacters = razorCompletionListProvider.TriggerCharacters.Concat(delegatedCompletionListProvider.TriggerCharacters);
-        var distinctTriggerCharacters = new HashSet<string>(allTriggerCharacters);
-        AggregateTriggerCharacters = distinctTriggerCharacters.ToImmutableHashSet();
     }
 
-    public ImmutableHashSet<string> AggregateTriggerCharacters { get; }
-
     public async Task<VSInternalCompletionList?> GetCompletionListAsync(
         int absoluteIndex,
         VSInternalCompletionContext completionContext,
         DocumentContext documentContext,
         VSInternalClientCapabilities clientCapabilities,
+        RazorCompletionOptions razorCompletionOptions,
         Guid correlationId,
         CancellationToken cancellationToken)
     {
         // First we delegate to get completion items from the individual language server
-        var delegatedCompletionList = IsValidTrigger(_delegatedCompletionListProvider.TriggerCharacters, completionContext)
-            ? await _delegatedCompletionListProvider.GetCompletionListAsync(absoluteIndex, completionContext, documentContext, clientCapabilities, correlationId, cancellationToken).ConfigureAwait(false)
+        var delegatedCompletionList = CompletionTriggerAndCommitCharacters.IsValidTrigger(_delegatedCompletionListProvider.TriggerCharacters, completionContext)
+            ? await _delegatedCompletionListProvider.GetCompletionListAsync(
+                absoluteIndex,
+                completionContext,
+                documentContext,
+                clientCapabilities,
+                razorCompletionOptions,
+                correlationId,
+                cancellationToken).ConfigureAwait(false)
             : null;
 
         // Extract the items we got back from the delegated server, to inform tag helper completion
@@ -49,17 +51,19 @@ public CompletionListProvider(RazorCompletionListProvider razorCompletionListPro
             : null;
 
         // Now we get the Razor completion list, using information from the actual language server if necessary
-        var razorCompletionList = IsValidTrigger(_razorCompletionListProvider.TriggerCharacters, completionContext)
-            ? await _razorCompletionListProvider.GetCompletionListAsync(absoluteIndex, completionContext, documentContext, clientCapabilities, existingItems, cancellationToken).ConfigureAwait(false)
+        var razorCompletionList = CompletionTriggerAndCommitCharacters.IsValidTrigger(_razorCompletionListProvider.TriggerCharacters, completionContext)
+            ? await _razorCompletionListProvider.GetCompletionListAsync(
+                absoluteIndex,
+                completionContext,
+                documentContext,
+                clientCapabilities,
+                existingItems,
+                razorCompletionOptions,
+                cancellationToken).ConfigureAwait(false)
             : null;
 
         var finalCompletionList = CompletionListMerger.Merge(razorCompletionList, delegatedCompletionList);
 
         return finalCompletionList;
     }
-
-    private bool IsValidTrigger(ImmutableHashSet<string> triggerCharacters, VSInternalCompletionContext completionContext)
-        => completionContext.TriggerKind != CompletionTriggerKind.TriggerCharacter ||
-           completionContext.TriggerCharacter is null ||
-           triggerCharacters.Contains(completionContext.TriggerCharacter);
 }

src/Razor/src/Microsoft.AspNetCore.Razor.LanguageServer/Completion/Delegation/DelegatedCompletionItemResolver.cs

@@ -8,6 +8,7 @@
 using System.Threading;
 using System.Threading.Tasks;
 using Microsoft.AspNetCore.Razor.LanguageServer.Hosting;
+using Microsoft.CodeAnalysis.Razor.Completion;
 using Microsoft.CodeAnalysis.Razor.Formatting;
 using Microsoft.CodeAnalysis.Razor.ProjectSystem;
 using Microsoft.CodeAnalysis.Razor.Protocol;