Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Semantic search #71268

Merged
merged 13 commits into from
Apr 2, 2024
Merged

Semantic search #71268

merged 13 commits into from
Apr 2, 2024

Conversation

tmat
Copy link
Member

@tmat tmat commented Dec 14, 2023

Implements Semantic Search IDE feature.

There are a few parts to it:

  1. Semantic Tool Window UI (View > Other Windows > C# Semantic Search)

image

The window is a Gladstone Tool Window. Currently, we need to construct the XAML tree for the window content manually due to some Gladstone limitations (tracked as TODOs in code). The window implementation should simplify significantly once these limitations are removed.

Like in Interactive Window, we use Ctrl+Enter to run the query and Escape to cancel running query.
Ctrl+Enter overrides "insert empty line above" editor command. We can come up with another key binding to avoid the override but Ctrl+Enter is very intuitive for "submission".

The implementation is in src\VisualStudio\CSharp\Impl\SemanticSearch\SemanticSearchToolWindow.cs.

  1. Semantic Search Workspace

A new workspace with kind SemanticWorkspace that has a solution with a single C# project that contains the query document (content of the tool window editor) and provides IDE services.

  1. Semantic Search service

This service runs in OOP only (and only on .NET 8+). The service creates a compilation, emits in-memory assembly with instrumentation injecting stack overflow and cancellation probes, executes Find method defined in the compilation and streams resulting symbols back in-proc as DefinitionItems to be displayed in FAR window.

  1. Semantic Search Reference Assemblies

We limit the set of APIs the search can use to Roslyn APIs and runtime APIs that we deem safe to run in our OOP.
The assemblies are generated from Roslyn CodeAnalysis assemblies and runtime reference assemblies by a build task implemented in Tools/SemanticSearch/BuildTask.

The task takes an assembly and text file that lists include/exclude patterns of API names in doc comment format (similar to banned APIs) and writes out an assembly, where excluded types and members are marked "private". The output assembly has a unique mvid and is meant only to be used as a reference assembly used for Semantic Search.

Follow up work:
#72585
#72617

SemanticSearch

@dotnet-issue-labeler dotnet-issue-labeler bot added Area-Infrastructure untriaged Issues and PRs which have not yet been triaged by a lead labels Dec 14, 2023
@tmat tmat force-pushed the SemanticSearch branch 4 times, most recently from 5cb898b to 6b8c3b0 Compare February 8, 2024 04:50
@tmat tmat force-pushed the SemanticSearch branch 3 times, most recently from bdc6a87 to 6c45020 Compare February 13, 2024 22:51
@tmat tmat force-pushed the SemanticSearch branch 4 times, most recently from effb755 to 84a30bd Compare March 1, 2024 20:08
@tmat tmat force-pushed the SemanticSearch branch 4 times, most recently from d3f4fb2 to 7470447 Compare March 15, 2024 21:25
@tmat tmat marked this pull request as ready for review March 18, 2024 19:30
@tmat tmat requested review from a team as code owners March 18, 2024 19:30
@tmat
Copy link
Member Author

tmat commented Mar 18, 2024

@dotnet/roslyn-ide ptal

Copy link
Member

@sharwell sharwell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feature must be able to operate when ServiceHub is disabled.

@tmat tmat disabled auto-merge March 29, 2024 22:38
Copy link
Member

@genlu genlu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks good to me (except for the part involves metadata manipulation, which is way over my head, could use some detailed comments there 😄)

@CyrusNajmabadi
Copy link
Member

Could you include some gifs of the whole experience in action? Thanks :)

tmat and others added 2 commits April 1, 2024 11:27
Co-authored-by: Cyrus Najmabadi <cyrus.najmabadi@gmail.com>
@tmat tmat merged commit 39ae946 into dotnet:main Apr 2, 2024
27 of 30 checks passed
@tmat tmat deleted the SemanticSearch branch April 2, 2024 22:26
@dotnet-policy-service dotnet-policy-service bot added this to the Next milestone Apr 2, 2024
333fred added a commit to 333fred/roslyn that referenced this pull request Apr 3, 2024
* upstream/main: (416 commits)
  Semantic search (dotnet#71268)
  Make more static
  Fix MEF import of IExternalCSharpCopilotCodeAnalysisService to allow null
  Make static
  Make private
  Add comments
  Add method name to TimeInQueue telemetry (dotnet#72841)
  switch to frozen
  Simplify
  Add test
  Downstream
  Use singular helper when creating checksumsw
  Use singular helper when creating checksumsw
  Remove ability for a project to change its language
  Revert "Avoid creating result temp for is-expressions (dotnet#72273)" (dotnet#72827)
  Localized file check-in by OneLocBuild Task: Build definition ID 327: Build ID 2420199
  Localized file check-in by OneLocBuild Task: Build definition ID 327: Build ID 2420199
  Localized file check-in by OneLocBuild Task: Build definition ID 327: Build ID 2420199
  Localized file check-in by OneLocBuild Task: Build definition ID 327: Build ID 2420199
  Improve generic type argument list error recovery (dotnet#69734)
  ...
@dibarbet dibarbet modified the milestones: Next, 17.11 P1 Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Infrastructure untriaged Issues and PRs which have not yet been triaged by a lead
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants