Semantic Snippets

[jmarolf]

Snippets as they exist today for .NET developers only do text expansion (cw -> Console.WriteLine) and have lacked any significant improvements in the last 10+ years. We would like to iterate on this experience for .NET developers using Visual Studio. Snippets based on the rich analysis capabilities of the roslyn api that can change based on context.

Proposal

1. Do not show snippets in the completion list for C#
2. For the set of some 40 snippets we ship for the language replace them with "snippets" written using the completion provider API
3. Work with same teams as we do no roslyn-analyzer to write api-specific snippets and ship them in the SDK
4. Still ship snippets so muscle memory like cw -> Console.WriteLine(); will work

Advantages

The CompletionProvider api has lots of advanced capabilities that allow us to be semantically aware of what is around the snippet. I've been playing around with what the API is capable of and there are a lot of possibilities.

• Only show completion for snippets when it is syntactically correct to do so
• Advanced filtering and sorting capabilities available to us via the completion api
• Populate values from members that are in scope (automatically include cancellation tokens etc.)
• Have different behavior depending on where the cursor is (surround with vs. expansion can just be where on the line you are typing)
• completion providers can come from nuget packages so we can have other teams such as the BCL

Open Implementation Questions

• In our own implementations we would use the Speculative Semantic Model and other tricks to ensure good performance, some of these helpers may need to get ported to like what was done for roslyn-analyzers if we want other teams to contribute
• We currently can create tab stop behavior by synthesizing an xml snippet on demand. Its unclear if this approach is something we want to continue doing.

Open Design Questions

1. What should the pre-selection and filtering behavior be? If the snippets are targeted enough, it may be sufficient for their behavior to be simple.
2. If the user has third-party snippets installed, we should still show those but the ~40 we own should not appear
3. Potentially add a Tools->Option toggle to show snippets again and/or disable this feature. Its unclear how people will react to the old snippets not being in the completion list.

Appendix

C# Snippet Behavior in Visual Studio

List of all snippets C# included in Visual Studio

Here is a list of all 39 "Expansion" snippets we ship for C# today. These appear in the completion list as their snippet text entry.

Snippet Text	Description
#if	if region directing
#region	region directive
attribute	define new attribute class (25 lines)
checked	checked statement
class	define new class
ctor	define new parameter-less constructor
cw	Console.WriteLine
do	do…while loop
else	else statement
enum	define a new enum
equals	override Equals(object obj) and `GetHashCode() methods (27 lines)
exception	define a new exception type (10 lines)
for	for loop
foreach	foreach block
forr	Reverse for loop
if	if statement
indexer	indexer property
interface	define a new interface
iterindex	define an interator class with and index property (43 lines)
iterator	define IEnumerator<ElementType> GetEnumerator()method
lock	lock statement
mbox	show message box (System.Windows.Forms.MessageBox.Show("Test");)
namespace	define a new namespace
prop	public int MyProperty { get; set; }
propfull	property with backing field (8 lines)
propg	Property with a get accessor and private set accessor
sim	int Main method
struct	define a new struct
svm	void Main method
switch	switch statement
testc	define new MSTest Test Class
testm	define new MSTest Test Method
try	try catch
tryf	try finally
unchecked	unchecked block
unsafe	unsafe block
using	using statement
while	while loop
~	deconstructor

Here is a list of all 21 "Surrounds With" snippets we ship for C# today. These require the user to select text and press te "Surround With" cord (Ctrl+K+S) and are never shown in the completion list.

Snippet Text	Description
#if	if region directing
#region	region directive
checked	checked statement
class	define new class
do	do…while loop
else	else statement
enum	define a new enum
for	for loop
foreach	foreach block
forr	Reverse for loop
if	if statement
interface	define a new interface
lock	lock statement
namespace	define a new namespace
struct	define a new struct
try	try catch
tryf	try finally
unchecked	unchecked block
unsafe	unsafe block
using	using statement
while	while loop

Visual Basic Snippet Behavior in Visual Studio

The VB snippet design is different they ship about 100 snippets that offer a more expansive set of capabilities however, they do not show them in the completion list and it is not expected that the user types out text such as cw or prop and gets the text expanded. Instead the user is expected to invoke the snippet menu (Ctrl+K+X).

Visual Studio Snippet Capabilities

Some relevant details pulled from docs

See the snipper schema docs here for more information.
There are two kinds of snippets that exist today in Visual Studio

• Surrounds With: surrounds a users selection
• Expansion: text expansion (i.e. cw -> Console.WriteLine();)

These VS snippets are allowed to have the following pre-defined variables

• $selected$ is a predefined variable. It represents the text that was selected in the editor before invoking the snippet. The placement of this variable determines where the selected text appears in the code snippet that surrounds that selection.
• $end$ is a predefined variable. When the user presses Enter to finish editing the code snippet fields, this variable determines where the caret (^) is moved to.

You can also define your own variables in the VS snippet engine and give them a default text but allow the user to tab to them and edit them on snippet insertion.

Each snippets can also have an Assembly element indicating that an <AssemblyReference> should be added to the project file if not present and an Import element that adds usings to the top of the file if not present.

VS Code Snippet Capabilities

Some relevant details pulled from docs

For more information see the docs here

VS Code's snippet system is similar to Visual Studio's in that it has a "trigger phrase" (such as for) that it uses as the decider for when to begin expansion, however its text transformation capabilities are a more dynamic as it allows regular expressions to be run over the output of a snippet.

The EBNF form of the regex syntax is defined as follows:

any         ::= tabstop | placeholder | choice | variable | text
tabstop     ::= '$' int
                | '${' int '}'
                | '${' int  transform '}'
placeholder ::= '${' int ':' any '}'
choice      ::= '${' int '|' text (',' text)* '|}'
variable    ::= '$' var | '${' var '}'
                | '${' var ':' any '}'
                | '${' var transform '}'
transform   ::= '/' regex '/' (format | text)+ '/' options
format      ::= '$' int | '${' int '}'
                | '${' int ':' '/upcase' | '/downcase' | '/capitalize' | '/camelcase' | '/pascalcase' '}'
                | '${' int ':+' if '}'
                | '${' int ':?' if ':' else '}'
                | '${' int ':-' else '}' | '${' int ':' else '}'
regex       ::= JavaScript Regular Expression value (ctor-string)
options     ::= JavaScript Regular Expression option (ctor-options)
var         ::= [_a-zA-Z] [_a-zA-Z0-9]*
int         ::= [0-9]+
text        ::= .*

They have the following predefined variables:

• TM_SELECTED_TEXT The currently selected text or the empty string
• TM_CURRENT_LINE The contents of the current line
• TM_CURRENT_WORD The contents of the word under cursor or the empty string
• TM_LINE_INDEX The zero-index based line number
• TM_LINE_NUMBER The one-index based line number
• TM_FILENAME The filename of the current document
• TM_FILENAME_BASE The filename of the current document without its extensions
• TM_DIRECTORY The directory of the current document
• TM_FILEPATH The full file path of the current document
• RELATIVE_FILEPATH The relative (to the opened workspace or folder) file path of the current document
• CLIPBOARD The contents of your clipboard
• WORKSPACE_NAME The name of the opened workspace or folder
• WORKSPACE_FOLDER The path of the opened workspace or folde

Example	Output	Explanation
${TM_FILENAME/[\\.]/_/}	example-123_456-TEST.js	Replace the first . with _
${TM_FILENAME/[\\.-]/_/g}	example_123_456_TEST_js	Replace each . or - with _
${TM_FILENAME/(.*)/${1:/upcase}/}	EXAMPLE-123.456-TEST.JS	Change to all uppercase
${TM_FILENAME/[^0-9^a-z]//gi}	example123456TESTjs	Remove non-alphanumeric characters

User Experiences

1.1 Snippet Discovery

A user starts typing new so they can start writing a new constructor. They are unaware of the built-in snippets Visual Studio provides but start typing the code for what they are trying to accomplish:

We will show the snippet for a constructer as soon as the type new as it is on our list of synonyms for that snippet:

If the user commits the snippet, it is persisted to the buffer as the “ghost text” preview had shown:

If the user dismisses the completion list then the editor is placed back as it was with new being left alone:

1.2 Using an Existing Snippet

A user types ctor (the text expansion for an existing snippet into the editor):

When completion comes up, we will do custom filtering to: A. Select the new constructor completion item even though none of the text they’ve entered matches the text in this item. B. Use the ghost text API to show the user what will happen if they commit the completion item.

If the user commits the snippet, it is persisted to the buffer as the ghost text preview had shown:

If the user dismissed the completion list then the editor is placed back as it was with ctor being left alone:

1.3 Semantic Awareness

Today the editor shows and commits snippets even if they do not make sense. Here the developer is being suggested the “for” snippet even though it can never be valid inside of a type:

In the new experience snippets will be aware of the context they are appearing in and only show if
they are reasonable. Notice we will now suggest FormattableString as a completion item assuming the
user is writing out a new property or field in this type:

However if the user is in a situation where this snippet makes sense we will: A. Filter to it in the completion list based on hard-coded synonyms. B. Show a ghost text preview of what the committed code will look like. C. Populate the snippet based on the semantic context in which it appears (notice how points.Length is automatically used in the snippet).

If the user commits the snippet, we will use the existing snippet expansion experience that Visual Studio offers today to allow the user to fill in the placeholders that are automatically generated for them:

If the user dismisses completion, they are left with the same contents in their buffer as they had before:

---

Tracking Issues

• Update snippets to use modern language features (eg "nameof") #5574
• foreach autocompletion should be offered when ctrl+. is pressed on a variable representing a collection #60628
• Snippets: If snippet and use of LSP's SnippetExpander #60860
• adding draft of detailed spec #57346
• Semantic Snippets: Console snippet first look #59303

[JoeRobich]

Notes from Design Review:

• Questions:
  • Should we wait for new Editor snippet experience?
    • Is a new snippet experience on the roadmap?
    • Would it cover our expected scenarios?
  • Will we work with the VS for Mac snippet engine?
  • Will single tab insert these new snippets?
    • Argument completion is adding an option for it to trigger on single tab, snippets could inherit this option/behavior.
    • 3rd party snippets implemented by completion provider would need special consideration to trigger the snippet behavior.
  • Will this clash with IntelliCode and whole line completion?

Proposal: Update current snippet provider to show description text in completion list. Validate the idea by porting the most used snippets.

[KirillOsenkov]

One thing that I don't think we can break is the presence of snippets in the completion list. Typing foreach + TAB + TAB is very important muscle memory for many users. One thing we can do better is only show snippets that make sense in the context, e.g. foreach is a statement so don't show it in the expression context. If users don't see foreach in completion it might cause confusion.

Also another important aspect of the experience is TAB TAB (snippet insertion) can create several fields and linked fields (represented as tags in text) that you can tab through, edit and finally commit using Enter. VB snippets work differently.

Also note that some snippets have special sauce, e.g. ctor, and switch for enums will expand the enum fields.

Two invocation shortcuts are also important: Ctrl+K,X (Insert Snippet) and Ctrl+K,S (Surround With). Surround With is heavily used, Insert Snippet perhaps not as much.

From the implementation perspective, I think if you get your prototype working in VS Windows, it should be relatively straightforward to make it work in VSMac as well. I anticipate no significant hurdles there. So I'd say when a preview of the working functionality is ready, we can take another look and try to make it work in VSMac as well.

[jmarolf]

@mikadumont the merged spec can be found here: https://github.com/dotnet/roslyn/blob/feature/semantic-snippets/docs/ide/specs/semantic_snippets.md

[jmarolf]

related to #5574

[neuecc]

ConsoleStream only has sync write.
https://source.dot.net/#System.Console/System/IO/ConsoleStream.cs,c0e0ba628fa4c224
UnixConsoleStream:
https://source.dot.net/#System.Console/System/ConsolePal.Unix.ConsoleStream.cs,13bf065fbb4646d5
WindowsConsoleStream:
https://github.com/dotnet/runtime/blob/f9f7f8079377039a8d42269434a41ca55d14d591/src/libraries/System.Console/src/System/ConsolePal.Windows.cs#L1076

In other words, I don't think Async makes sense.

[CyrusNajmabadi]

@neuecc : https://learn.microsoft.com/en-us/dotnet/api/system.io.textwriter.writelineasync?view=net-7.0

[neuecc]

Console.Out creates
new StreamWrite(ConsolePal.OpenStandardOutput())
Eventually, StreamWriter(TextWriter) calls WriteAsync and FlushAsync of the internal Stream.
If they are not overridden, they are in a "sync over async" state on the ThreadPool, simulating asynchronous operation.

[CyrusNajmabadi]

Feel free to give the runtime feedback if you feel that WriteLineAsync will have problems on console streams. In the worst case, you will have an impl that is as bad as WriteLine would be. In hte best case, you have an impl that is a good async citizen. The snippets encode best practices for async code to ensure that you're doing the right hting, and you can benefit from any improvements the runtime makes now and in the future. Emitting WriteLine would make that not possible.

[neuecc]

Will there ever be a truly asynchronous implementation of ConsoleOut? If not, we should revise the snippet after such an implementation is established. As it stands, we're implementing a snippet that performs worse than WriteLine and is less user-friendly.

However, if we're concerned about performance, we should avoid writing to the console every time in the first place.

[KirillOsenkov]

I’m more worried about breaking people’s muscle memory. Why not add ‘cwa’ for the Async version and keep ‘cw’ the classic version.

Anecdotally, 100% of usages I’m seeing is console apps that are sync, as well as small throwaway Program.cs programs and tools. I’ve never seen Console.WriteLine in an async context.

[CyrusNajmabadi]

Will there ever be a truly asynchronous implementation of ConsoleOut?

I don't see any reason why not. Esp. as many OS' are happy to provide async io.

Why not add ‘cwa’ for the Async version and keep ‘cw’ the classic version.

The very intentional idea here was that you just use one snippet and you get the best emit per context. It's not on you to remember which to use and when, the system figures it out for you.

[CyrusNajmabadi]

Anecdotally, 100% of usages I’m seeing is console apps that are sync, as well as small throwaway Program.cs programs and tools. I’ve never seen Console.WriteLine in an async context.

If you have a sync app, then we'll emit C.WL. We only emit WLA if it's an async context. :-)

[CyrusNajmabadi]

As it stands, we're implementing a snippet that performs worse than WriteLine and is less user-friendly.

Why would it perform worse than WL?

[CyrusNajmabadi]

However, if we're concerned about performance, we should avoid writing to the console every time in the first place.

I see no relation between those ideas. You can want to write to the console and still not block a thread. Perf and async IO go hand in hand.

[neuecc]

Sync over Async is a BAD PRACTICE for async, right?
I am saying that is the situation.

[neuecc]

Note that using ConsoleWrite for StandardOut is slow, so I have created my own library to handle this.
https://github.com/Cysharp/ZLogger

[mayuki]

I understand that it is generally preferred to use async methods within async contexts, and I think that the ability to switch snippets according to the context is great.

However, when it comes to Console, its implementation is not asynchronous on many platforms, and making it asynchronous incurs additional costs (Task allocation, etc.). Many use cases of Console.WriteLine simply display messages or are used for printf debugging and are not directly used in cases where performance is required.

Above all, Console.WriteLine is used as an idiom called Cosnole.WriteLine regardless of whether it is in an async context or not.

For example, if I write a simple console tool with top-level statements and have a line like await new HttpClient().GetStringAsync("http://www.google.com");, it will become an async context, but it seems unfamiliar to have Console.Out.WriteLineAsync after that. (It doesn’t happen now.)

I think it would be best to keep them separate as cw and cwa.

[MisinformedDNA]

If Console.Out.WriteLineAsync is always sync over async, then there is no benefit to it. @stephentoub has spoken out against the practice.

[stephentoub]

Console.Out is always a synchronized text writer (TextWriter.Synchronized), with all operations guarded by a Monitor; all operations that write to the console are guarded by the same lock, and any time we've tried to change that, we've had to revert because significant numbers of apps have taken dependencies on being able to do things like lock (Console.Out) { Console.ForegroundColor = ...; ... multiple writes ...; Console.ResetColor(); }. Async operations on such a writer get converted into synchronous ones. Using await Console.Out.WriteLineAsync(...) instead of Console.WriteLine(...) has no upside and only overhead / more complicated code.

[neuecc]

VS 17.8.0 Preview 2.0 is still Console.Out.WriteLineAsync.

[CyrusNajmabadi]

Unclear what is left here @akhera99 . Can yuo please update or close this out?

[CyrusNajmabadi]

Unclear what is left here @akhera99 . Can yuo please update or close this out?

[akhera99]

Closing since the working list of snippets is being updated here: #64144