Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Mark Mode spec (add-on) #5804

Merged
merged 31 commits into from
Jul 22, 2022
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
51e8b1a
Keyboard Selection Spec
carlos-zamora Sep 20, 2019
bc668e0
PR changes (first of many)
carlos-zamora Sep 27, 2019
67c6334
Updated to reflect Keybinding Args change
carlos-zamora Nov 22, 2019
4f38557
include 'shift' in keybindings
carlos-zamora Nov 22, 2019
de95782
more mark mode details. Added responses to PR feedback into the spec
carlos-zamora Nov 27, 2019
23b425e
remove mark mode for separate PR (easier discussion)
carlos-zamora May 8, 2020
f71275c
PR changes
carlos-zamora May 8, 2020
81c5a7c
Merge branch 'master' into dev/cazamor/spec-keyboard-selection
carlos-zamora Jun 23, 2020
de8dc2f
Merge branch 'master' into dev/cazamor/spec-keyboard-selection
carlos-zamora Jun 23, 2020
3af5ae6
update default bindings
carlos-zamora Jun 23, 2020
29b00b9
Introduce Mark Mode Spec (add-on)
carlos-zamora May 8, 2020
f23c033
add mouse interaction + corner cases
carlos-zamora May 13, 2020
8e1cebd
whoops. Forgot to save haha
carlos-zamora May 13, 2020
dd0720e
remove 'hold' mode
carlos-zamora May 28, 2020
dbf1e4e
better images and structure
carlos-zamora May 28, 2020
dcc27b6
update date and misspelling
carlos-zamora May 28, 2020
db0e1d3
update with meeting results
carlos-zamora Jun 26, 2020
0b58885
merge master + fix spelling
carlos-zamora Jul 14, 2020
ecc88d1
merge main branch
carlos-zamora Jul 14, 2020
7e46f8b
Y-Beam Mockups and Design
carlos-zamora Jul 17, 2020
e2cad94
address zadji comments
carlos-zamora Aug 4, 2020
246583b
exclude vsdx from spell checker
carlos-zamora Aug 4, 2020
5070c2c
Merge branch 'main' into dev/cazamor/spec-ks/mark-mode
carlos-zamora Sep 23, 2021
314d70c
update with upstream changes and address some missed feedback from Du…
carlos-zamora Sep 23, 2021
ec24abf
update spec to use simpler mark mode design
carlos-zamora Jun 1, 2022
1e0bf1f
Merge branch 'main' into dev/cazamor/spec-ks/mark-mode
carlos-zamora Jun 1, 2022
6ff24a6
spellcheck
carlos-zamora Jun 2, 2022
27022e7
add more details on the selection markers
carlos-zamora Jul 6, 2022
c34cce1
address feedback from zadji
carlos-zamora Jul 18, 2022
f730f17
words are hadr
carlos-zamora Jul 18, 2022
29bcb2e
Merge branch 'main' into dev/cazamor/spec-ks/mark-mode
DHowett Jul 22, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/actions/spelling/excludes.txt
Original file line number Diff line number Diff line change
Expand Up @@ -80,4 +80,5 @@ SUMS$
^tools/ReleaseEngineering/ServicingPipeline.ps1$
^\.github/actions/spelling/
^\.gitignore$
\.vsdx$
^\XamlStyler.json$
1 change: 1 addition & 0 deletions .github/actions/spelling/expect/expect.txt
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,7 @@ conwinuserrefs
coord
coordnew
COPYCOLOR
copymode
CORESYSTEM
cotaskmem
countof
Expand Down
Original file line number Diff line number Diff line change
@@ -1,26 +1,33 @@
---
author: Carlos Zamora @carlos-zamora
created on: 2019-08-30
last updated: 2021-09-17
last updated: 2022-06-1
issue id: 715
---

# Keyboard Selection

## Abstract

This spec describes a new set of non-configurable keybindings that allows the user to update a selection without the use of a mouse or stylus.
This spec describes a new set of key bindings that allows the user to create and update a selection without the use of a mouse or stylus.

## Inspiration

ConHost allows the user to modify a selection using the keyboard. Holding `Shift` allows the user to move the second selection endpoint in accordance with the arrow keys. The selection endpoint updates by one cell per key event, allowing the user to refine the selected region.

Mark mode allows the user to create a selection using only the keyboard, then edit it as mentioned above.
### Creating a selection
Mark Mode is a ConHost feature that allows the user to create a selection using only the keyboard. In CMD, pressing <kbd>ctrl+m</kbd> enters mark mode. The current cursor position becomes a selection endpoint. The user can use the arrow keys to move that endpoint. While the user then holds <kbd>shift</kbd>, the selection endpoint ('start') is anchored to it's current position, and the arrow keys move the other selection endpoint ('end').

Additionally, pressing <kbd>shift+arrow</kbd> also initiates a selection, but it anchors the first selection endpoint to the cursor position.

Other terminal emulators have different approaches to this feature. iTerm2, for example, has Copy Mode (documentation [linked here](https://iterm2.com/documentation-copymode.html)). Here, <kbd>cmd+shift+c</kbd> makes the current cursor position become a selection endpoint. The arrow keys can be used to move that endpoint. However, unlike Mark Mode, a key binding <kbd>c+space</kbd> is used to change the start/stop selecting. The first time it's pressed, the 'start' endpoint is anchored. The second time it's pressed, the 'end' endpoint is set. After this, you can still move a cursor, but the selection persists until a new selection is created (either by pressing the key binding again, or using the mouse).

Though tmux is not a terminal emulator, it does also have Copy Mode that behaves fairly similarly to that of iTerm2's.


## Solution Design

The fundamental solution design for keyboard selection is that the responsibilities between the Terminal Control and Terminal Core must be very distinct. The Terminal Control is responsible for handling user interaction and directing the Terminal Core to update the selection. The Terminal Core will need to update the selection according to the preferences of the Terminal Control.
The fundamental solution design for keyboard selection is that the responsibilities between the Terminal Control and Terminal Core must be very distinct. The Terminal Control is responsible for handling user interaction and directing the Terminal Core to update the selection. The Terminal Core will need to update the selection according to the direction of the Terminal Control. Terminal Core maintains the state of the selection.

Relatively recently, TerminalControl was split into `TerminalControl`, `ControlInteractivity`, and `ControlCore`. Changes made to `ControlInteractivity`, `ControlCore`, and below propagate functionality to all consumers, meaning that the WPF terminal would benefit from these changes with no additional work required.

Expand All @@ -32,7 +39,7 @@ Relatively recently, TerminalControl was split into `TerminalControl`, `ControlI

The first branch will be updated to _modify_ the selection instead of usually _clearing_ it. This will happen by converting the key event into parameters to forward to `TerminalCore`, which then updates the selection appropriately.

#### Idea: Make keyboard selection a collection of standard keybindings
#### Abandoned Idea: Make keyboard selection a collection of standard keybindings
One idea is to introduce an `updateSelection` action that conditionally works if a selection is active (similar to the `copy` action). For these key bindings, if there is no selection, the key events are forwarded to the application.

Thanks to Keybinding Args, there would only be 1 new command:
Expand Down Expand Up @@ -74,7 +81,7 @@ This idea was abandoned due to several reasons:
4. 12 new items in the command palette is also pretty excessive.
5. If proven wrong when this is in WT Preview, we can revisit this and make them customizable then. It's better to add the ability to customize it later than take it away.

#### Idea: Make keyboard selection a simulation of mouse selection
#### Abandoned Idea: Make keyboard selection a simulation of mouse selection
It may seem that some effort can be saved by making the keyboard selection act as a simulation of mouse selection. There is a union of mouse and keyboard activity that can be represented in a single set of selection motion interfaces that are commanded by the TermControl's Mouse/Keyboard handler and adapted into appropriate motions in the Terminal Core.

However, the mouse handler operates by translating a pixel coordinate on the screen to a text buffer coordinate. This would have to be rewritten and the approach was deemed unworthy.
Expand Down Expand Up @@ -110,25 +117,117 @@ For `SelectionExpansion = Buffer`, the selection endpoint will be moved to the b

**NOTE**: In all cases, horizontal movements attempting to move past the left/right viewport boundaries result in a wrap. Vertical movements attempting to move past the top/bottom viewport boundaries will scroll such that the selection is at the edge of the screen. Vertical movements attempting to move past the top/bottom buffer boundaries will be clamped to be within buffer boundaries.

Every combination of the `SelectionDirection` and `SelectionExpansion` will map to a keybinding. These pairings are shown below in the UI/UX Design --> Keybindings section.

**NOTE**: If `copyOnSelect` is enabled, we need to make sure we **DO NOT** update the clipboard on every change in selection. The user must explicitly choose to copy the selected text from the buffer.

### Mark Mode

Mark Mode is a mode where the user can create and modify a selection using only the keyboard.

When no selection is present, the user may use the `markMode` action to enter mark mode. Upon doing so, a selection will be created at the current cursor position.

When in mark mode, the user may...
- press <kbd>ESC</kbd> to clear the selection and exit mark mode
- invoke the `markMode` action to exit mark mode
- invoke the `copy` action (this includes right-clicking the terminal) to copy the selected text, clear the selection, and exit mark mode
- move the cursor in the following ways:
- arrow keys --> move by character
- ctrl + left/right --> move by word
- ctrl + home/end --> move to the beginning/end of the buffer
- home/end --> move to the beginning/end of the line respectively
- pgup/pgdn --> move up/down by viewport respectively
- expand the selection in the following ways:
- shift + arrow keys --> move the "end" endpoint by character
- ctrl + shift + left/right --> move the "end" endpoint by word
- ctrl + shift + home/end --> move the "end" endpoint to the beginning/end of the buffer
- shift + home/end --> move the "end" endpoint to the beginning/end of the line respectively
- shift + pgup/pgdn --> move the "end" endpoint up/down by viewport respectively

As with mouse selections, keybindings are still respected and pressing a key that is not bound to a keybinding (or mentioned above) will clear the selection and exit mark mode.

#### Corner cases

- In mark mode, if a selection was created via the keyboard, moving the cursor moves at the "end" endpoint. This is consistent with conhost.
- If a user creates a selection using the mouse, then enters mark mode, mark mode inherits the existing selection as if it was made using the keyboard.
- If `copyOnSelect` is enabled, the selection is copied when the selection operation is "complete". Thus, the selection is copied when the `copy` keybinding is used or the selection is copied using the mouse.
- If `copyOnSelect` is enabled, `ESC` is interpreted as "cancelling" the selection, so nothing is copied. Keys that generate input are also interpreted as "cancelling" the selection. Only the `copy` keybinding or copying using the mouse is considered "completing" the selection operation, and copying the content to the clipboard.

**NOTE** - Related to #3884:
If the user has chosen to have selections persist after a copy operation, the selection created by Copy Mode is treated no differently than one created with the mouse. The selection will persist after a copy operation. However, if the user exits Copy Mode in any of the other situations, the selection is cleared.

#### Block Selection
A user can normally create a block selection by holding <kbd>alt</kbd> then creating a selection.

If the user is in Mark Mode, and desires to make a block selection, they can use the `toggleBlockSelection()` action. `toggleBlockSelection()` takes an existing selection, and transforms it into a block selection (or vice-versa).

All selections created in Mark Mode will have block selection disabled by default.

#### Rendering during Copy Mode
Since we are just moving the selection endpoints, rendering the selection rects should operate normally. We need to ensure that we still scroll when we move a selection point past the top/bottom of the viewport.

In ConHost, output would be paused when a selection was present. Windows Terminal does not pause the output when a selection is present, however, it does not scroll to the new output.

#### Interaction with Mouse Selection
If a selection exists, the user is basically already in Copy Mode. The user should be modifying the "end" endpoint of the selection when using the `updateSelection()` bindings. The existing selection should not be cleared (contrary to prior behavior). However, the half-y-beam will not be drawn. Once the user presses the `copyMode` or `moveSelectionPoint` keybinding, the half-y-beam is drawn on the targeted endpoint (which will then be "start").

During Copy Mode, if the user attempts to create a selection using the mouse, any existing selections are cleared and the mouse creates a selection normally. However, contrary to prior behavior, the user will still be in Copy Mode. The target endpoint being modified in Copy Mode, however, will be the "end" endpoint of the selection, instead of the cursor (as explained earlier in the flowchart).


#### Abandoned Idea: Copy Mode
Copy Mode is a more complex version of Mark Mode that is intended to provide a built-in way to switch the active selection endpoint. This idea was abandoned because we would then run into a user education issue. Rather than reinventing the wheel, selection should feel natural like that of a text editor, and any diversion from that model should be introduced separately (i.e. keybindings). Doing so ensures that users can "hit the ground running" when trying to make a selection, but won't be hindered by new functionality that is available.

Copy Mode is a mode where the user can create and modify a selection using only the keyboard. The following flowchart covers how the new `copymode()` action works:

![Copy Mode Flowchart][images/CopyModeFlowchart.png]
carlos-zamora marked this conversation as resolved.
Show resolved Hide resolved

**NOTE**: `copyMode()` refers to the action, whereas `updateSelection()` refers to the underlying function that is being called in the code.

If a selection is not active, a "start" and "end" selection point is created at the cursor position. `updateSelection()` calls then move "start" and "end" together as one position.

Invoking `copyMode()` again, will then anchor "start" (meaning that it will be kept in place). Subsequent `updateSelection()` calls move the "end" selection point.

Invoking `copyMode()` essentially cycles between which selection point is targeted.

## UI/UX Design

### Key Bindings

There will only be 1 new command that needs to be added:
| Action | Keybinding Args | Description |
|--|--|--|
| `selectAll` | | Select the entire text buffer.
| `selectAll` | none | Select the entire text buffer. |
| `markMode` | none | Toggle mark mode. If no selection exists, create a selection at the cursor position. Otherwise, use the existing selection as one in mark mode. |
| `toggleBlockSelection` | none | Transform the existing selection between a block selection and a line selection. |
| `switchSelectionEndpoint` | none | If a selection is present, switch which selection endpoint is targeted when in mark mode or quick edit mode. |

By default, the following key binding will be set:
```JS
{ "command": "selectAll", "keys": "ctrl+shift+a" },

// Copy Mode
{ "command": "copyMode", "keys": "ctrl+shift+m" },
{ "command": "toggleBlockSelection" },
{ "command": "switchSelectionEndpoint" },
```

### Y-Beam

A y-beam will be used to identify which selection endpoint is currently being moved when in mark mode.

When we're moving the cursor (this happens when mark mode is entered from no existing selection), a full y-beam will be displayed at the cursor position.

![Y-Beam Example][./images/Y-Beam.png]
carlos-zamora marked this conversation as resolved.
Show resolved Hide resolved

When <kbd>shift</kbd> is held, we're expanding the selection. In this case, the y-beam will be split, and the relevant half will be rendered on the active endpoint.

![Separated Y-Beam Example][./images/Half-Y-Beam.png]

**NOTE:** Both half y-beams could have been presented as shown in the image below. This idea was omitted because then there is no indication for which half y-beam is currently focused.

![Both Separated Y-Beams Example][./images/Split-Y-Beam.png]
carlos-zamora marked this conversation as resolved.
Show resolved Hide resolved

### Miscellaneous

When mark mode is enabled, the cursor will stop blinking.

## Capabilities

### Accessibility
Expand All @@ -151,20 +250,23 @@ N/A

### Performance, Power, and Efficiency

N/A

## Potential Issues

### Grapheme Clusters
When grapheme cluster support is inevitably added to the Text Buffer, moving by "cell" is expected to move by "character" or "cluster". This is similar to how wide glyphs are handled today. Either all of it is selected, or none of it.

### Circling the buffer
As usual, if the buffer is circling, the selection should be updated to follow the content (and "scroll up" appropriately).

In the event that one endpoint "scrolls" off the buffer, we must clamp "start" to the buffer origin. Conversely, in the event that both endpoints "scroll" off the buffer, the selection must be considered cleared.

## Future considerations

### Word Selection Wrap
At the time of writing this spec, expanding or moving by word is interrupted by the beginning or end of the line, regardless of the wrap flag being set. In the future, selection and the accessibility models will respect the wrap flag on the text buffer.

## Mark Mode

This functionality will be expanded to create a feature similar to Mark Mode. This will allow a user to create a selection using only the keyboard.


## Resources

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.