Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scope subcommand followed by other tokens #23

Merged
merged 44 commits into from
Jul 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
3e4ef4a
Scope lowercase terms when followed by other terms
victor-gp Oct 10, 2022
80382b0
Fix broken adb syntax test
victor-gp Oct 10, 2022
beae871
Add tests showing what's wrong with other-term-seq
victor-gp Oct 10, 2022
dd43878
Scope argument after subcommand
victor-gp Oct 10, 2022
db357da
Update syntax tests for other-def sequences
victor-gp Jan 23, 2023
83941df
Move other-def-seq include to other-def
victor-gp Jan 23, 2023
4094410
Rename other-def -> keyword, other-variable -> variable
victor-gp Jun 30, 2023
1dbc324
Rename "..other.def.." scope to "keyword"
victor-gp Jun 30, 2023
adc4446
Annotate a refactor idea for enum1|enum2|enum3 args
victor-gp Jun 30, 2023
09e84e7
Scope "subcommand --option"
victor-gp Jun 30, 2023
32cdd01
Scope "keyword [--option]"
victor-gp Jun 30, 2023
437fc88
Scope "keyword [ARGUMENT]"
victor-gp Jun 30, 2023
ceefab3
Scope "keyword ARGUMENT"
victor-gp Jun 30, 2023
2157efe
Require a long space after "keyword ARG" to fix regressions
victor-gp Jun 30, 2023
9887fbd
Add gdiffm util to get regression diff with main
victor-gp Jul 1, 2023
57748ff
Rename scripts/env.sh to ../utils
victor-gp Jul 1, 2023
a064b0f
Don't scope the 2nd term after a subcommand
victor-gp Jul 1, 2023
f59e14e
Handle "disconnect [HOST[:PORT]]" from adb
victor-gp Jul 1, 2023
fda6af9
Handle ""pair HOST[:PORT] ..." from adb
victor-gp Jul 1, 2023
e42fe00
Handle "shell [-e ESCAPE] ..." from adb
victor-gp Jul 1, 2023
11071bb
Handle "shell [-e ESCAPE] ..." from adb
victor-gp Jul 1, 2023
938f74b
Handle "reboot [bootloader|recovery|..." from adb
victor-gp Jul 1, 2023
6a70341
Fix a stray, outdated syntax assertion
victor-gp Jul 1, 2023
f8bf10d
Rename regression test utils
victor-gp Jul 17, 2023
310a626
Reorganize & document dev utils
victor-gp Jul 17, 2023
bd64ca0
Remove subcommand-def-post
victor-gp Jul 17, 2023
3de9f62
Handle "bat <SUBCOMMAND>" from bat
victor-gp Jul 17, 2023
885f8c8
Separate {{long_space}} and {{single_2_space}}
victor-gp Jul 17, 2023
833cbb9
Don't scope "or:" as keyword other
victor-gp Jul 17, 2023
751858f
Properly `set: then-pop` after each keyword-seq-def match
victor-gp Jul 17, 2023
aa81feb
Scope bat in "Usage: bat"
victor-gp Jul 18, 2023
a700040
Handle "mount [options] ..." from mount
victor-gp Jul 18, 2023
22ceeb2
Change a todo comment priority
victor-gp Jul 18, 2023
7e46345
Add lookahead_long_space variable
victor-gp Jul 18, 2023
01e0bbf
Rename lookahead_40 to _lte_40_chars
victor-gp Jul 18, 2023
5950372
Isolate keyword-def-seq last resort matches
victor-gp Jul 29, 2023
c862758
docs: Update scripts/{env.sh -> utils}
victor-gp Jul 30, 2023
682bc7d
docs: Write command line arguments <LIKE_THIS>
victor-gp Jul 30, 2023
76dd911
docs: Update the Regression Tests section
victor-gp Jul 30, 2023
7e18cd8
docs: Development: move a subsection
victor-gp Jul 30, 2023
f597e53
Fix regex for lookahead_single_2_space
victor-gp Jul 30, 2023
07b0b4e
Don't scope plain words with lookahead_single_2_space
victor-gp Jul 30, 2023
fc97338
Add a todo for adb's lowercase headings
victor-gp Jul 30, 2023
2800545
Merge branch 'main' into scope-subcommand-followed-by-args
victor-gp Jul 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ If `help()` doesn't work with a command's alternative help option, you can still

The syntax works nicely with most `bat` themes. In some cases, even better than with the default Monokai theme (used in the above example). You can find examples of working themes [here](https://github.com/victor-gp/cmd-help-sublime-syntax/tree/demo/docs/examples/theme).

To use a different theme, just append `--theme=<your preferred theme>` to your `bat` commands. More info [here](https://github.com/sharkdp/bat#highlighting-theme).
To use a different theme, just append `--theme='<THEME_NAME>'` to your `bat` commands. More info [here](https://github.com/sharkdp/bat#highlighting-theme).

You may also want to experiment with setting `--italic-text=always`. A few themes use italics on some elements, including Monokai.

Expand Down
46 changes: 30 additions & 16 deletions docs/Development.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ The project uses two classes of tests:
- regression tests: catch-all integration tests

There's a number of command-line utils I use to streamline development, including tests.
You can load them with `$ source scripts/env.sh`
You can load them with `$ source scripts/utils`

### Syntax tests

Expand All @@ -74,13 +74,13 @@ You can run them with `$ tests/syntax.py`
If you expect the tests to pass, it's better to run them with the *summary* option: `$ tests/syntax.py -s`

Syntax tests are also useful to *debug* the workings of the syntax line by line.
If you can't make heads or tails of why something doesn't work, run `$ tests/syntax.py -d tests/syntax/<test file>`
If you can't make heads or tails of why something doesn't work, run `$ tests/syntax.py -d tests/syntax/<TEST_FILE>`

`env.sh` also contains an util to run with debug and pipe the output to `less`: `$ debug tests/syntax/<test file>`
`scripts/utils` also contains an util to run with debug and pipe the output to `less`: `$ debug tests/syntax/<TEST_FILE>`

The `syntax -d` output is a bit hard to grasp unless you understand Sublime Syntaxes very well (not my case). So have patience and ask for help if you get stuck.

Lastly, there's an util to create a syntax test from a `tests/source/` help message: `$ mksyn tests/source/<source file>`
Lastly, there's an util to create a syntax test from a `tests/source/` help message: `$ mksyn tests/source/<SOURCE_FILE>`

As a rule of thumb, if there's a syntax test for a command, it should also be in `tests/source`.

Expand All @@ -89,24 +89,29 @@ For that, I like to pair both within the same commit, so the former serves as an

### Highlight regression tests

These take all the samples (help messages from actual commands) in `tests/source` and run them through `bat` + `cmd-help`.
Then they store the result (syntax highlighted text) in `tests/highlighted/`.
There are a bunch of samples (help messages from actual commands) in `tests/source`.

You can run them with `$ tests/highlight_regression`
Regression tests take all these samples and run them through `bat` + `cmd-help`, storing the result (syntax highlighted text) in `tests/highlighted/`.

Run them with `$ tests/highlight_regression`

This script runs the entire suite of highlight regression tests. They usually take < 5s, but it depends on how fast Docker + `bat` update the syntax theme.

**They're good for quickly validating if your change breaks existing functionality.**

Because the result files contain color escape characters, they should be opened with `less -R`
#### Displaying regression test differences

To query their changes with respect to the git index, do `$ gdiff` (from `env.sh`)
Because the result files in `tests/highlighted` contain color escape characters (e.g.: `ESC[0m`), they should be opened with `less -R`.

For the diff between the index and the last commit (so staged changes), do `$ gdiffs`
To check highlight regression changes with respect to the git index, load `scripts/utils` and do `$ reg`

These are also a great place to look at when looking for unhandled patterns, to document pending tasks.
There are a few more utils based on `reg`:

To query them do: `less -R tests/highlighted/*`
- For the regression diff between the index and the last commit (so staged changes), do `$ regs`
- For the regression diff between your branch and `main`, do `$ regm`
- To show the regression diff from a commit (HEAD by default), do `$ regshow`

or `less -R tests/highlighted/<test file>`
#### Syntax tests > regression tests to illustrate changes

After a big syntax change that causes many highlight test files to update, it's ok not to include them in the same commit as the syntax change.

Expand All @@ -125,7 +130,7 @@ These track the syntax's coverage for the themes included with `bat`. The motiva
You can run them with `$ tests/theme_regression`

It runs a synthetic help message through `bat` + `cmd-help`, twice for each theme: with and without italics enabled.
Then it store the result in `tests/theme_regression/`, but deleting the italics version if it makes no difference.
Then it stores the result in `tests/theme_regression/`, but deleting the italics version if it makes no difference.

Everything I mentioned on `highlight_regression` applies here, just replacing `highlight` for `theme`.

Expand All @@ -148,16 +153,25 @@ I should probably make these into GitHub Issues, but I haven't got the chance to
There's also a [Roadmap doc](./Roadmap.md) that defines some long-term goals for the project.
Look there if you want to break entirely new ground.

### Spotting issues in regression tests

The corpus of highlighted help messages in `tests/highlighted` is also a great place to look for pending work:

- do `$ less -R tests/highlighted/*`
- iterate through the files with `:n` (next) and `:p` (previous)
- scroll through the highlighted text and look for tokens that should (not) be colorized

## Sample development workflow

1. Search pending tasks with the to-do marker keywords. Choose one.
1. Update the assertion/s related to that task, negate it to make it fail.
1. Make whatever changes in the syntax to try and fix that test case.
1. Run `tests/syntax.py` to check the assertion now passes without breaking anything else.
1. Repeat steps 3-4 until you get it right.
1. Run `tests/highlight_regression.sh` to ensure the changes don't have unintended consequences in the larger body of help messages.
1. Run `tests/theme_regression.sh` to make double sure.
1. Repeat steps 3-5 until you get it right.
1. Commit, early and often!
1. Run `tests/theme_regression.sh` to check that you didn't break support for any theme.
1. [Submit a Pull Request](https://github.com/victor-gp/cmd-help-sublime-syntax/pulls)

## Contact

Expand Down
7 changes: 3 additions & 4 deletions scripts/make_syntax_test.sh
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
#!/bin/bash

# usage: scripts/make_syntax_test.sh tests/source/<file>
# or: mksyn tests/source/<file> (if you $ source scripts/env.sh)

# usage: scripts/make_syntax_test.sh tests/source/<FILE>
# or: mksyn tests/source/<FILE> (if you $ source scripts/utils)

set -euo pipefail

# change dir to project root
cd "$(dirname "${BASH_SOURCE[0]}")/.."

if [ $# -ne 1 ] || [ ! -f "$1" ]; then
>&2 echo "usage: $0 tests/source/<file>"
>&2 echo "usage: $0 tests/source/<FILE>"
exit 2
fi

Expand Down
30 changes: 20 additions & 10 deletions scripts/env.sh → scripts/utils
Original file line number Diff line number Diff line change
@@ -1,29 +1,39 @@
#!/bin/bash
# usage: $ source scripts/utils
# shellcheck shell=bash

function gdiff {
# regression tests diff
function reg {
GIT_PAGER="LESS=R less" git diff "$@" -- tests/{highlighted,theme}
}
function gdiffs {
gdiff --staged "$@"
}

# to exclude e.g. THEME-italics, do:
# gdiff -- ":^*-italics*"
# reg -- ":^*-italics*"

function regs {
reg --staged "$@"
}

function regm {
reg main "$@"
}

function gshow {
# regression tests show
function regshow {
GIT_PAGER="LESS=R less" git show "$@" -- tests/{highlighted,theme}
}

alias git0='GIT_CONFIG=/dev/null git'

debug() {
# run a syntax test in debug mode
function debug {
tests/syntax.py "$@" -d | less -R
}

demo() {
function demo {
"$@" --help | bat --no-config -pl cmd-help --pager='less -R'
}

demo_it() {
function demo_it {
"$@" --help | bat --no-config -pl cmd-help --pager='less -R' --italic-text=always
}

Expand Down
Loading