Add usage examples GIFs

.pre-commit-config.yaml

@@ -6,6 +6,7 @@ repos:
 
     hooks:
     -   id: check-added-large-files
+        args: ['--maxkb=1024']
     -   id: check-case-conflict
     -   id: check-json
     -   id: check-symlinks

README.md

@@ -1,5 +1,7 @@
 # Nosey Parker: Find secrets in textual data
 
+## Overview
+
 Nosey Parker is a command-line tool that finds secrets and sensitive information in textual data. It is useful both for offensive and defensive security testing.
 
 **Key features:**
@@ -30,7 +32,9 @@ Prebuilt binaries are available for x86_64 Linux and x86_64/aarch64 macOS on the
 This is a simple way to get started and will give good performance.
 
 
-### Docker image
+### Docker images
+
+<details>
 
 A prebuilt multiplatform Docker image is available for the latest release for x86_64 and aarch64:
 ```shell
@@ -49,6 +53,8 @@ $ docker pull ghcr.io/praetorian-inc/noseyparker-alpine:edge
 
 **Note:** The Docker images run noticeably slower than a native binary, particularly on macOS.
 
+</details>
+
 
 ### Arch Linux package
 
@@ -96,13 +102,16 @@ The scanning and reporting steps are implemented as separate commands because yo
 
 
 ### Getting help
+
 Running the `noseyparker` binary without arguments prints top-level help and exits.
 You can get abbreviated help for a particular command by running `noseyparker COMMAND -h`.
 More detailed help is available with the `help` command or long-form `--help` option.
 
 The prebuilt releases also include manpages that collect the command-line help in one place.
 These manpages converted into Markdown format are also included in the repository [here](docs/v0.17.0/man/man1).
 
+If you have a question that's not answered by this documentation, feel free to [start a discussion](https://github.com/praetorian-inc/noseyparker/discussions/new/choose).
+
 
 ### Terminology and data model
 
@@ -143,7 +152,7 @@ This is Nosey Parker's top-level unit of reporting.
 
 ## Usage examples
 
-### Docker note
+### NOTE: When using Docker...
 
 If you are using the Docker image, replace `noseyparker` in the following commands with a Docker invocation that uses a mounted volume:
 
@@ -154,7 +163,10 @@ docker run -v "$PWD":/scan ghcr.io/praetorian-inc/noseyparker:latest <ARGS>
 The Docker container runs with `/scan` as its working directory, so mounting `$PWD` at `/scan` in the container will make tab completion and relative paths in your command-line invocation work.
 
 
-### Scan filesystem content for secrets
+### Scan inputs for secrets
+
+#### Filesystem content, including local Git repos
+![Screenshot showing Nosey Parker's workflow for scanning the filesystem for secrets](docs/usage-examples/gifs/02-scan-git-history.gif)
 
 <details>
 
@@ -181,7 +193,8 @@ Run the `report` command next to show finding details.
 ```
 </details>
 
-### Scan Git history by URL, GitHub username, or GitHub organization name
+
+#### Git repos given URL, GitHub username, or GitHub organization name
 
 <details>
 
@@ -204,29 +217,8 @@ Providing an access token gives a higher API rate limit and may make additional
 See `noseyparker help scan` for more details.
 </details>
 
-### Summarize findings
-
-<details>
-
-Nosey Parker prints out a summary of its findings when it finishes scanning.
-You can also run this step separately:
-```
-$ noseyparker summarize --datastore np.cpython
-
- Rule                      Distinct Groups   Total Matches
-───────────────────────────────────────────────────────────
- PEM-Encoded Private Key             1,076           1,192
- Generic Secret                        331             478
- netrc Credentials                      42           3,201
- Generic API Key                         2              31
- md5crypt Hash                           1               2
-```
-
-Additional output formats are supported, including JSON and JSON lines, via the `--format=FORMAT` option.
-</details>
-
 
-### Report finding details
+### Report findings
 
 <details>
 
@@ -277,9 +269,38 @@ Showing 3/29 occurrences:
 ```
 
 (Note: the findings above are synthetic, invalid secrets.)
+Additional output formats are supported, including JSON, JSON lines, and SARIF, via the `--format=FORMAT` option.
+</details>
+
+#### Human-readable text format
+![Screenshot showing Nosey Parker's workflow for rendering its findings in human-readable format](docs/usage-examples/gifs/03-report-human.gif)
+
+#### JSON format
+![Screenshot showing Nosey Parker's workflow for rendering its findings in JSON format](docs/usage-examples/gifs/04-report-json.gif)
+
+
+### Summarize findings
+
+<details>
+
+Nosey Parker prints out a summary of its findings when it finishes scanning.
+You can also run this step separately:
+```
+$ noseyparker summarize --datastore np.cpython
+
+ Rule                      Distinct Groups   Total Matches
+───────────────────────────────────────────────────────────
+ PEM-Encoded Private Key             1,076           1,192
+ Generic Secret                        331             478
+ netrc Credentials                      42           3,201
+ Generic API Key                         2              31
+ md5crypt Hash                           1               2
+```
+
 Additional output formats are supported, including JSON and JSON lines, via the `--format=FORMAT` option.
 </details>
 
+
 ### Enumerate repositories from GitHub
 
 <details>

docs/usage-examples/.gitignore

@@ -0,0 +1 @@
+scratch/

docs/usage-examples/README.md

@@ -0,0 +1,26 @@
+# Usage Examples
+
+## Overview
+This directory contains scripts to produce GIFs of Nosey Parker usage examples.
+
+These scripts are written using [`vhs`](https://github.com/charmbracelet/vhs), a console script runner and recording tool.
+The example script fragments are in the [`examples`](examples) directory.
+Each one is a `vhs` fragment, which is cobbled together with some common settings using the [`record-examples.zsh`](record-examples.zsh) script.
+
+The generated GIFs are stored in the [`gifs`](gifs) directory.
+
+## Generating GIFs
+- Install `vhs`
+- Install `noseyparker`
+- `./record-examples.zsh`
+
+## Errata: `vhs`
+This tool, though it seems to be very popular, is full of significant bugs.
+What I've seen in a day's usage:
+
+- The `Show` and `Hide` commands seem to have a race condition, not reliably working without adding `Sleep 1s` calls before and after.
+  Otherwise, you are likely to get the hidden commands included in the rendered output.
+
+- The `Source` command is unsuitable for its main use case (putting common settings in a separate file); the settings seem to be applied, then partially forgotten.
+
+- The framerate for GIF output is unreliable and doesn't correspond with `Sleep` durations, especially at higher framerates

docs/usage-examples/common-config.tape

@@ -0,0 +1,23 @@
+Require echo
+Require less
+Require noseyparker
+
+# Set WindowBar Colorful
+# Set WindowBarSize 0
+
+Set Margin 0
+
+Set Padding 10
+Set FontSize 20
+Set LineHeight 1.1
+Set Width 1200
+Set Height 750
+
+Set Shell "bash"
+Set TypingSpeed 25ms
+
+# Use a lower framerate to get more predictable GIF animation speed.
+# See https://github.com/charmbracelet/vhs/issues/88.
+Set Framerate 20
+
+Output "output.gif"

docs/usage-examples/examples/01-getting-help.tape

@@ -0,0 +1,16 @@
+Hide Type "clear" Enter Show Sleep 1s
+Type "# Run without arguments or with `-h` for help" Sleep 500ms Enter
+Type "noseyparker" Sleep 1s Enter
+Sleep 5s
+
+
+Hide Type "clear" Enter Show Sleep 1s
+Type "# Get more detailed help with `--help`" Sleep 500ms Enter
+Type "noseyparker --help | less" Sleep 1s Enter
+Sleep 5s Type "q"
+
+
+Hide Type "clear" Enter Show Sleep 1s
+Type "# Get detailed subcommand help with `CMD --help`" Sleep 500ms Enter
+Type "noseyparker scan --help | less" Sleep 1s Enter
+Sleep 5s Type "q"

docs/usage-examples/examples/02-scan-git-history.tape

@@ -0,0 +1,13 @@
+Hide
+Sleep 1s
+Type "git clone -q 'https://github.com/Plazmaz/leaky-repo' && clear" Enter
+Sleep 1s
+Show
+
+Sleep 1s
+
+Type "# Scan the complete history of a local Git repository with the `scan` command." Sleep 0.5s Enter
+Type "# This example assumes you have a local clone of https://github.com/Plazmaz/leaky-repo." Sleep 1s Enter
+
+Type "noseyparker scan leaky-repo" Sleep 0.5s Enter
+Sleep 5s

docs/usage-examples/examples/03-report-human.tape

@@ -0,0 +1,13 @@
+Hide
+Sleep 1s
+Type "noseyparker -q scan --git-url 'https://github.com/Plazmaz/leaky-repo' && clear" Enter
+Sleep 1s
+Show
+
+Sleep 1s
+
+Type "# View findings in human-readable format with the `report` command." Sleep 0.5s Enter
+Type "# This example assumes you already have a datastore from scanning at `datastore.np`." Sleep 1s Enter
+
+Type "noseyparker report --color always | less -RS" Sleep 0.5s Enter
+Sleep 5s Type "q"

docs/usage-examples/examples/04-report-json.tape

@@ -0,0 +1,18 @@
+Hide
+Sleep 1s
+Type "noseyparker -q scan --git-url 'https://github.com/Plazmaz/leaky-repo' && clear" Enter
+Sleep 1s
+Show
+
+Sleep 1s
+
+Type "# View findings in JSON format with the `report --format json` command." Sleep 0.5s Enter
+Type "# You can redirect the output to a file using the `-o FILE` option." Sleep 0.5s Enter
+Type "# This example assumes you already have a datastore from scanning at `datastore.np`." Sleep 1s Enter
+
+Type "noseyparker report --format json -o findings.json" Sleep 0.5s Enter
+Sleep 1s
+
+Type "# You can pretty-print JSON using `jq`." Enter
+Type "jq -C . findings.json | less -RS" Enter
+Sleep 5 Type "q"

docs/usage-examples/record-examples.zsh

@@ -0,0 +1,24 @@
+#!/bin/zsh
+
+set -e
+
+rm -rf scratch gifs && mkdir scratch gifs
+
+for example in examples/*.tape; do
+    echo "### $example"
+    scratch_dir="scratch/$(basename "$example" .tape)"
+    mkdir "$scratch_dir"
+
+    preprocessed="$scratch_dir/input.tape"
+
+    cat common-config.tape "$example" >"$preprocessed"
+
+    (cd "$scratch_dir" && vhs input.tape && cp output.gif "../../gifs/$(basename "$example" .tape).gif")
+done
+
+# other examples to create:
+# - listing rules
+# - checking rules
+# - reporting in SARIF format
+# - reporting in JSONL format
+# - dumping the report JSON schema