diff --git a/docs/FAQ.md b/docs/FAQ.md index 0a269ac..71ce207 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -7,12 +7,15 @@ * [Can I run termshark on Android?](#can-i-run-termshark-on-android) * [If I load a big pcap, termshark doesn't load all the packets at once - why?](#if-i-load-a-big-pcap-termshark-doesnt-load-all-the-packets-at-once---why) * [Termshark is too bright!](#termshark-is-too-bright) -* [Termshark's colors are limited...](#termsharks-colors-are-limited) +* [Termshark's colors are wrong!](#termsharks-colors-are-wrong) +* [What settings affect termshark's colors?](#what-settings-affect-termsharks-colors) +* [How do I rebuild termshark?](#how-do-i-rebuild-termshark) * [Where are the config and log files?](#where-are-the-config-and-log-files) * [The console is too narrow on Windows](#the-console-is-too-narrow-on-windows) * [Can I pass extra arguments to tshark?](#can-i-pass-extra-arguments-to-tshark) * [How does termshark use tshark?](#how-does-termshark-use-tshark) * [How can I make termshark run without root?](#how-can-i-make-termshark-run-without-root) +* [Why do is termshark generating traffic on port 5037?](#why-is-termshark-generating-traffic-on-port-5037) * [How can termshark capture from extcap interfaces with dumpcap?](#how-can-termshark-capture-from-extcap-interfaces-with-dumpcap) * [Termshark is laggy or using a lot of RAM](#termshark-is-laggy-or-using-a-lot-of-ram) * [How much memory does termshark use?](#how-much-memory-does-termshark-use) @@ -89,31 +92,78 @@ If you start to page up quickly, you will likely approach a range of packets tha Termshark v2 supports dark-mode! Hit Esc to bring up the main menu then "Toggle Dark Mode". See the [User Guide](UserGuide.md#dark-mode). -## Termshark's colors are limited... +## Termshark's colors are wrong! -By default, termshark respects the `TERM` environment variable and chooses a color scheme based on what it thinks the terminal is capable of, via the excellent [tcell](https://github.com/gdamore/tcell) package. You might be running on a terminal that can display more colors than `TERM` reports - so you can try adjusting your `TERM` variable e.g. if `TERM` is `xterm`, try +See [this answer](#what-settings-affect-termsharks-colors). -```bash -export TERM=xterm-256color +If termshark's background is a strange color like dark blue or orange, maybe a tool like base16-shell has remapped some of the colors in the +256-color-space, but termshark is unaware of this. Try setting this in `termshark.toml`: + +```toml +[main] + ignore-base16-colors = true ``` -or even +## What settings affect termshark's colors? -```bash -export TERM=xterm-truecolor -``` +Unfortunately there are several :-/ -then re-run termshark. +First of all, your terminal emulator's `TERM` variable determines the range of colors available to termshark e.g. `xterm-16color`, `xterm-256color`. -tcell makes use of the environment variable `COLORTERM` when determining how to emit color codes. If `COLORTERM` is set to `truecolor`, then tcell will emit truecolor color codes when the application changes the foreground or background color. If you connect to a remote machine with ssh to run termshark, the `COLORTERM` variable will not be forwarded. If that leaves you with `TERM=xterm` for example, then termshark, via tcell, will fall back to 8-color support. Here again you can change `TERM` or add a setting for `COLORTERM` to your remote `.bashrc` file. +If you also have `COLORTERM=truecolor` set, and the terminal emulator has support, 24-bit color will be available. Termshark will emit these 24-bit +ANSI color codes and color should be faithfully reproduced. -If you run termshark under tmux or screen and always have `TERM` set in a way that doesn't make full use of your terminal emulator, you can configure termshark to always override it. Add the following to your `termshark.toml` file: +You can override the value of `TERM` with termshark's `main.term` setting in `termshark.toml` e.g. ```toml [main] term = "screen-256color" ``` +When termshark runs, it will load your selected theme if it's available in the terminal's color mode. If not, it will choose the built-in `default` +theme which is available in every mode. If you run in truecolor mode, and your chosen theme is only defined for 256-colors, termshark will load the +256-color theme. + +Termshark will load its theme from `~/.config/termshark/themes/` if it can find it, otherwise it will look in its built-in database. Termshark has +themes called `default`, `dracula`, `solarized` and `base16` built-in. + +If you are using [base16-shell](https://github.com/chriskempson/base16-shell), then you might have colors 0-21 of your terminal's 256-color-space +remapped. If you are running in 256-color mode, and your theme specifies RGB colors, termshark will choose the closest match among those in the +256-color-space. Termshark will ignore colors 0-21 as match candidates if `BASE16_SHELL` is set in the environment. It will also ignore these colors +if you set `main.ignore-base16-colors` in `termshark.toml`. Otherwise, termshark will assume colors 0-21 are displayed "normally", and may pick these +remapped colors as the closest match to a theme's color - resulting in incorrect colors. + +## How do I rebuild termshark? + +If you don't have the source, clone it like this: + +```bash +$ git clone https://github.com/gcla/termshark +``` + +You'll get best results with the latest version of Golang - 1.15 as I write this - but anything >= 1.12 will work. + +Set your environment: + +```bash +$ export GO111MODULE=on +``` + +Change to the termshark dir and type + +```bash +$ go generate ./... +$ go install ./... +``` + +The generate step is only necessary if you have changed any files under `termshark/assets/themes/`. If not, just run + +```bash +$ go install ./... +``` + +Termshark will be installed as `~/go/bin/termshark`. + ## Where are the config and log files? You can find the config file, `termshark.toml`, in: @@ -261,6 +311,12 @@ sudo setcap cap_net_raw,cap_net_admin+eip /usr/sbin/dumpcap You can find more detail at https://wiki.wireshark.org/CaptureSetup/CapturePrivileges. +## Why do is termshark generating traffic on port 5037? + +See [this issue](https://github.com/gcla/termshark/issues/98). + +TL;DR - try deleting `/usr/lib/wireshark/extcap/androiddump`. + ## How can termshark capture from extcap interfaces with dumpcap? Termshark doesn't always capture using dumpcap. It will try to use dumpcap if diff --git a/docs/UserGuide.md b/docs/UserGuide.md index 435407d..838fcac 100644 --- a/docs/UserGuide.md +++ b/docs/UserGuide.md @@ -17,13 +17,17 @@ Termshark provides a terminal-based user interface for analyzing packet captures - [Packet List View](#packet-list-view) - [Packet Structure View](#packet-structure-view) - [Packet Hex View](#packet-hex-view) + - [Marking Packets](#marking-packets) - [Copy Mode](#copy-mode) - [Packet Capture Information](#packet-capture-information) - [Stream Reassembly](#stream-reassembly) - [Conversations](#conversations) + - [Command-Line](#command-line) + - [Macros](#macros) - [Configuration](#configuration) - [Dark Mode](#dark-mode) - [Packet Colors](#packet-colors) + - [Themes](#themes) - [Config File](#config-file) - [Troubleshooting](#troubleshooting) @@ -153,7 +157,7 @@ $ sudo tcpdump -i eth0 -w - icmp | termshark [sudo] password for gcla: ``` -If the termshark UI is active in the terminal but you want to see something displayed there before termshark started, you can now issue a SIGTSTP signal (on Unix) and termshark will suspend itself and give up control of the terminal. In bash, this operation is usually bound to ctrl-z. +If the termshark UI is active in the terminal but you want to see something displayed there before termshark started, you can now issue a SIGTSTP signal (on Unix) and termshark will suspend itself and give up control of the terminal. In bash, this operation is usually bound to `ctrl-z`. ```console $ termshark -r foo.pcap @@ -190,7 +194,7 @@ When the filter widget is green, you can hit the "Apply" button to make its valu ### Changing Views -Press `tab` to move between the three packet views. You can also use the mouse to move views by clicking with the left mouse button. When focus is in any of these three views, hit the `\` key to maximize that view: +Press `tab` or `ctrl-w ctrl-w` to move between the three packet views. You can also use the mouse to move views by clicking with the left mouse button. When focus is in any of these three views, hit the `\` key to maximize that view: ![max](/../gh-pages/images/max.png?raw=true) @@ -198,15 +202,20 @@ Press `\` to restore the original layout. Press `|` to move the hex view to the ![altview](/../gh-pages/images/altview.png?raw=true) -You can also press `<`,`>`,`+` and `-` to change the relative size of each view. +You can also press `<`,`>`,`+` and `-` to change the relative size of each view. To reset termshark to use its original relative sizes, hit `ctrl-w +=`. All termshark views support vim-style navigation with `h`, `j`, `k` and `l` along with regular cursor keys. ### Packet List View -Termshark's top-most view is a list of packets read from the capture (or interface). Termshark generates the data by running `tshark` on the input with the `-T psml` options, and parsing the resulting XML. Currently the columns displayed cannot be configured, and are the same as Wireshark's defaults. When the source is a pcap file, the list can be sorted by column by clicking the button next to each column header: +Termshark's top-most view is a list of packets read from the capture (or interface). Termshark generates the data by running `tshark` on the input +with the `-T psml` options, and parsing the resulting XML. Currently the columns displayed cannot be configured, and are the same as Wireshark's +defaults. When the source is a pcap file, the list can be sorted by column by clicking the button next to each column header: ![sortcol](/../gh-pages/images/sortcol.png?raw=true) -You can hit `home` to jump to the top of the list or `end` to jump to the bottom. Sometimes, especially if running on a small terminal, the values in a column will be truncated (e.g. long IPv6 addresses). To see the full value, move the purple cursor over the value: +You can hit `home` or `gg` to jump to the top of the list and `end` or `G` to jump to the bottom. You can jump to a specific packet by entering its +number - as a prefix - before hitting `gg` or `G` Sometimes, especially if running on a small terminal, the values in a column will be truncated +(e.g. long IPv6 addresses). To see the full value, move the purple cursor over the value: ![ipv6](/../gh-pages/images/ipv6.png?raw=true) @@ -222,6 +231,16 @@ As you navigate the packet structure, different sections of the bottom view - a Termshark's bottom view shows the bytes that the packet comprises. Like Wireshark, they are displayed in a hexdump-like format. As you move around the bytes, the middle (structure) view will update to show you where you are in the packet's structure. +### Marking Packets + +To make it easier to compare packets, you can mark a packet in the packet list view and then jump back to it later. Termshark's marks are modeled on vim's. Set a mark by navigating to the packet and then hit `m` followed by a letter - `a` through `z`. + +![marks1](/../gh-pages/images/marks1.png?raw=true) + +To jump back to that mark, hit `'` followed by the letter you selected. To jump back to the packet that was selected prior to your jump, hit `''`. When you exit termshark or load a new pcap, these marks are deleted; but termshark also supports cross-pcap marks which are saved in termshark's config file. To make a cross-pcap mark, hit `m` followed by a capital letter - `A` through `Z`. If you jump to a cross-pcap mark made in another pcap, termshark will load that pcap back up. To display your current marks, use the [command-line](#command-line) `marks` command: + +![marks2](/../gh-pages/images/marks2.png?raw=true) + ### Copy Mode Both the structure and hex view support "copy mode" a feature which lets you copy ranges of data from the currently selected packet. First, move focus to the part of the packet you wish to copy. Now hit the `c` key - a section of the packet will be highlighted in yellow: @@ -298,6 +317,67 @@ In the first pop-up menu, you can choose how to extend the current display filte ![convs3](/../gh-pages/images/convs3.png?raw=true) +### Command-Line + +For fast navigation around the UI, termshark offers a vim-style command-line. To activate the command-line, hit the ':' key: + +![cmdline1](/../gh-pages/images/cmdline1.png?raw=true) + +Many of termshark's operations can be initiated from the command-line. After opening the command-line, hit tab to show all the commands available: + +- **capinfo** - Show the current capture file properties (using the `capinfos` command) +- **clear** - Clear the current pcap +- **convs** - Open the conversations view +- **filter** - Choose a display filter from those recently-used +- **help** - Show one of several help dialogs +- **load** - Load a pcap from the filesystem +- **logs** - Show termshark's log file (Unix-only) +- **map** - Map a keypress to a key sequence (see `help map`) +- **marks** - Show file-local and global packet marks +- **quit** - Quit termshark +- **recents** - Load a pcap from those recently-used +- **set** - Set various config properties (see `help set`) +- **streams** - Open the stream reassemably view +- **theme** - Set a new termshark theme +- **unmap** - Remove a keypress mapping made with the `map` command + +Some commands require a parameter or more. Candidate completions will be shown when possible; you can then scroll up or down through them and hit tab +or enter to complete the candidate. Candidates are filtered as you type. Hit enter to run a valid command or hit `ctrl-c` to close the command-line. + +### Macros + +To support navigational shortcuts that are not directly built-in to the termshark UI, you can now create simple keyboard macros. These are modeled on +vim's key mappings. To create a macro, open the [command-line](#command-line) and use the `map` command. The first argument is the key to map and the +second argument is a sequence of keypresses that your first key should now map to. Termshark uses vim-syntax for keys. To express a keypress for a +printable character, simply use the printable character. Here is the syntax for the other keys that termshark understands: + +- `` +- `` +- `` +- `-` +- modifiers - ``, `` +- ``, ``, ``, `` +- ``, `` +- ``, `` + +Here are some example macros: + +- `map /` - hit ctrl-s to activate the display filter widget +- `map :quit` - hit f1 to quit termshark without asking for confirmation +- `map ZZ` - another way to quit quickly! +- `map d` - toggle dark-mode + +A termshark user requested the ability to move up and down the packet list but to keep focus on the packet structure view. This can be accomplished by setting these macros: + +- `map ` +- `map ` + +Then with focus on the packet structure view, hit `f5` to go down a packet and `f6` to go up a packet. + +Macros are saved in the termshark config file. To display the current list of macros, simply type `map` from the command-line with no arguments. + +![macros](/../gh-pages/images/macros.png?raw=true) + ## Configuration ### Dark Mode @@ -312,10 +392,117 @@ Your choice is stored in the termshark [config file](UserGuide.md#config-file). By default, termshark will now display packets in the packet list view colored according to Wireshark's color rules. With recent installations of Wireshark, you can find this file at `$XDG_CONFIG_HOME/wireshark/colorfilters`. Termshark doesn't provide a way to edit the colors - the colors are provided by `tshark`. You can read about Wireshark's support [here](https://www.wireshark.org/docs/wsug_html_chunked/ChCustColorizationSection.html). If you don't like the way this looks in termshark, you can turn it off using termshark's main menu. +### Themes + +Termshark can be themed to better line up with other terminal applications that you use. Most of termshark's UI elements have names and you can tie colors to these names. Here is an example theme: + +```toml +[dracula] + gray1 = "#464752" + ... + orange = "#ffb86c" + purple = "#bd93f9" + red = "#ff5555" + white = "#f8f8f2" + yellow = "#f1fa8c" + +[dark] + button = ["dracula.black","dracula.gray3"] + button-focus = ["dracula.white","dracula.magenta"] + button-selected = ["dracula.white","dracula.gray3"] + ... + +[light] + button = ["dracula.black","dracula.white"] + button-focus = ["dracula.black","dracula.purple"] + button-selected = ["dracula.black","dracula.gray3"] + ... +``` + +Termshark finds themes in two ways - from: + +- `$XDG_CONFIG_HOME/termshark/themes/*.toml` (e.g. `~/.config/termshark/themes/dracula.toml`) +- from a small database compiled-in to the termshark binary. + +The termshark command-line provides two commands to interact with themes: + +- `theme` - choose a new theme from those provided and apply it. +- `no-theme` - use no theme. + +Termshark saves your selected theme against the terminal color mode, which can be one of + +- 16-color +- 256-color +- truecolor i.e. 24-bit color + +The theme is saved in `termshark.toml` under, respectively, the keys: + +- `main.theme-16` +- `main.theme-256` +- `main.theme-truecolor` + +This means that if you run termshark on the same machine but with a different terminal emulator, you might need to re-apply the theme if the color +mode has changed (e.g. `xterm` v `gnome terminal`) + +If you are running in truecolor/24-bit color, termshark will make the 256-color themes available too. Terminal emulators that support 24-bit color +will support 256-color mode as well. + +If you have enabled termshark's packet colors - shown in the packet list view - then these colors will be reproduced faithfully according to +Wireshark's rules. These colors don't adhere to termshark's themes. + +#### Built-in Themes and Base16 + +Termshark has four themes built-in: + +- `default` - termshark's original color scheme (16-color, 256-color, truecolor) +- `dracula` - colors based on [Dracula theme](https://draculatheme.com/) project (256-color, truecolor) +- `solarized` - based on [Ethan Schoonover's](https://ethanschoonover.com/solarized/) work (256-color, truecolor) +- `base16` - (256-color, truecolor) + +If you make another, please submit it! :-) + +[Base16](https://github.com/chriskempson/base16) is a set of guidelines for building themes using a limited range of 8 colors and 8 grays. The +[base16-shell](https://github.com/chriskempson/base16-shell) project is a set of scripts that remap colors 0-21 in the 256-color space of a terminal +emulator. If you're in 256-color mode, this lets you have consistent coloring of your work in a terminal emulator, whether it's typing at the shell, +or running a TUI. If you use base16-shell, choose termshark's `base16` theme to make use of your shell theme's colors. + +#### Make a Theme + +The layout of a theme is: + +- the color definitions - `[mytheme]` +- set the foreground and background color of UI elements for + - dark mode - `[dark]` + - regular/light mode - `[light]` + +Here's an [example theme](https://raw.githubusercontent.com/gcla/termshark/master/assets/themes/dracula-256.toml) to follow. This [tcell source +file](https://github.com/gdamore/tcell/blob/fcaa20f283682d6bbe19ceae067b37df3dc699d7/color.go#L821) shows some sample color names you can use. + +The UI elements are listed in the `[dark]` and `[light]` sections. Each element is assigned a pair of colors - foreground and background. The colors +can be: + +- a reference to another field in the theme toml e.g. `dracula.black` +- a color that termshark understands natively e.g. `#ffcc43`, `dark green`, `g50` (medium gray). + +Save your theme toml file under `~/.config/termshark/themes/` with a suffix indicating the color-mode e.g. `mytheme-256.toml`. + +If your theme is a truecolor theme (suffix `-truecolor.toml`), then RGB colors will be reproduced precisely by termshark and so by the terminal +emulator. If your theme is a 256-color theme (suffix `-256.toml`), you can still use RGB colors in your toml, and termshark will then try to pick the +closest matching color in the 256-color space. If termshark detects you are using base16-shell, then it will ignore colors 0-21 when choosing the +closest match, since these will likely be remapped by the base16-shell theme. + +Hopefully the meaning of the UI-element names is guessable, but one detail to know is the difference between focus, selected and unselected. In a +gowid application, one widget at a time will have "focus". For example, if you are navigating the packet's tree structure (the middle pane), one level +of that protocol structure will be shown in blue, and will be the focus widget. If you hit tab to move to the hex view of the packet's bytes (the +lower pane), then focus will move to the hex byte under the cursor; but the previously blue protocol structure in the middle pane will still be +obvious, shown in grey. That protocol level is now "selected", but not in "focus". So selected is a way to highlight a widget in a container of +widgets that will have focus when control returns to the container. Unselected means neither focus nor selected. + ### Config File Termshark reads options from a TOML configuration file saved in `$XDG_CONFIG_HOME/termshark/termshark.toml` (e.g. `~/.config/termshark/termshark.toml` on Linux). All options are saved under the `[main]` section. The available options are: +- `auto-scroll` (bool) - if true, termshark will automatically scroll down when packets are read in a live-capture mode (e.g. `-i eth0`) - `browse-command` (string list) - termshark will run this command with a URL e.g. when the user selects "FAQ" from the main menu. Any argument in the list that equals `$1` will be replaced by the URL prior to the command being run e.g. ```toml @@ -331,7 +518,7 @@ Termshark reads options from a TOML configuration file saved in `$XDG_CONFIG_HOM - `conv-resolve-names` (bool) - if true, have tshark provide conversation data with ethernet names resolved. - `conv-use-filter` (bool) - if true, have tshark provide conversation data limited to match the active display filter. - `conv-types` (string list) - a list of the conversation types termshark will query for and display in the conversations view. Currently limited to `eth`, `ip`, `ipv6`, `udp`, `tcp`. -- `copy-command` (string) - the command termshark executes when the user hits ctrl-c in copy-mode. The default commands on each platform will copy the selected area to the clipboard. +- `copy-command` (string) - the command termshark executes when the user hits `ctrl-c` in copy-mode. The default commands on each platform will copy the selected area to the clipboard. ```toml [main] @@ -340,9 +527,16 @@ Termshark reads options from a TOML configuration file saved in `$XDG_CONFIG_HOM - `copy-command-timeout` (int) - how long termshark will wait (in seconds) for the copy command to complete before reporting an error. - `dark-mode` (bool) - if true, termshark will run in dark-mode. -- `disable-shark-fin` (bool) - if true then it's safe to go back in the water. +- `debug` (bool) - if true, run a debug web-server on http://localhost:6060. Shows termshark/golang internals - in case of a problem. +- `disable-shark-fin` (bool) - if true then turn off the shark-fin screen-saver permanently. +- `disk-cache-size-mb` (int) - how large termshark will allow `$XDG_CACHE_HOME/termshark/pcaps/` to grow; if the limit is exceeded, termshark will delete pcaps, oldest first. Set to -1 to disable (grow indefinitely). - `dumpcap` (string) - make termshark use this specific `dumpcap` (used when reading from an interface). +- `ignore-base16-colors` (bool) - if true, when running in a terminal with 256-colors, ignore colors 0-21 in the 256-color-space when choosing the best match for a theme's RGB (24-bit) color. This avoids choosing colors that are + remapped using e.g. [base16-shell](https://github.com/chriskempson/base16-shell). +- `key-mappings` (string list) - a list of macros, where each string contains a vim-style keypress, a space, and then a sequence of keypresses. +- `marks` (string json) - a serialized json structure representing the cross-pcap marks - for each, the keypress (`A` through `Z`); the pcap filename; the packet number; and a short summary of the packet. - `packet-colors` (bool) - if true (or missing), termshark will colorize packets according to Wireshark's rules. +- `pager` (string) - the pager program to use when displaying termshark's log file - run like this: `sh -c " termshark.log"` - `pcap-bundle-size` - (int) - load tshark PDML this many packets at a time. Termshark will lazily load PDML because it's a slow process and uses a lot of RAM. For example, if `pcap-bundle-size`=1000, then on first loading a pcap, termshark will load PDML for packets 1-1000. If you scroll past packet 500, termshark will optimistically load PDML for packets 1001-2000. A higher value will make termshark load more packets at a time; a value of 0 means load the entire pcap's worth of PDML. Termshark stores the data compressed in RAM, but expect approximately 10MB per 1000 packets loaded. If you have the memory, can wait a minute or two for the entire pcap to load, and e.g. plan to use the packet list header to sort the packets in various ways, setting `pcap-bundle-size` to 0 will provide the best experience. - `pcap-cache-size` - (int) - termshark loads packet PDML (structure) and pcap (bytes) data in bundles of `pcap-bundle-size`. This setting determines how many such bundles termshark will keep cached. The default is 32. - `pdml-args` (string list) - any extra parameters to pass to `tshark` when it is invoked to generate PDML. @@ -359,6 +553,7 @@ Termshark reads options from a TOML configuration file saved in `$XDG_CONFIG_HOM term = "screen-256color" ``` +- `theme` (string) - the currently selected theme, if absent, no theme is used. - `tshark` (string) - make termshark use this specific `tshark`. - `tshark-args` (string list) - these are added to each invocation of `tshark` made by termshark e.g.