Internal kitty shell has broken backspace when term != xterm-kitty

[egnor]

Describe the bug
If the term config variable is xterm instead of xterm-kitty, the internal kitty shell (shift-control-escape) seems to have something wrong with its line editing.

(I find it necessary to set term to xterm or similar, otherwise upon ssh to many hosts, they complain that xterm-kitty is not a known terminal type.)

To Reproduce
Steps to reproduce the behavior:

1. Edit kitty.conf to set term xterm
2. Start kitty
3. Press shift-control-escape (with default keymap) to launch a kitty shell windowpane
4. Type something
5. Backspace to erase
6. Instead of erasing, you get extra spaces

Screenshots
This is after typing "hello", then pressing backspace 5 times.

Environment details

kitty 0.22.0 (887347106d) created by Kovid Goyal
Linux skully 5.11.0-22-generic #23-Ubuntu SMP Thu Jun 17 00:34:23 UTC 2021 x86_64
Ubuntu 21.04 \n \l
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=21.04
DISTRIB_CODENAME=hirsute
DISTRIB_DESCRIPTION="Ubuntu 21.04"
Running under:X11
Loaded config files:
  /home/egnor/.config/kitty/kitty.conf

Config options different from defaults:
background            Color(red=48, green=48, blue=48)
background_opacity    0.85
enable_audio_bell     False
font_size             8.0
initial_window_height (50, 'cells')
initial_window_width  (80, 'cells')
linux_display_server  x11
remember_window_size  False
resize_in_steps       True
tab_bar_style         powerline
term                  xterm
visual_bell_duration  0.1

Additional context
This can be reproduced with a kitty.conf as simple as

term xterm

[kovidgoyal]

DO NOT change TERM. If you change TERM everything will break. To solve
your SSH problems see
https://sw.kovidgoyal.net/kitty/faq/#i-get-errors-about-the-terminal-being-unknown-or-opening-the-terminal-failing-when-sshing-into-a-different-computer

And the internal shell uses readline, which is what is breaking when you
use the incorrect TERM

[egnor]

(Kovid, avert your eyes)

In case anyone else is walking this dark path, I found the issue. It only happens with the binary download, and probably only on linux; if you build kitty from source, the problem goes away.

The reason is that the binary download uses bundled copies of shared libraries like libreadline.so.8 and libncursesw.so.6, rather than using the system ones. This keeps everything self-contained which is nice and robust. However, the bundled libraries were built with a prefix set to a temporary build directory instead of the normal system prefix. This is also sensible, because it keeps them isolated from the underlying system and its vagaries.

However, this isolation goes a little too far in this case (though Kovid might disagree). Specifically, the bundled libncurses (used by libreadline in the kitty shell) does not search the standard system terminfo directories like /lib/terminfo or /usr/share/terminfo. Kitty sets $TERMINFO to make sure it sees the xterm-kitty definition, but that's it. Furthermore, for whatever reason, with Python using readline using ncurses, an unrecognized terminal type doesn't generate any errors. It just silently starts doing all kinds of bogus things on the terminal, like sending a space for backspace. So you don't even get a breadcrumb like "terminal type xterm not found".

For those of us who have already chosen this dark path (THIS IS NOT GENERAL ADVICE), you can resolve this by adding this to your kitty.conf:

env TERMINFO_DIRS=/usr/share/terminfo:/etc/terminfo:/lib/terminfo

This causes the bundled libncurses to add these system directories to its search paths. (If your system puts terminfo files elsewhere, list them instead.) If you like to use kitty shell from non-kitty terminals or remotely on other hosts, make a similar TERMINFO_DIRS setting in your shell startup files as well. (You'll still need the kitty.conf edit as well, since kitty's built-in shift-control-Esc shell will see the illicit TERM but won't see shell startup files.)

[kovidgoyal]

Haha, I refuse to look away :)

Might be worth adding --with-terminfo-dirs to the bypy build script for
ncurses. That will fix this, and shouldn't affect using TERM=xterm-kitty
since kitty also sets the TERMINFO env var to point to its bundled db

[kovidgoyal]

And patched bypy accordingly: kovidgoyal/bypy@6379b39