TCell $TERM not recognised

[lyneca]

TCell seems to need to recognise the value of $TERM. There's an issue over at the TCell repo, but it's pretty old. This means that up won't work for many people using less-popular terminals (Kitty for me).

A workaround at the moment is to alias:

$ alias up="TERM=xterm up "

Is there some way to bypass this from the up side?

[kalbasit]

Having the same issue with tmux-256color.

[akavel]

Do I understand it correctly, that the result is you're getting an error message terminal entry not found, and up is not starting? (But starts with the TERM=xterm workaround?) This was not clear to me from your report, only after reading the issue at the tcell repo I started to understand that this is probably what you are trying to tell me. So I wanted to ask if you confirm my understanding?

I'm also kinda confused now, given that the tcell issue seems to claim it's resolved :/ why did they write so, if the issue still happens? I seem to be using a very new version of tcell :/

[lyneca]

Yup, sorry for the unclear report. I'm happy to alias, and having uncommon $TERM variables isn't that, well, common, so it's not really that much of an issue.

I think that TCell's solution may have been to just hardcode a bunch of $TERM values, which may be why it's not working.

[akavel]

Would it make any sense if I checked internally if TERM is set, and if not, then set it to some fixed value? Should it be xterm? It feels rather hacky, but probably better than an error, and one should still always be able to override it...

[kalbasit]

@akavel in most cases TERM is set. The problem is that it seems to not recognize some of the TERM, or at least TCell is not able to.

[gunar]

Same here with screen-256color-bce

[akavel]

Hmm; from gdamore/tcell#225 and a comment block in tcell/terminfo/terminfo.go, I gather that support for some terminals is distributed only with tcell sources, as gzipped helper files. That feels like a somewhat weird choice for me at the moment; I wonder why it's not parsed dynamically from OS-provided data for unknown TERMs? It seems like termbox is doing that; I wonder if its code could be adapted somehow to provide support for tcell... or is such dynamic parsing already available somewhere in tcell?

edit: Some research on the topic from tcell repo & linked issues in other repos:

gdamore/tcell

Recognizing more terminals · Issue #118 · gdamore/tcell (Sep 1, 2016 comment by gdamore)

The reason its a separate program is because of the CGO and curses (or ncurses) dependency -- I didn't want to write a full terminfo database parser (and the terminfo database format is not necessarily standard or portable, although ncurses uses the same format everywhere).

More terms by joshuarubin · Pull Request #136 · gdamore/tcell (Sep 10, 2017 comment by gdamore)

gdamore commented on Sep 10, 2017:

[...] The problem is that there is not a standard for the entries. I have another idea I want to explore using tput and co to build entries if they are missing. [...]

5paceToast commented on Sep 12, 2017:

[...] Given the contents of that standard, however, we can reasonably assume that curses can be considered a reference implementation (as it significantly influenced the design thereof). [...]

gdamore commented on Sep 12, 2017:

[...] The binary format is non standard. Sadly I don't consider ncurses to be "standard" even though most of the common implementations simply are ncurses based. [...]

fixes #167 Break up the database by gdamore · Pull Request #168 · gdamore/tcell

This completely restructures the database of terminal types, putting
each terminal in its own file. We also compress the database files,
and use infocmp instead of trying to use the C level API.

The mkdatabase script will rebuild the entire database from the terminfo
files on the system. Individual entries can also be built by simply running
the mkinfo program with the terminal type.

zyedidia/micro

terminal entry not found · Issue #20 · zyedidia/micro

This should be fixed in v1.4.1 without the need for the mkinfo program

Terminal entry not found · Issue #1093 · zyedidia/micro

This problem is resolved for me by using the latest nightly build. Currently micro 1.4.2-dev 29

From reading some other issues as well I think that you have to use 1.4.2 or newer for some terminals like Termite, there was still some unsharpness in 1.4.1

Wikipedia

Terminfo - Wikipedia

Terminfo data is stored as a binary file, making it less simple to modify than termcap. The data can be retrieved by the terminfo library from the files where it is stored. The data itself is organized as tables for the boolean, numeric and string capabilities, respectively. This is the scheme devised by Mark Horton, and except for some differences regarding the available names is used in most terminfo implementations.[5] X/Open does not specify the format of the compiled terminal description. In fact, it does not even mention the common tic or infocmp utilities.[6][7] Because the compiled terminfo entries do not contain metadata identifying the indices within the tables to which each capability is assigned, they are not necessarily compatible between implementations. However, since most implementations use the same overall table structure (including sizes of header and data items), it is possible to automatically construct customized terminfo libraries which can read data for a given implementation. For example, ncurses can be built to match the terminfo data for several other implementations.[8]

Summary as of now:

• the format of the terminfo databases is reportedly not well/officially standardized, though ncurses seems to be parsing them in some way, which seems to be somewhat of an unofficial standard (?)
  • actually, it seems to vary (as in "internal implementation detail") with a particular terminfo implementation available on particular OS; the protocol seems to be standardized at the level of a C library API IIUC, not at the level of compiled terminfo files, unfortunately
• https://github.com/nsf/termbox-go has some partial implementation of a parser for usual format of the DB
• tcell also has some parser, provided as a separate program, somewhat tweaked in a fork in https://github.com/zyedidia/micro
• TODO: find out more about some "infocmp" reported in one issue in micro
• TODO: is there maybe some third party Go library for parsing terminfo files created by someone in the meantime?

[akavel]

I've added a detailed error message, outlining suggested steps that can be taken in order to try and mitigate the issue. I don't currently have good ideas how to resolve this in a better way. If you have some, I'm open to discussion.

Thanks, and I'm sorry I couldn't find a better way for now.

[masaeedu]

Thanks for adding that error message @akavel. That's already quite helpful for someone who doesn't understand the inner workings of this terminal stuff; unfortunately I'm trying to figure out what to do about a missing xterm-24bit entry, and still can't work out next steps from this error message:

error: terminal entry not found
Your terminal code:
        TERM=xterm-24bit
was not found in the database provided by tcell library. Please try checking if
a supplemental database is found for your terminal at one of the following URLs:
        https://github.com/gdamore/tcell/raw/master/terminfo/database/c9/c93be29d
        https://github.com/gdamore/tcell/raw/master/terminfo/database/c9/c93be29d.gz
If yes, download it and save in the following directory:
        $HOME/.tcelldb/c9/
then try running "up" again. If that does not work for you, please first consult:
        https://github.com/akavel/up/issues/15
and if you don't see your terminal code mentioned there, please try asking on:
        https://github.com/gdamore/tcell/issues
Or, you might try changing TERM temporarily to some other value, for example by
running "up" with:
        TERM=xterm up
Good luck!

The URLs in the error message don't seem to exist anymore. I realize this may be a case of RTFM, but is there a way to substitute those fixed URLs with some basic information on what's going on and what kind of file(s?) we're trying to retrieve and stick in our ~/.tcelldb/xyz folder?

[jean]

@masaeedu not sure if this helps, but searching that repo for 'xterm' comes up with {"name":"xterm-24"}, which looks similar, here and here.