Contribute new 8-bit theme: Ansi 16

[chtenb]

I've been working on an adaptation of base16 to solve a few compatibility issues: Ansi 16.

• In contrast with the base16 highlighting scheme, Ansi 16 will work well with any ANSI terminal color theme and does not require that the bright colors are set to a specific scheme. In fact, the Ansi 16 guidelines encourage that the bright colors be compatible with the non-bright colors.
• In contrast with the pre-included ansi theme, Ansi 16 does make use of the bright ANSI colors, not just the first 8 base colors.

I've put together a .tmTheme called terminal-ansi16 that implements the Ansi16 scheme for bat. This is included in this PR. Apart from improving compatibility, the scope assignment rules are also more sophisticated than both the pre-included ansi and base16 themes. You can judge this by comparing https://github.com/chtenb/ansi16/blob/0a16273296bbd5d7f5335d62e7605071743794a9/terminal-ansi16.tmTheme with https://github.com/sharkdp/bat/blob/master/assets/themes/ansi.tmTheme and by looking at the screenshot comparisons below.

After this PR has been merged I intend to release a short tutorial somewhere which explains how to use bat, delta and VSCode to create a development setup where the syntax highlighting is fully equivalent between the editor and the terminal.

To get an idea of what that looks like, you can take a look at screenshots of the VSCode theme I just released. These themes set the integrated terminal theme to the same color palette as their in-editor theme. I've used the .tmTheme included in this PR for the diff views (using delta) in the terminal.

[chtenb]

@sharkdp In case this PR slipped your attention, do you think this could be merged?

[Enselic]

Sorry if I'm asking silly questions now, but I figured some input is better than no input. I am no expert in this area, so while this might be an obvious improvement to someone that is well acquainted with these kinds of themes, it is not obvious to me.

Could you perhaps show side by side comparisons (screenshots of bat) of the best existing theme vs your new theme? Along with details about what environment (terminal theme, etc) you are in.

Another blocker for getting this merged is that CI is failing. I think you just need to add your theme to a list in a test case. git grep on other theme names should make you find the right place easily.

[chtenb]

Thanks for your response @Enselic ! Ansi16 is meant as a drop-in replacement for the precluded ansi theme, so here is a side-by-side comparison screenshot (ansi <-> terminal-ansi16):

The terminal colors used are from my synthwave material theme

background: '#262335'
foreground: '#cccccc'
black: '#676e95'
red: '#f07178'
green: '#c3e88d'
yellow: '#ffcb6b'
blue: '#94afeb'
magenta: '#c792ea'
cyan: '#b9dedf'
white: '#b4b8d0'
brightBlack: '#676e95'
brightRed: '#ff5370'
brightGreen: '#87de97'
brightYellow: '#f78c6c'
brightBlue: '#b9dedf'
brightMagenta: '#c792ea'
brightCyan: '#91f5f8'
brightWhite: '#dddddd'

If you need any other information just let me know. I will make sure the tests are passing.

[chtenb]

For the sake of completeness, here is the same comparison again, but this time against the base16 theme

[chtenb]

Thanks @keith-hall, tests are passing now :)

[chtenb]

@Enselic is there anything else you need me to do?

[Enselic]

@chtenb I'm busy with other stuff at the moment, but I have not forgotten about this. It might just take some time before I get time to take a deeper look at this.

[chtenb]

@Enselic No worries! Just making sure this didn't get stuck in limbo. In the meantime, I've updated the OP with some more explanation and context.

[Enselic]

I've studied this a bit more and now have the following question (sorry again if it is silly): If the problem with base16 is "a few compatibility issues", why not work towards fixing those by amending base16, rather than attempting to introduce a new standard?

[chtenb]

Good question. Let me see if I can explain this clearly.

First of all there is the issue of practicality. The base16 standard is well established, thousands of users expect it to work the way it works. We can't just change that, and if we tried the maintainer would reject the proposals. And that's fair.

Secondly, I don't think the base16 standard is broken. I mentioned that Ansi 16 "fixed" a few issues, but it doesn't do that by "fixing" base16. Rather, Ansi 16 makes different design choices, ending up with a different standard which purposefully happens to be compatible with ANSI.

The big difference in design is the fact that base16 also aims to support theming of background colors. The standard ANSI colors don't incorporate background colors and so by design base16 cannot be compatible with ANSI terminal color palettes, and it doesn't aim to be.

In contrast, Ansi 16 uses all of the 16 colors as foreground colors and does not say anything about background coloring. The reason is that in my experience theming of background colors is rarely needed. Most text environments will be able to accomplish different background shades by mixing the background with a foreground colors by some alpha value. So this allows us to use all 16 ANSI colors for foreground purposes, which results in the delightful facts that 1) we are ANSI compatible 2) we have more colors to our disposal and therefore can make our syntax highlighting guidelines much more sophisticated and specific.

[Enselic]

I use macOS Monterey with default terminal color settings, and here is how terminal-ansi16 vs base16 looks for me, both with Light and with Dark system themes.

Click to expand!

Dark, base16

Dark, terminal-ansi16

Light, base16

Light, terminal-ansi16

So clearly there is a difference, but to my eyes there is no dramatic improvement. Again, sorry if I am just unable to see what others see. But I think there needs to be quite a big improvement for us to justify adding a new theme to the default set of themes.

So I'm thinking maybe as a first step you should try to convince people to use your theme as a custom asset. And if it becomes popular among users, we can consider adding it to the default set of themes.

[chtenb]

Thanks for your reply and for looking into this. I agree with your judgement based on your screenshots, and I also agree with
your statement that

there needs to be quite a big improvement for us to justify adding a new theme to the default set of themes.

However, looking at the screenshots I think this might be an unfair comparison because I see no more than 6 colors. This could be due to the terminal color palette not having bright colors defined, or it could be due to the rust syntax definition being very rudimentary.

Would you mind posting the terminal color palette that you are using? I don't have access to mac OS, so I can't try your setup.

In the meantime I will have a look at the rust syntax file that is used by bat.

Also last but not least, I think that a comparison with the ansi theme is the most fair, since ansi16 is meant as a drop in replacement for that.

PS. For some more context and example of compatibility trade-offs with base16 and base16-256 you can also have look at this thread #1033

[Enselic]

Sorry for doing a silly comparison. Before exploring further, I would like to bring up another proposal/question: If terminal-ansi16 is a drop in replacement for ansi, why not improve the ansi theme rather than introducing a new theme? You mentioned "The standard ANSI colors don't incorporate background colors", but I would expect todays systems to do that, right?

[chtenb]

Sorry for doing a silly comparison. Before exploring further, I would like to bring up another proposal/question: If terminal-ansi16 is a drop in replacement for ansi, why not improve the ansi theme rather than introducing a new theme?

I would not be opposed to this, but I didn't want to be to blunt. However, this would bring the additional benefit that existing users of the ansi theme would experience an "upgrade" for free.

You mentioned "The standard ANSI colors don't incorporate background colors", but I would expect todays systems to do that, right?

The ANSI colors are only 8 colors: black, red, green, yellow, blue, purple, cyan and white. The ANSI standard colors don't say anything about colors being background colors, they are just colors. These 8 colors can be activated in a terminal by special designated control characters. There are more rendering parameters than just toggling these colors, and one of these parameters is "increased intensity". Modern terminals often implement this parameter by having an alternative color version of each of the 8 ANSI colors, which is used whenever this increased intensity parameter is on. This is why terminal palettes actually have ANSI 16 colors in practice, even though there are only 8 in the strictest sense of the specification.

Apart from these, you also have

• the default back and foreground colors
• the ANSI extended color set, which consists of another 256 color codes

All modern terminals support customizing the default back and foreground color and the 16 ANSI colors in what is often called a "terminal palette". The ANSI extended set usually are not customizable. (Side note: base16-256 relies on ANSI extended being customizable by your terminal).

If you're interested in more details, here is a neat article that discusses terminal control sequences in more depth

Now what I meant in that quote was that if you look at the 16 colors defined by the base16 standard you can see that 8 of the colors are actually shades of grey, instead of being an "increased intensity" version of their dual ANSI color.

The reason base16 repurposed these color codes is for background things like rendered "selected text", just like the commentary points out. However, I find that modern terminals often implement things like "selected text" by applying some color/alpha blend formula on the colors of each character+background instead of applying a dedicated ANSI color. So that is why ANSI 16 doesn't repurpose these colors codes, which frees them up for regular syntax highlighting things.

[chtenb]

Sorry for doing a silly comparison. Before exploring further, I would like to bring up another proposal/question: If terminal-ansi16 is a drop in replacement for ansi, why not improve the ansi theme rather than introducing a new theme?

I would not be opposed to this, but I didn't want to be to blunt. However, this would bring the additional benefit that existing users of the ansi theme would experience an "upgrade" for free.

I just thought about this some more and I realized that there may be the following potential issue.
Ansi 16 relies on the fact that all 16 ANSI colors are properly visible as foreground colors. However, in some light terminal color palettes the white color may defined to be not very visible, and conversely in dark terminal palettes the black color may be ill defined as well. The current bat ansi theme refrains from using black and white at all, it only uses ansi colors 1-6, thereby avoiding this issue when it exists. So that is a difference I did not mention before and may be a reason to keep both the ansi theme and the ansi16-terminal theme.

[Enselic]

As I'm going through the PR inbox I'm not sure what to do with this PR to move it forward. What do you think @chtenb ?

[chtenb]

To come back to one of your earlier posts: some time ago I made some improvements on the Rust support of the tmThemes, because I did some Rust programming myself. Here is a screenshot comparing ansi16-terminal vs ansi

From a user perspective this ansi16-terminal theme attempts to give a better highlighting experience because of the sophisticated scope assignment definitions and use of bright ansi colors, compared to the ansi theme. There is a bunch of languages that I've explicitly taken care for by making the scope assignments play well with the most commonly used TextMate grammars. These languages include C, C++, HTML, CSS, Haskell, PureScript, Javascript, JSON, Python, Rust, several markup languages. I intend to improve and maintain the scope assignment logic over time as I'm using this daily and accept sensible PRs.

All in all it's a direct upgrade from the ansi theme, except for the fact that ansi16 requires that all the terminal colors should be properly visible as foreground color. This is sensible, but will not always be the case. Therefore this should be mentioned in the docs.

I still think that this theme would be a valuable addition to the programming community. My proposal would be:

1. I update the README.md at https://github.com/chtenb/ansi16 to cover the practical side of using the theme, instead of dumping the design guidelines there (as is currently the case) and mention the languages that have "enhanced support".
2. I update this PR to point to the newest version and update the README of bat to include the requirement that all the terminal colors should be properly visible as foreground color in order to use the ansi16-terminal theme.
3. This PR gets merged
4. I write a blogpost about some cool use cases for this theme with various developer tools. See for instance my latest post at 🚀 Use light/dark theme base on terminal theme or system variable. dandavison/delta#447

[Enselic]

I'm sorry, but I still don't see a big benefit for bat users here. Rather than saying nothing for another half year I will close this PR now. Thank you for trying to explain everything. It just doesn't "click" for me where I go "sounds great, we need this".