View data in CLI

[loveorigami]

I have file tst.php

require_once 'vendor/autoload.php';
d([1, 2, 3]);

run this file in cli with php-kint v3

and with php-kint v1

Why data displaying is not correct in php-kint v3?

[jnvsor]

That's weird - are you running them both in the same terminal?

In either case - your terminal doesn't seem to support unicode and/or color codes. On windows?

[loveorigami]

Yes, on windows )

[jnvsor]

Could you var_dump the value of the KINT_WIN and DIRECTORY_SEPARATOR constants?

[loveorigami]

In v1

in v3

[loveorigami]

I am find solution

Kint\Renderer\CliRenderer::$cli_colors = false;
d([1, 2, 3]);

[jnvsor]

Yeah that turns off the colors, but it's not supposed to look like that to begin with:

┌──────────────────────────────────────────────────────────────────────────────┐  
│ $a                                                                           │  
└──────────────────────────────────────────────────────────────────────────────┘  
...  
════════════════════════════════════════════════════════════════════════════════

I'd like your help to fix this, but I have no idea when I'll get around to it

[michabbb]

never code on windows 🙈 😂

[DRSDavidSoft]

Now that Windows 11 & Microsoft Terminal support VT100 colors and UTF-8 table characters, I'm looking for a way to enable it back on CLI!

Config

Kint config

// enable cli colors (only works on *nix environment)
Kint\Renderer\CliRenderer::$cli_colors = true;

// force utf-8 output on windows
Kint\Renderer\CliRenderer::$force_utf8 = true;

VT100 processing

• Windows: make sure you're on at least Windows 10 build 16257 or later (I suggest using 1909 or later, or Windows 11)

  This registry modification will force it to be always ON:

  REG ADD HKCU\CONSOLE /f /v VirtualTerminalLevel /t REG_DWORD /d 1

  To make sure the UTF-8 console codepage is used, try chcp 65001:
  batch :: change encoding to UTF-8 to display unicode characters correctly @chcp 65001>nul

• PHP: to enable ENABLE_VIRTUAL_TERMINAL_PROCESSING from php-side (minimum version 7.2.0

  // enable `ENABLE_VIRTUAL_TERMINAL_PROCESSING` flag (required PHP 7.2.0 as minimum)
  
  if ( substr(PHP_OS, 0, 3) === 'WIN' )
  {
      sapi_windows_vt100_support(STDERR, true);
      sapi_windows_vt100_support(STDOUT, true);
  }

  Note: this flag is enabled by default in my tests, but if somehow it gets disabled, you can enable it again (e.g. I found by calling exec() on some binaries, it might get disabled)

Console Emulator

• Use the default cmd.exe (or better known as conhost.exe) on modern Windows versions only!
• Use better, electron-based terminals (such as VS Code's integrated terminal emulator)
• Use dedicated awesome terminals such as ConEmu which had VT100 support from wayyyy back -- tip: Cmder is also based on this, and works very good
• Use the new Microsoft Terminal on 10 and later
• If you're still on Windows 7, try Mintty -- which is a very awesome project itself

General Advice & Conclusions

never code on windows 🙈 😂

I have to agree with you, but just a minor correction: never code on (old, non-modern versions of) windows!

Off-topic rants

Microsoft did a very good overhaul of the outdated console host and related console stuff in recent years, I can confidently say that the new stuff is on par with macOS and Linux-level CLI development.

Also, PHP on CLI for the win! I don't like any of the "Python" cli development, personally. IMHO, the syntax of PHP is far superior! :)

[DRSDavidSoft]

Results

[jnvsor]

@DRSDavidSoft That's great!

I don't code on windows here, so I'd have to set up a VM to test this stuff myself. I know about WSL but I didn't know it'd work like this with just normal cmd nowadays (Or are these screenshots with WSL?)

Does this basically mean that in normal circumstances you can run kint from a windows terminal and colors and utf will work out of the box? If that's the case I may just change the defaults, remove the windows-specific code there, and close this issue.

[DRSDavidSoft]

@jnvsor Thanks for all your work!

I mainly code on Windows at the office, and for a project, I'm developing a CLI script with PHP. Note: I haven't used WSL, it's just native php.exe builds for windows from here i.e. the same stuff that WAMP and IIS use.

The screenshots you see are native cmd.exe without WSL, the only changes I've done is tweaks to the color schme.

Sidenote (off-topic): I've been using Cygwin since long before WSL/1 was a thing, so I don't have much experience with WSL either -- although, in my tests, mingw/Cygwin build of PHP behave the same.

Does this basically mean that in normal circumstances you can run kint from a windows terminal and colors and utf will work out of the box? If that's the case I may just change the defaults, remove the windows-specific code there, and close this issue.

Considering Windows 11 and PHP 8.x, yes! But since Kint is still supporting the older PHP versions on Windows, the time hasn't come yet to do this.

I'm sad to say, until you bump the minimum supported PHP version by Kint from 5.6 to 7.2, it might not.

For now, I'd suggest to consider something like this:

• set the default to false
• if PHP_VERSION is 5.6 or less, return false immediately
• does this function exist: sapi_windows_vt100_support()
  • no? return false immediately
• does that function return false? return false immediately
  • otherwise, return true

Here's a code for that:

Kint\Renderer\CliRenderer::$force_utf8 = false;

if ( substr(PHP_OS, 0, 3) === 'WIN' )
    Kint\Renderer\CliRenderer::$force_utf8 = checkWindowsVT100();

function checkWindowsVT100()
{
    if ( !function_exists('sapi_windows_vt100_support') )
        return false; // note: no need to check for PHP 5.x as this function only exist on PHP 7.2.0

    if ( sapi_windows_vt100_support(STDOUT) )
        return true;

    return false;
}

A bit of history:
I clearly remember having the same difficulties mentioned in the beginning of this issue, in the days of using PHP/7.0 -- namely, no VT100 for color support and the unicode table drawing characters getting garbled. That's why I stopped using Kint on the CLI and only used it for CGI scripts (which it worked awesome-ly).

After I upgraded to PHP/7.4, I modified my existing logging utilities to output in color. A couple of days ago, I needed to dump a variable, and I remembered to use Kint, so I checked to see if it would work in my project. It did, however, the color was turned off by default!

So I dig into the code to see if I could make the colors for Kint output correctly, and it does!

Since PHP/7.x is EOL now, and the versions of Windows 10 (and older) that does not support VT100 correctly are also EOL, I think it's a good time to implement the code I suggested, so Kint would output in all its glory, with the unicode tables, and the colors!

After PHP 5 and 7 support is dropped from Kint (maybe in 4-5 years, when Windows 10 is also EOL), then it would be possible to treat a Windows/PHP environment the same as on Linux and macOS (and thus removed all Windows-specific code for CLI).

Anyways, sorry for the long comment! I'll be glad to help, if any. Especially, with testing, if needed.

[jnvsor]

@DRSDavidSoft That's great!

I'm sad to say, until you bump the minimum supported PHP version by Kint from 5.6 to 7.2, it might not.

I was worried about that too: I'd have to postpone this until v5. But since we have a function to check for support I can slap this in a minor release instead.

The only thing I'm worried about is this:

the versions of Windows 10 (and older) that does not support VT100 correctly

What kind of overlap do we have between buggy windows versions that won't show utf8 and/or color control codes correctly that sapi_windows_vt100_support still returns true for? I wouldn't want to completely fubar the output with control codes just because someone's on a buggy old windows version (As opposed to a buggy new windows version)

Or should I just ignore that edge case entirely and just check with the function? (I'll take your word for it since you're half the userbase of this feature hehe)

Here's my WIP diff, if you want to give it a whirl:

@@ -69,8 +76,8 @@ class CliRenderer extends TextRenderer
     {
         parent::__construct();
 
-        if (!self::$force_utf8) {
-            $this->windows_output = KINT_WIN;
+        if (!self::$force_utf8 && KINT_WIN && (!KINT_PHP72 || !\sapi_windows_vt100_support(STDOUT))) {
+            $this->windows_output = true;
         }
 
         if (!self::$terminal_width) {

[DRSDavidSoft]

@jnvsor Awesome, I did some research in order to answer confidently.

I was worried about that too: I'd have to postpone this until v5. But since we have a function to check for support I can slap this in a minor release instead.

That would be a good idea, I'll be glad to test Kint 4.1. Also, I figure there could be other kinds of PHP-5.6 related code that might get dropped in Kint 5, when backwards compatibility for PHP-5.6 is no longer needed.

What kind of overlap do we have between buggy windows versions that won't show utf8 and/or color control codes correctly that sapi_windows_vt100_support still returns true for? I wouldn't want to completely fubar the output with control codes just because someone's on a buggy old windows version (As opposed to a buggy new windows version)

I did some tests to be sure, and also looked at the PHP source code. This function will not return true, if the host console doesn't support VT100. According to the PHP source code, it's calling the GetConsoleMode() from Win32 SDK, and checks if the ENABLE_VIRTUAL_TERMINAL_PROCESSING flag is present.
Since the flag is only introduced in recent versions of Windows 10 where VT100 is handled correctly, it simply doesn't exist in the older versions of Windows.
This means that if the Windows' Console Host doesn't support VT100 processing, the PHP's VT100 support function will return false, which is consistent with my tests (running PHP 7.2 and PHP 8.x on Windows 7 or Server 2008). (n/b: MSDN also provides a code sample which also confirms this method).

In short, you can be assured that the terminal will support VT100 for certain, if this functions returns true.

Or should I just ignore that edge case entirely and just check with the function?
I'd say that this leaves little to no edge cases, and for some (really unlikely) scenarios, you can ignore it completely.

The thing is, if you check with sapi_windows_vt100_support() and it returns true, I'd say there is zero chance that it would be a false positive and render CLI color/utf-8 incorrectly.

On the other hand, what if it returns false, and it's decided that the host for certain doesn't support VT100 processing, i.e. a false negative? That's where the edge cases arise, since the user could be using another terminal emulator on Windows (such as ConEmu, which has support for VT100 color/utf-8) -- but Kint decides not to render the color/utf-8, even though the actual terminal emulator supports the VT100 escape sequences.

For the false negative, I'd say you can safely ignore it, as the worst case scenario is that the user would be missing the color/utf-8, but the results would render correctly (same as the current Kint 4.0 behavior).

In this unlikely scenario, the users can either a) force-enable color output for Kint themselves, b) don't care and ignore not having the colors, c) upgrade their OS to a supported one. Honestly, the Windows 7 and the buggy Windows 10 releases are all EOL (the same way all PHP 5.x and PHP 7.x are EOL now), so I wouldn't stress on that much.

You can also test for other terminal emulator (such as ConEmu) specifically like this, but that's a whole another matter entirely.

if ( filter_var(getenv('ConEmuANSI'), FILTER_VALIDATE_BOOLEAN, FILTER_NULL_ON_FAILURE) === true )
    Kint\Renderer\CliRenderer::$force_utf8 = true;

Pros: Better support for ConEmu, which is an awesome terminal emulator
Cons: Only for ConEmu, missing all other terminal emulators, such as Microsoft Terminal, Mintty, etc ...

(I'll take your word for it since you're half the userbase of this feature hehe)
Lol flattered to hear that, I feel like I'm the >1% of people who develop CLI applications with PHP on Windows 😅 (I also obviously use them on Linux too, so it might be more cross-platform that Windows-only)

Here's my WIP diff, if you want to give it a whirl:

So clean and efficient! I didn't know there were macros like KINT_PHP72 or KINT_WIN specifically for checking PHP version and OS type! :D

I tested it out, and the logic also seems to make sense.

The only issue I'd see with this is the false negatives, which as I mentioned, I don't consider to be a top priority. In fact, it's up to the terminal emulator to announce to PHP and Kint that it supports VT100!

I'm not sure if you're doing this, but you need to also check for the TERM environment variable on *nix too, for example if TERM=xterm, TERM=xterm-256color or TERM=cygwin, then VT100 is supported. Otherwise, you shouldn't output color on Linux.

All Linux-based terminal emulators will set this environment variable properly, which means that the terminal emulator is announing to the application that it will support the VT100 escape codes.

If it's missing, you can still output the VT100 color codes, but there is NO guarantee that what will the terminal emulator do with this. In fact, that's part of the problem on Windows: the conhost.exe did not specify to the program that it would support VT100 escape codes. Now, if the program would output the escape codes blindly, Windows would display garbled characters, whereas most Linux terminal emulators would display it correctly.

As always, I'm super sorry for the really long comment, hopefully it wasn't entirely off-topic or useless.
Thanks for the work you do!

Looking forward for the changes to Kint

[DRSDavidSoft]

@michabbb Hey Micheal, may I ask why the downvote? Is it something that I said?

[jnvsor]

So clean and efficient! I didn't know there were macros like KINT_PHP72 or KINT_WIN specifically for checking PHP version and OS type! :D

Yeah but they're undocumented and only used internally so you shouldn't rely on them in your own code :)

It's actually giving me trouble with psalm because psalm gets their type wrong and won't let me fix it with annotations because they're constants...

the user could be using another terminal emulator on Windows (such as ConEmu, which has support for VT100 color/utf-8) -- but Kint decides not to render the color/utf-8, even though the actual terminal emulator supports the VT100 escape sequences.

I remember using kint in CLI on windows with the git shell installation (Though I avoided windows as much as possible) and it worked with colors just fine, though that might have been because I was running it from a linux server through SSH...

For the false negative, I'd say you can safely ignore it, as the worst case scenario is that the user would be missing the color/utf-8, but the results would render correctly (same as the current Kint 4.0 behavior).

I plan on replacing the current windows output with plain ascii anyway since it's clearly not working correctly. That should improve things for basically everyone.

I'm not sure if you're doing this, but you need to also check for the TERM environment variable on *nix too, for example if TERM=xterm, TERM=xterm-256color or TERM=cygwin, then VT100 is supported. Otherwise, you shouldn't output color on Linux.

That's a good point. I know some programs (ls comes to mind) remove colors when being piped to something else, so I'm pretty sure there's more to check with this than just $TERM...

I'll dig into it, but I doubt I'll add checks for colors in linux. (At least in 4.x) I personally use less -R occasionally so I suspect there may be people relying on it outputting colors, and configuration would get a lot more complex if I started adding a bunch of flags for that too. It'd need to be rethought out from scratch

[DRSDavidSoft]

the user could be using another terminal emulator on Windows (such as ConEmu, which has support for VT100 color/utf-8) -- but Kint decides not to render the color/utf-8, even though the actual terminal emulator supports the VT100 escape sequences.

I remember using kint in CLI on windows with the git shell installation (Though I avoided windows as much as possible) and it worked with colors just fine, though that might have been because I was running it from a linux server through SSH...

I believe that's exactly it, the raw output is passed from Windows to Linux in this scenario, and the Windows-side doesn't care what's that being passed, escape code, or some other binary data. Then, in the Linux side, the GUI Terminal emulator (like GNOME Terminal or Konsole) actually parse and render the received escape code data.

The reverse can also work correctly, e.g. if I ssh into my Linux box from Windows, I can run htop or screenfetch and it would render correctly on Windows. (sidenote: PuTTY, a popular SSH client for Windows with built-in Terminal emulator, already supported the escape code since the ancient XP era! That's also what Kitty is based on.)

I plan on replacing the current windows output with plain ascii anyway since it's clearly not working correctly. That should improve things for basically everyone.

That's a good idea, although I suggest mostly focusing on the general color/utf-8 based rendering, as it's more cross-platform, and Microsoft is finally getting their act together to fix these annoying bugs, so the conhost.exe finally work correctly like on almost all other terminal emulators for other OSes out there.

In the future (especially on Windows 11), I expect the VT100-based codes to work 100% correctly out of the box, like how it does now on macOS, and Linux.

That's a good point. I know some programs (ls comes to mind) remove colors when being piped to something else, so I'm pretty sure there's more to check with this than just $TERM...

I'll dig into it, but I doubt I'll add checks for colors in linux. (At least in 4.x) I personally use less -R occasionally so I suspect there may be people relying on it outputting colors, and configuration would get a lot more complex if I started adding a bunch of flags for that too. It'd need to be rethought out from scratch

That's also a great idea! I'd suggest having a general cli_color_enabled value, that is configured by the user (and defaults to true), and a general cli_color_supported that would work both for Windows, and Linux. (names can be anything, I used these for example).

That way, the user can set cli_color_enabled to false, in order to force no color/utf-8 output.
Otherwise, Kint can do a check for determining the value of cli_color_supported, based on the current running platform (e.g. Windows or Linux).

If it's on Windows, check for *_vt100_support() return value. Otherwise, check for the proper environment variables.

This can allow the codebase to be simplified in this regard, while also becoming more logically understandable.

In any case, if the either cli_color_enabled is false, or cli_color_supported is false, instead of color/utf-8 a simplified version could be rendered for the user.

Once again, sorry for the long comment, thanks for your development, I really appreciate it! :D

[jnvsor]

That's a good idea, although I suggest mostly focusing on the general color/utf-8 based rendering

Yeah it's mostly just because we still support < 7.2. I'd love to get rid of the edge cases, but not just yet

That way, the user can set cli_color_enabled to false, in order to force no color/utf-8 output.

Well that already exists it's CliRenderer::$cli_colors - I'm just thinking we'd need another property to force it one way or another if we enable automatic detection, like we have CliRenderer::$force_utf8 for windows and mode constants for Kint::$enabled_mode

Edit: Anyway it's 4am here so I should sleep. Try 0aa8437 and see if that does the job, then I'll push it to master

[DRSDavidSoft]

Confirmed that this works correctly

PHP Version	VT100 Support	Kint CLI Result	Correct Behavior?
7.0 (no vt100 support)	No (Win 7)	No color	✔
8.1	No (Win 7)	No color	✔
7.0 (no vt100 support)	Yes (Win 11)	No color	✔
8.1	Yes (Win 11)	Color	✔

Thanks again, and good night!

n/b 7.2-7.4 also works in place of 8.1,
5.6 also works in place of 7.0

---

May I also suggest the following flags:

• CliRenderer::$cli_colors (i.e. does the user request color?)
  • true: force yes
  • false: force no
  • null: auto detect

reason: in this case, $cli_colors means force the value specified by the user. if it's ok to differentiate between false and null, then I'd suggest this

or:

• CliRenderer::$bypass_detect_cli:
  • false: try to detect if the cli supports color.
    only allow color if both this and $cli_colors is true
  • true: force the value provided by the user, regardless of detection

reason: I believe the term force in the flag name could be ambiguous, I mean what would be the desired behavior if $cli_colors is false, but $force_* is true? One outcome is to not output color since $cli_colors is false, but I'd argue that since $force_* is true, I'd expect that Kint should output color.

However, the term bypass_detection clearly means stop detecting the color support, respect the value specified by the user, even though if the console doesn't support color.

---

Also, instead of this:

[' ', '=', ' ', '|', ' ', '-', ' '],

I suggest the following:

['.', '=', '.', '|', '\'', '-', '\''],

[jnvsor]

Confirmed that this works correctly

Great!

May I also suggest the following flags

The bool|null option definitely seems like the way to go, but that'd be a breaking change. It'd have to wait for a major version and that's probably quite a ways off. I'd rather work on smaller easier to implement features and drop some minor versions instead

I suggest the following

My first thought would be that the . should be , to match ' but that depends on the terminal font, so my second thought is that if your terminal font has a slanted ' glyph it won't line up. Something like:

.---.
| a |
’---’

I tried extending the top/bottom and sides and it didn't look right either:

|---|
| a |
|---|

-----
| a |
-----

-----
| a |
|---|

In the end I don't want to put much time into something hopefully no users will ever have to see, so I'll leave it like that. If anyone prefers the � they can always go back hehe

 --- 
| a |
 --- 

[jnvsor]

Never thought I'd be closing this one without a mountain of work. Glad to get it off the docket :)