How does sixel scrolling work?

[j4james]

Having seen the results of the scrolling test, I've realised now that I don't actually know how the VT340 determines when and how much it should scroll. Clearly it's possible to have an image that fills the entire bottom line without scrolling, but there are also cases in some of the other tests where an extra line has been scrolled despite the image fitting within the previous line.

So what I've done is create a little script that generates images using a range of aspect ratios, and a range of sizes, and which overflow the bottom of the screen in varying amounts. For each test case, the script indicates how much the image has overflowed, and how much it has scrolled, e.g. something like this:

What I'm hoping to find is which combination of values causes the screen to scroll more than expected (i.e. the scrolled value is greater than the overflow count). I have a theory that it only occurs when the aspect ratio is 4 or more (i.e. when the expanded sixel height is greater than the line height).

Here's the script: https://gist.github.com/j4james/a3eb04b6c3fb6fa8f76e3c3bc5ac223b

Whenever you have a chance I'd appreciate you giving it a run and letting me know the results. Just the cases that scroll more than expected, and just something basic like 5/120/3/4 for 5:1 aspect ratio, 120px height, overflow of 3, but scrolled 4.

I've tried to be selective in the range of values tested, so it's not too much hassle, but if it's not obvious from the results what is going on, feel free to expand the test with other values.

[j4james]

By the way, I have a couple of theories for how it might work, but my best guess for the special cases is this:

AR	Height	Overflow	Scroll
4:1	216	5	6
5:1	120	3	4
6:1	360	9	10
15:1	360	9	10

It wouldn't surprise me at all if that is incorrect though.

[hackerb9]

Results were pretty close:

AR	Height	Overflow	Scroll
4:1	240	6	7
5:1	120	3	4
6:1	360	9	10
15:1	360	9	10

I added your script to version control as testscroll.sh. I made a couple changes. First is that it now prompts the user for the scrolled amount. Second, if overflow and scrolled amount differ, it adds the results to a table in testscroll.md.

[j4james]

These results align with what would have been my second guess: if you're about to write out a sixel line that isn't going to fit, you calculate the number of rows required using (imageHeight/lineHeight)+1 rather than ceil(imageHeight/lineHeight) (which would have been more "correct"). This results in an extra row when the image height is a multiple of 20 (the line height).

But note that this can only occur when the aspect ratio is 4:1 or above. At lower aspect ratios the sixel height is such that it would never be overflowing if it was also an exact multiple of the line height.

[hackerb9]

Interesting. It sounds like the way the VT340 does it is not something we'd want to recommend that sixel implementations actually copy (unless they're going for strict VT340 compatibility).

I'm making a list of these deficiencies I'm finding in the VT340 firmware with a vague notion that someday someone, perhaps me, will patch the firmware for the VT340 so it doesn't have these rough spots.

(Another one that's mildly needling at me is way the VT340 usually leaves the text cursor overlapping the graphics, but not always, so I can't just presume I need a newline).

[j4james]

It sounds like the way the VT340 does it is not something we'd want to recommend that sixel implementations actually copy

Yeah, I was thinking about that too. I don't think it's a big deal either way, because it only effects the higher aspect ratios that are unlikely to be used (and I don't think any of the open source terminals support aspect ratios at all). But personally I'd like my implementation to match the VT340 as closely as possible, as a long as it's not going to have any negative effects on real-world apps (which I don't think is a problem here).

Another one that's mildly needling at me is way the VT340 usually leaves the text cursor overlapping the graphics, but not always, so I can't just presume I need a newline

I've been meaning to discuss this with you, because I've seen you mention that before, and I very much disagree. This behavior is essential for rendering images on the bottom of the screen (or margin) without triggering a scroll. And if you don't want the cursor to overlap the bottom of the image, make sure you don't end with a DECGNL, but follow the image with an LF, VT or FF. I think that should guarantee the cursor is immediately below (at least for 1:1 AR).

If you do things the XTerm way, then you're going to end up with the screen scrolling by default. So that forced them to introduce a new mode to try and work around the problem, which weirdly places the cursor to the right of the image (which to be fair that was copied from RLogin). But that doesn't work if you're rendering something in the bottom right corner, which still triggers a scroll (at least on XTerm). If they'd just matched the original VT340 behavior, none of these half-baked hacks would be necessary.

[hackerb9]

Clearly, you've thought about this more than I have. Now that you state it, it's pretty obvious that leaving off the final graphics newline and adding a text newline afterward solves the problem.

[AutumnMeowMeow]

Hi! This is Autumn from my new account. (It's time to retire the old one -- transition stuff).

I've been using DECSDM to draw into the bottom row recently: https://gitlab.com/klamonte/jexer/-/issues/91

I notice in the README the discussion on the status line: https://github.com/hackerb9/vt340test#vt340-screen-resolution

Even if one wanted to muck with that, the status line ignores sixel commands, so it is still not possible to use it for graphics.

So even with DECSDM enabled, the bottom edge of the graphics screen is the row above the status line?

On the reference to DECSSDT: https://vt100.net/docs/vt510-rm/DECSSDT.html - does that work on the VT340 this way? It says level 4 required, so maybe not. I'm confused by:

If you select no status line (Ps = 0), then the terminal uses the line as an additional user window line to display data.

It seems like they are basically saying that the VT4xx/5xx have one more text row by default, but you can enable the VT3xx status line behavior with Ps = 1 or 2.

Anyway, thank you both so much on doing this project! :)

[j4james]

So even with DECSDM enabled, the bottom edge of the graphics screen is the row above the status line?

That would be my assumption. In the VT330/VT340 manual, DECSSDT 0 is documented as "No status line (The 25th line is blank)". It's like a completely separate display. You can control who gets to write to it, but you can't combine it with the main display, and you can't output graphics there.

At least that's my understanding - @hackerb9 might correct me on this.

If you select no status line (Ps = 0), then the terminal uses the line as an additional user window line to display data.

It seems like they are basically saying that the VT4xx/5xx have one more text row by default, but you can enable the VT3xx status line behavior with Ps = 1 or 2.

I wouldn't be surprised if that's a mistake in the documentation. At the very least the default is suspect, because different parts of the manual list different default values (sometimes 0, sometimes 1).

And if you look for videos of the VT5xx on YouTube, the most data lines I've ever seen used is 25 (not counting when they've switched to a 42 or 53 line mode). Below that you'll either see the status line and keyboard line, or just blank space. I've never seen any user-content down there. It's possible it could be configurable with DECSSDT 0, but it doesn't appear to be that way by default.

It says level 4 required

That level documentation is device specific. Typically a hardware terminal will only support two levels configurable with DECSCL - level 1 for VT100 compatibility, and their default level. So when they say level 4 required, they just mean it won't work in level 1. In the DEC STD 070 documentation it's listed as level 2 and above, although at level 2 it's an extension.

[hackerb9]

Hi, Autumn. Good to hear from you again.

I've been using DECSDM to draw into the bottom row recently: https://gitlab.com/klamonte/jexer/-/issues/91

So even with DECSDM enabled, the bottom edge of the graphics screen is the row above the status line?

I didn't know that DECSDM was useful for that. Nifty. Are you using it as a graphical status line?
I just double-checked and @j4james is, as usual, correct: the status line is never used for graphics on the VT340. Enabling DECSDM does not affect the size of the graphics screen, no matter the setting of DECSSDT on the VT340.

On the reference to DECSSDT: https://vt100.net/docs/vt510-rm/DECSSDT.html - does that work on the VT340 this way? It says level 4 required, so maybe not.

The VT340 interprets DECSSDT differently:

Ps	Status Line Selected
0	No status line (The 25th line is blank.)
1 (default)	Indicator status line
2	Host-writable status line

Anyway, thank you both so much on doing this project! :)

You're welcome. And I have to say Jexer is looking pretty nice these days; there's been a lot of progress since last I looked. Just for fun I tried the Jexer demos on my VT340. It didn't work right, but I had expected it to not work at all.