VT340's text cursor placement: glitch or not? #25
Comments
|
Hey @j4james. Perhaps you are misunderstanding what I was calling a "glitch". The behavior I'm describing does not occur when the image size is a multiple of the character cell height. For 90% of image heights, one can just splat a sixel file to a VT340, send a single newline, and then print text without it overwriting any of the visible image. Any integer multiple of the cell height fits into the "not a glitch" bin. I do get that you are saying that the way it works is a bit of a trick in that an image of height 20 does overlap with the text even though it does not seem to, it is just that the bottom four lines of pixels are transparent. However, the way that was implemented on the VT340 has flaws, as 10% of the heights require special treatment. Fortunately, that implementation was not part of the Sixel standard and I have never seen any software that relied on such odd behaviour. Modern implementations only need to make the same choice if they are trying to be strictly VT340-compatible. The question in my mind is: Could DEC have done better with the text cursor placement? I think so, but perhaps it would have been too resource intensive. DEC surely knew they could |
Yeah, but it sounded to me like you were proposing a change in the cursor positioning algorithm that would break apps that rely on the behavior I'm describing.
Even if you're right (and I'm not convinced you are), that doesn't make their choice a bug. You're just proposing a competing protocol. We've already got something like eight different cursor positioning algorithms - I really don't think that adding a ninth one is going to help interoperability. |
|
Ah, I think I understand you better. You thought I was proposing something that would affect programs that output images in integer multiples of the character height. It's actually the opposite. I think that the way that those images works is ideal: splat sixels, send new line, write text. The So, basically, what I'm hoping you'll have a guess at is: what was DEC thinking? In particular, you have pointed out that the text cursor is moved to the location of the top pixel of the sixel cursor. I think you are correct, but why did they do that? Wouldn't it have worked better to just use the bottom pixel? And, if they had thought they had a good solution, wouldn't they have included a description of how it worked in the programmer reference? I could be wrong, but the vagueness at that point in the manual makes me wonder if they already had plans for someday fixing the text cursor placement algorithm. Or was there some benefit beyond "it was quick and easy" to some image heights requiring two newlines instead of one? |
I don't think so, no. Consider the three test cases I presented above. For cases 1 and 2 (24px and 42px), if you aligned with the bottom pixel of the sixel cursor, the text cursor would either end up immediately below the visible image, or two lines below (depending on whether you terminated with a GNL or not). But for case 3 (60px), the text cursor would either be overlapping the image, or immediately below. So either you're going to get and unwanted gap for cases 1 and 2, or you're getting overlap in case 3. You can't win.
First off, the programmer reference isn't meant to be an instruction manual for terminal emulator developers. It's meant for programmers writing apps targetting the terminal, and they don't really need to know details like that. They've assumedly got a terminal in front of them to experiment with. Think of the difference between what you'd get in a programming language manual for typical users of the language, vs the information that compiler developers need. Also, I strongly suspect the manuals were written by support staff that often wouldn't have known all the intricate details of the terminal, which is how we ended up with weird documentation errors like the But all that aside, the bottom line for me is that if you're developing a terminal EMULATOR, you don't get to decide whether you like the way the terminal was built or not. You emulate it as best you can. If you aren't developing an emulator, well then you have no need for any of the information we've gathered here - just make up your own protocol. This is why I didn't want to waste any more time producing additional documentation. What's the point of us collecting all of this information about the VT340's behavior, and then telling developers to ignore it and do whatever they want? |
A fair question. I see this information as being useful for a broad range of people. Yes, of course it is necessary for TE developers who want to emulate a terminal as exactly as possible. And there is no doubt that your efforts have already had a huge impact on how VT340 emulation is done. Just look at how DECSDM has gone from being worse than useless to actually functioning in most emulators. More precise emulation is a primary goal for creating the errata and marginalia for the VT330/VT340 Programmer Reference. And, there are other terminal developers who do not want to create an "emulator", but are creating something new. As far as it is fun and enjoyable, they are willing to add VT340 compatibility. I do not know if you think that is something that is worthwhile, but I do. To that end, I think creating documentation that is easier to digest (and test!) is valuable. I am even willing to work with people to discuss how a VT340 might have behaved in a theoretical context. Consider the recent work on Contour where a developer went the extra mile to make the sixel rendering overlap in a way that was reminiscent of the VT340. That was impressive, and I definitely think there should be an option for doing that. But we should be clear that it was already beyond the realm of precise VT340 emulation: the character cell size was not 20 pixels high. I was speculating about how the VT340 would act if it had a different font size, but it is important to be clear to developers that there is no backwards compatibility issue if they extrapolate the overlap quirk differently than I did. And, finally, I see all this information as useful for people who simply want to have images available from the command line, scrolling within the text instead of in a different window. These people could be application developers or just UNIX users who use tools like img2sixel or ImageMagick's convert. These are people who are not interested in emulation for old applications, but just getting something done now. And I count myself among such people when I say, I think it is healthy for sixel to drop the quirks and limitations of the past implementations. I agree that sixel is a dead-end technology, but it is not dead, yet. There is nothing yet that can replace it and so it is still useful and still used. Creating a new protocol is a massive undertaking at which I would have little aptitude. And there are three small improvements to sixel that, while not implemented in the VT340, make a huge difference and still fit within the protocol specification: ① more colors, ② more resolution, and ③ alignment of graphics with text. ① I do not know where you stand on color palettes, but I think we agree that allowing more than 16 colors is not VT340 emulation. For me, that's okay: I do not need VT340 emulation and the sixel specs allow for more colors. Personally, I like that XTerm bothered to require a special escape sequence to use more colors in VT340 mode. That way I can write programs that use the new features but still work on a VT340. ② I think we agree that any terminal that offers more than 800x480 pixels is not restricting itself to precise VT340 emulation. Whether the resolution is increased by putting more pixels in each character cell or by increasing the number of cells, I think we also agree that modern terminals can have higher resolutions than a VT340 and still be considered as speaking the sixel protocol. ③ Alignment of graphics and text. This is a fairly new issue for me. You have helped to point out to me and others that the way the VT340 behaves differs from what DEC documented, particularly with how graphics and text overlap. I think that information will be extraordinarily valuable for people who want to have strict VT340 emulation. I hope we can agree that it is also useful to have terminals which exceed the VT340's capabilities. In particular, the VT340 has a way of drawing text (or graphics) beneath a sixel image which works almost every time, with a few undocumented idiosyncrasies. Of course, strict VT340 emulators will copy those exactly. But there is the question of how sixel terminals in general should behave. I know you are burnt out on this, but I'm hoping you can help me vet an idea for sixel terminals which are not aiming to be precise VT340 emulators. Specifically, I am wondering if it would make sense for sixel terminals to move the text cursor's row to the bottom-most opaque pixel. I believe, in 1:1 AR, it would produce nearly identical results to a VT340, with the exceptions being that those few "quirks" I mentioned would now be normalized. It would not have problems with causing scrolling for full screen images. Text sent after a new line would always be in the closest empty row. As I mentioned, this is not an idea for terminals that aim for strict VT340 compatibility. However, programs for terminals which behaved that way would work on my VT340, with only an occasional glitch where the text would cover up a few lines of pixels. (But that was already the case on the VT340.) So, if you're not too exhausted, I'd like to know if you can see any flaws. Is this "bottom-most opaque pixel" idea sound for a general (not VT340 specific) sixel rendering engine? (And, if you are too exhausted: I hope you rest well and take care.) |
Allowing more than 256 colors is largely an extension to the standard, which provides significant added value, and is a requirement for supporting modern sixel applications. Changing the cursor placement algorithm is none of those things. You're just introducing needless incompatibility. And though 256+ colors is not strictly backwards compatible, it is still technically possible to create a 256-color image that can fallback to a reasonable approximation on both the VT340 and VT240. I'm also hoping to get my implementation to auto adapt the color handling to match the source image, so it should be almost perfectly backwards compatible by default.
If this is achieved by increasing the number of cells, then you're not introducing any backwards compatibility issues at all, so I don't have a problem with that. And again that is providing significant added value.
Well you've yet to convince me of that. First off you're not proposing something that is backwards compatible. And second, I don't see what significant added value you're providing (like what awesome new application can someone develop with this change that they couldn't before). For me it just looks like a step backwards.
This would be a question to ask of the people that actually want to create a non-standard sixel engine. I don't care because I have no interest in doing that. But that brings me to the next problem with your proposal, which is convincing other people that it's a good idea. It's hard enough trying to convince people to follow a standard. But you think you can convince people they should ignore the standard, but also all agree to ignore it in the same way. And if you want an idea of how easy it is to get terminal devs to agree to anything, check out the "accepted specifications" folder of the terminal working group: Remember there are already around eight different ways in which terminals are implementing cursor positioning. And they probably all have users who have come to depend on their specific quirks. You need to convince them to break their own backwards compatibility, but not in order to be standard-compatible, but just to be incompatible in a different way. You better have a really good sales pitch. |
|
Hey guys. On the documentation comment I would like to share my view, since @hackerb9 earlier asked as well. I personally prefer precise but concise specification because they ought to be easy to understand and hard to misinterpret. Maybe that's a German stereotypical thing or just personal taste. But before dragging you into that, I am certainly trying to conform to the classic VT340 Sixels as much as I understand it without causing me a burnout. But I also want to do it simple for backwards compatibility reasons. Not because I believe in it as a future image protocol (obviously) :-). If at any time one comes up with such a doc, I would still read and evaluate it.
Pardon me? How come? It's the first time I read something like that. Any refs? @j4james ? |
@christianparpart I don't mean that there are eight modes in the standard or anything like that, so it's not something you need to worry about. I was just pointing out that nearly every terminal that implemented sixel has invented their own algorithm for positioning the cursor, so they're almost all incompatible with each other (and of course most aren't compatible with any of the DEC terminals either). |
Ah, the holy grail of documentation. ☺ Still, I expect it will be easier than trying to write in the verbose, sometimes confusing DEC style. By the way, my intention is still to first create a test suite that demonstrates the various ways a generic "sixel terminal" may differ from a VT340. Once I understand fully, I'll be able to write tersely.
Definitely not the future, but I find sixels useful in the present for solving everyday tasks. I guess I'm not your target audience, but a sixel terminal like Contour is exactly what I want, not an exact VT340-emulator. I appreciate that Contour does not change my screen's foreground and background text color every time I view an image and that Contour on a high resolution displays shows high resolution photos. While I think the VT340 is an amazing machine and well worth learning from, the sixel specification does not require rigidly following every quirk of a particular implementation. |
i've spent the past few days developing a Notcurses application which runs on the AMOLED i've built into my computer case. it shows two live images of the inside of my case, and displays temperatures/fan rpms/pump rpms etc. atop the places where they're being measured. for instance, the fans have their measured RPMs displayed atop the graphics. i'm not sure if this constitutes an "awesome new application", but it's definitely elicited praise from everyone i've shown it to. |
It's not about awesome. It's about something that's basic and has been broken on the VT340 and that I believe programmers have simply been ignoring because it is a silly flaw. But, it's also possible that I'm misunderstanding something. So, before I bore you with a long discussion, would you be willing to show me how you would write a sixel program that asks for a number and then draws a square of that size with the number printed as text underneath? Here's how I would do it: read -p "Size? " s
convert -size ${s}x${s} xc:blue sixel:-
echo ${s}x${s}
Fortunately, I am not trying to do that. Let's clear something up: I am not encouraging people to ignore the sixel standard. Or, if I am, I do not intend to be and would appreciate it if you pointed out where in the standard I have gone wrong. |
This is Python, but I'm sure it should be easy enough to convert to a shell script for someone that knows what they're doing. Doesn't depend on any third-party libraries. def convert(w,h,c):
import sys
sys.stdout.write('\033P9;1q#%d' % c)
sys.stdout.write(('!%d~-' % w) * (h//6))
if h%6: sys.stdout.write('!%d%s-' % (w,'?@BFN^'[h%6]))
sys.stdout.write('\033\\')
if (h-1)%20 < 18-(h-1)//20%3*2: sys.stdout.write('\n')
s = int(input('Size? '))
convert(s,s,1)
print('%dx%d' % (s,s))
By standard I mean the VT340 standard, which is defined by the reference device. I thought that's what we were discussing. As in 16 colors is standard for a VT340. An 80x24 cell display is standard for a VT340. And aligning the text cursor position with the top of the last sixel band is standard for a VT340. If you're suggesting terminals do something other than that (specifically regarding the cursor position), then you're encouraging them to ignore the VT340 standard, and they won't be compatible with terminals that are following that standard. If a terminal doesn't want to be compatible with the VT340, that's of course their choice to make, but then they're either all accepting the fact that they're not going to be interoperable with each other, or they're agreeing on using something else as a standard. I don't know which of those two options you're in favor of anymore, but I really don't care. I'm happy to collaborate with anyone that wants to follow the VT340 standard (or any of the other DEC devices for that matter). But if you want to invent your own protocol, or your own sixel variant, that's of zero interest to me. |
Perfect, so you are seeing the same peculiar behaviour as I am. That is what I am referring to as a "glitch". And, of course, any exact VT340 emulator should act in exactly that way. But what I am trying to figure out is why. It seemed that you thought it was a feature, not a bug, and I'd like to know more about that. |
|
I guess it depends what you want from the text cursor position. You seem to have a specific use case in mind, and think that anything that doesn't meet your needs exactly must be a bug. My only expectation is that it should be easily predictable, and should not force a viewport scroll when drawing as far as the bottom of the screen. The VT340 algorithm achieves that. It also works ideally for what I would consider a common use case - images that are exactly aligned to text boundaries. That's the bit that I consider a feature. But even if it didn't have that feature, I wouldn't consider it a bug. A bug is something that is completely inexplicable, like the background colors changing randomly when the screen scrolls, or random bits of the image being corrupted in the media copy command. A design choice that you personally don't like is not an indication that something is broken. |
Yes, that's definitely true. Moreover, I think it was a conscious decision by DEC to use an algorithm that works not only for images that are multiples of the text height, but for 90% of all possible heights, which is way more than 90% of actual use cases since it includes all powers of 2 and multiples of ten. It is an extremely useful feature. I was confused only by the purpose of the other 10%. Did it somehow make the VT340 better? I didn't see how. If I can read between the lines, it sounds like you're saying that this 10% behaviour was not to add a useful feature, but it is still explicable and thus not a bug. Would it be fair to say that you would explain the VT340's algorithm as necessitated by hardware limitations? That is: the more obvious choice of covering 100% of cases was simply not possible on 1987 hardware. Right? |
You really are overthinking this. Here's how I imagine their design meeting went: Dev A: Where do you think we should be put the text cursor when sixel mode is exited? They may have chosen the top rather than the bottom because it worked well for cell-aligned images, but it's equally possible they just flipped a coin. Nobody was sitting in that meeting thinking about analysing the pixels in the image to decide what would look best when dumping am image from the Linux command line. That's not the responsibility of the terminal. If that's something your app requires, you can write the code yourself. |
Heh, yeah, I could image that that's how the design meeting went. I had just wanted to check with you because DEC does have a history of designing what appears to humble folks like myself to be questionable choices that in retrospect were actually pretty clever. For example, modern sixel terminals were presuming that the end of sixel data should always have a newline appended, until you showed why DEC had very specifically not done that. And, yes, I am probably am overthinking it, but I'll have you know I was presuming it was a VMS DCL command prompt, not Linux. ☺
That's just my imagining, but I find it slightly more plausible than DEC just flipping a coin. But, either way, the only redeeming feature of this quirky behaviour would be that it is exactly how a VT340 operates. Yes? |
That's the only thing that matters as far as a terminal emulator is concerned. But as I've mentioned before, it's predictable and easily implemented, which is a benefit for both app devs and terminal devs. The biggest advantage in using the DEC algorithm, though, is that you're guaranteed interoperability with both the original hardware terminal, and any terminal emulator that takes their job seriously. If you invent your own protocol, there's no guarantee you'll ever be compatible with anyone but yourself. |
|
Putting emulators aside, it sounds like we agree that this peculiar behaviour had no purpose on the VT340. Good. While I still don't see some of the things you're saying, my reservations are based on another presumption which may be false. To clarify, would you be please show me a program which displays a sixel file on a VT340 and then prints the filename on the line beneath it? For example, I would do something like this as a shell script: cat "$1"
echo "$1" |
|
If the sixel file is appropriately formatted (as demonstrated at the top of this issue), then a |
Great! I was hoping we'd agree that
I mean the first empty text row underneath the image.
You're talking about newlines at the end of a text/sixel file, I presume. That's a good point. Fortunately, I believe it's fairly trivial to workaround. Doing so also removes the restriction that the sixel file must be "appropriately formatted" ahead of time. Here's a revised version of my script that removes any GNL at the end of the file and adds a single text NL before the filename. #!/bin/bash
awk 'BEGIN {RS="\x1B\\\\";}
{gsub("-$|\n",""); printf $0;}
END {print "\x1B\\"; print FILENAME}' "$@"Would this still work just fine? If not, can you please show me a program that does? |
I have no idea how the MediaCopy command handles different region sizes.
By "empty", I assume you mean it hasn't been overwritten with sixel content? But does the area cleared by the background select count as empty or not? Are transparent pixels empty? What if they're drawn with the background color? Or a color that matches the background that was already there? What if all of the sixel output was transparent but enough GNLs have been emitted that the start of the sixel output is now offscreen? In case it's not obvious, I'm not actually expecting you to answer these questions, because there isn't any one "right" answer. It depends entirely on the needs of the application that is producing the sixel output.
That's the most common example, but there are a whole lot of control characters that might appear in a text file that would alter where the cursor finally ends up. If you're happy to put certain restrictions on the file content then it's a simple problem to solve, but if you want to be able to handle every possible scenario then things get a lot more complicated. Same applies to sixel. |
Well, that's no good. I'll have to document it better when I get home to my VT340. My recollection is that VT340 MediaCopy can output slightly different formats, based on toggles in the VT340 settings. I was thinking of the Level 2 1:1 AR Sixel output that my mediacopy.sh script requests. The output is reasonably similar to img2sixel and ImageMagick's convert, so you can use output from those as a stand-in. (In 16-color mode, of course).
Thank you for clarifying as I had been taking your questions literally. It sounds like you're saying this are many "right" answers for displaying a sixel file on a VT340 and putting text underneath. Would you be willing to show just one answer that you'd consider correct? |
Scroll up to the top of this issue where you quoted my comment from the contour repo. That's the only format I use. |
What about interoperability with typical sixel files, in particular, how does one display files created by a VT340, on a VT340? My goal is to understand the implications of your description of the occasional glitches in the VT340 text cursor placement as "not a bug". I am having trouble seeing how it could work if the glitches are not supposed to be ignored by application programmers. It should be simple to show a sixel file and print the filename underneath. I get that there may be oddball files which display incorrectly, but I'm not concerned with those at the moment, just the ones created by a VT340. If you would please share an example program it would help immensely. |
Let's take a collaborative approach. If you can calculate the cursor position using the algorithm you think DEC was trying to write (let's call that I've written some example code below which includes an implementation of algorithm # This is the algorithm that you think DEC was trying to write.
def cursor_pos_using_algorithm_a(sixel_data):
pass # TODO
# This is the algorithm that DEC actually produced.
def cursor_pos_using_algorithm_b(sixel_data):
offset_of_last_sixel = gnl_count(sixel_data) * aspect_ratio(sixel_data) * 6
return offset_of_last_sixel // 20
# Helper function for counting the GNLs in an image.
def gnl_count(sixel_data):
return sixel_data.count('-')
# Helper function for determining the aspect ratio of an image.
# This is not a complete implementation - just a POC.
def aspect_ratio(sixel_data):
import re
m = re.search('"([0-9]+);([0-9]+)', sixel_data)
return int(m.group(1)) // int(m.group(2)) if m else 2
# This is how many line feeds you need to add to the image.
def linefeeds_required(sixel_data):
return cursor_pos_using_algorithm_a(sixel_data) - cursor_pos_using_algorithm_b(sixel_data) |
|
We can definitely work collaboratively on rigorously defining Algorithm A at some point in the future, but I'm actually trying to determine something more fundamental. First, let's make a distinction between algorithms meant to be implemented in the terminal and those — let's call them "prime" — which are meant to be used in an application program. If a terminal implements Algorithm B, then it seems there is no simple way — let's call it Algorithm B-prime — for an application to predict the text cursor position 100% of the time. There is, however, a simple way that works almost 100% of the time, Algorithm A-prime: “After sending a sixel image, a text new line will advance the text cursor to the next empty row.” @j4james, I've been asking you for Algorithm B-prime, the 100% perfect algorithm for an app that is designed to run on a terminal which uses Algorithm B, because I tried and failed to find an easy implementation of B-prime. Please correct me if I'm wrong, but I believe B-prime requires parsing the sixel data. For that reason, it seems highly improbable that any app would actually attempt it. Why bother parsing sixels when Algorithm A-prime is so much simpler and almost never goes wrong? In fact, as you indicated earlier, for images that are a multiple of the character cell height, Algorithm A-prime never fails. Before I conjecture further, let's double check some of my presumptions.
Let me know if you agree with these points. I expect most of the above will be non-controversial and we can move forward from there. |
I'm not interested in Algorithm A in the least, but you keep asking me to write code for you, while you haven't made any attempt to do so yourself. I'm trying to get you to put something down on paper that backs up all your claims. So far all you've done is define your algorithm in terms of its fantasy achievements. "Algorithm A is defined as the implementation in a terminal that would make Algorithm A-prime work correctly", "Algorithm A is not difficult to implement", "Algorithm A has no glitches for any height image". These points can't be disputed because there's isn't a concrete implementation to critique. Algorithm A can do whatever you want it to do because it only exists in your imagination. At the same time you always refer to Algorithm B in the negative while pretending that's an objective assessment. If it differs from your fantasy algorithm then that difference is referred to as a glitch, like it must have been a mistake in the code. And yet you can't actually point out a bug anywhere in the algorithm. It's essentially only two lines - it's shouldn't be that hard to spot. I don't know what exactly it is you're trying to achieve here, but I am certain I don't want to be involved any longer. If you're trying to prove the VT340's algorithm is broken, you're wasting your time, but I'm not going to argue that point - you're welcome to keep believing that if you wish. If you're just trying to invent a new protocol, I'm also not interested - I think the best place to discuss that topic would probably be with the "Good Image Protocol" folks. |
|
@j4james I am sorry you don't want to be involved in this issue anymore, but I do thank you for your help and apologize for frustrating you. I had hoped to focus only on what I know you to be an expert on: the DEC VT-340. It was my mistake thinking you'd want to consider a hypothetical Algorithm A which the VT-340's Algorithm B appeared to be approximating. I'm sorry about that. Fortunately, the information I hoped to learn from you didn't require talking about Algorithm A. I figured that you would know if there was a perfect Algorithm B-prime which DEC wanted application developers to use when displaying on actual VT-340 hardware. (I'd already searched and been stumped.) I'm willing to be wrong -- I often am -- but from our inability to come up with any plausible B-prime, I think I can safely conclude, at least for now, that DEC actually did intend for programmers to be doing the natural thing: to have text show up on the next free line, just print it after sending a sixel image. The VT-340 won't always render it perfectly, but it's close enough. Do let me know if you ever have different insight into what DEC wanted programmers to do. Until that time, I'll close this issue. Thanks! |
From @j4james:
“The VT340's behavior may not be to your liking, but it's not a bug or a glitch. It actually serves a very useful purpose when you're working with images that you want to align on text boundaries.
For example, if you want to output an image that is exactly one text cell in height, you need to output 4 sixel bands (24px height) with the last 4 pixels being transparent. If you end that with a GNL, the text cursor is exactly below the image.
Similarly if you want an image two text cells in height, it's 7 sixel bands (42px height) with the last 2 pixels transparent. And for an image three text cells in height, it's exactly 6 sixel bands (60px height).
In every case the text cursor ends up exactly below the visible portion of the image, overwriting only the transparent pixels. And if you leave off the final GNL, you can get the text cursor on the bottom row of the image instead.
Example test cases:
Please do not try to "fix" sixel. If you don't like it, and feel the need to change the standard, rather put that effort into a new graphics protocol instead.”
Originally posted by @j4james in contour-terminal/contour#825 (comment)
The text was updated successfully, but these errors were encountered: