✅ We are removing SEU colour support support in 3.0.0 ✅

[worksofliam]

By the time Code for IBM i 3.0.0 releases, we are planning on removing SEU colour support.

This is what it looked like.

We are still sometime away from 3.0.0. If you take a look at our milestones, there is a planned 2.11.0 release which still has some time to go. 3.0.0 will likely be sometime this year (2024). It also means that any new issues that arise with colour support will be closed, and going forward suggest disabling SEU colour support in the VS Code settings.

This means when you open up source code in Visual Studio Code, instead of the colours appearing, you will instead see the character equivalent of the byte.

You can track the issue here: #2056

Why are we doing this?

As time has gone by, SEU is really one of the only editors that has support for a colour system like this. We are not trying to be like SEU, so therefore we don't feel that it is worth the time maintaining such a system in Code for IBM i.

To fit in better with the general development ecosystem, as well as the move to UTF8, colour bytes (which is typically ebcdic) just don't fit in well.

As @JonFP pointed out:

There never was any "support" - it was just an accidental by-product of the way the 5250 datastream works

What are your options?

1. You can continue to use an older version of Code for IBM i that will have SEU colour support.
2. You can (and should) look into removing SEU colour bytes from your source code.

Leave feedback

If you are affected by this change, you are welcome to leave a comment here. We also welcome comments about how users can migrate away from SEU colors (e.g. stripped out the bytes to remove them).

[chrjorgensen]

You can easily remove SEU colour codes from source members by creating and running a SQL procedure - for more information, see this comment.

[JonFP]

I would suggest not to use wording like "that has support for a colour system" in any announcement. There never was any "support" - it was just an accidental by-product of the way the 5250 datastream works. Like many others I had been working with SEU for eons before I even knew it could be done!

In other words just tell 'em it doesn't work!

[thomprl]

So if you open a source with this hex code color will it still compile or have to be removed? And if it still compiles will it save the hex code and if someone opens in SEU still be there?

[chrjorgensen]

@thomprl I know two of the hexcodes used for SEU colouring is converted into the same UTF-8 value - so when the member is saved, the colour codes will be the same, i.e. two colours made be changed into only one colour.

This should not affect compilation, though, since colour codes were only used in comment lines.

[starbuck5250]

To clarify a little, columns the compiler ignores everything in any comment area, including columns 1-5 of fixed RPG, and right-hand comments. I have plenty of code where someone put an attribute byte in column 1 of say an F-spec. Ugh.

On a marketing note, I 100% agree with Jon. These aren't colour codes, they're 5250 attribute bytes. They were never WYSIWYG. Good riddance.

[jkyeung]

First, I want to reinforce Jon and Buck: To call it "supported" in SEU is... well, bluntly, I consider that to be a complete misunderstanding.

Even among experienced SEU users who deliberately used attribute bytes, a disturbing percentage of them didn't understand what they were doing.

Perhaps the clearest illustration of how accidental this "feature" is: Let's say you put an attribute byte in column 1 of the source, like Buck mentioned. If it's an RPGLE member, then by default SEU opens it horizontally scrolled such that column 6 is the leftmost visible column on the screen. The vast majority of SEU users do not bother to scroll those first five columns into view, they just leave it like that (which is, after all, what the SEU designers were intending and expecting). In this circumstance, any attribute bytes in the first five columns are not displayed, and thus are not in the 5250 data stream, and thus have no effect. Those lines will simply be the default green! (Well, characters on that line will be displayed with default attributes up until an on-screen attribute byte is encountered, which would still affect subsequent characters.)

Moreover, users who do not scroll those first five columns into view are prone to unknowingly propagating any junk that's in there (whether it's attribute bytes, coder initials, revision ID, or whatever), via copying those lines in the course of doing maintenance work on the member. (This is perhaps my least favorite quirk of SEU, and why putting stuff in columns 1 to 5 is such a horrible practice.)

[jkyeung]

Despite the strong consensus that VS Code should not attempt to emulate 5250 behavior when encountering attribute bytes, there is still a sense in which I believe they need to be "supported", which is: Since they are a reality in existing codebases, there should be some sensible defined handling of them.

Some issues:

1. They need to be reliably detected. They are not "colour codes" and they are not EBCDIC, per se. They are a specific range of hex values (x'20' through x'3F') and they need to be intercepted before any attempt to translate them to any text encoding, EBCDIC or otherwise.
2. We need to decide what to do with them when we encounter them. The options that come to mind are
   1. Convert them to blanks upon reading, which is destructive but simple. This action removes the need for any other decision-making or implementation. No special handling is required to write the blanks upon save. (I hope it goes without saying that closing the buffer without saving should leave the original source untouched, attribute bytes and all.)
   2. Leave them intact and write them back out unchanged upon save. I think this is how it's supposed to be working now, with 5250 color emulation disabled, but it's not working quite right. In particular, x'25' and x'3F' do not make the round trip. (x'25' becomes a linefeed and x'3F' becomes '\Z'.)

So, we're already seemingly close to supporting (ii). I hope it is not too hard to fix.

Incidentally, this procedure is also not quite right. I don't know why some of the possible attribute byte values were left out of the TRANSLATE operation.

[jkyeung]

Smartly enough, VS Code doesn't support blinking text.

I know. That is what @chrjorgensen said. I contend that (1) you could have simply done what ACS did and not blink those, and (2) you seem to have attempted to implement nondisplay (even though he said you didn't), which to me is no better or worse than attempting to implement blinking. Why not just be complete, if you're going to do it at all?

The bigger question right now, which I am posing to the community:

Is the current behavior when 5250 color emulation is disabled sufficient?

In my opinion, it isn't. I strongly believe that every single byte of source code on the IBM i should be able to survive a round trip (as though untouched), with no bytes added, changed, or removed unless explicitly done so by the user. The extension could offer facilities to make it easy to replace them with blanks, but that should either be explicitly invoked by the user, or if automated, preceded by a warning notification and an option to abort or open in read-only mode.

[worksofliam]

I do believe in Code for IBM i, we are not going to do anything with those bytes. We are just going to let VS Code render them as it used to (it used to display SUB, SYS, etc, characters in the place of them) and it can be up to the user to determine what to do with them.

Though, @chrjorgensen does provide a nice SQL routine to remove them.

[jkyeung]

I do believe in Code for IBM i, we are not going to do anything with those bytes. We are just going to let VS Code render them as it used to (it used to display SUB, SYS, etc, characters in the place of them) and it can be up to the user to determine what to do with them.

I don't know that that is sufficient. As I reported, there are at least 2 codes that do not survive the round trip. From my point of view, this does not satisfy the very basic principle that if you open a member in VS Code, and do nothing other than save it right back, the contents should be exactly the same as before, byte for byte. (I can see good arguments for updating the timestamp or leaving the timestamp alone; either would be fine with me.)

At the very, very minimum, the extension should give the user a warning along the lines of "sorry, we found 5250 attribute bytes in this code and we cannot guarantee that they will be written back correctly; do you want to proceed anyway, or abort?".

Though, @chrjorgensen does provide a nice SQL routine to remove them.

Except that it doesn't remove all of them. It should remove all of them.

[JDubbTX]

Just chiming in with my 2 cents.

• I actually liked that Code4i handled the color bytes, even if the implementation was incomplete. I'm not going to shed a tear over it, but it gave me a bullet point when talking to RDi folks in the category of here is something that RDi doesn't do. It was also kind of cool to see when opening up 30 year old source members something that someone long since retired set up just to make their comments a bit more impactful.
• I agree that the best alternative is to leave them alone. Don't alter them, full stop. The goofy representation that shows up at least makes you aware and gives you the choice of how to handle them, either use fancy sql to strip them out or just deal with it. It's really not anything to lose sleep over.

[jkyeung]

• I agree that the best alternative is to leave them alone. Don't alter them, full stop.

See my new issue.

[BrianGodsend]

As @JDubbTX points out, if this feature goes away, so be it. I personally find that folks that color tend to over-color to the point of dramatically reducing readability. However, every once in awhile, coloring is used sparingly and has a very impactful effect.

That said, as others have pointed out, simply opening and re-saving a member should not change 1-byte of the source code ... including the 5250 display attributes.

So, can you please clarify what removal of support really means? Are the 5250 display attributes going to be stripped from source? If not, will the full range of all display attribute codes (x20 - x3F and xA0-xBF) be preserved? If the codes are not being stripped and not being preserved, will you add a warning about codes being altered?