Provide encoding-related APIs for editor extensions

[haoqunjiang]

Currently there are only two fields in the TextEditorOptions API, a few other TextEditor-related APIs, and none of them is able to deal with the text buffer's encoding.
Comparing to Atom's TextEdit API, that is far from enough.

Most importantly, the API limitation makes it (seems) impossible for vscode-editorconfig to implement features like charset support, which is a crucial need for many people.

[alexdima]

👍 Also trimTrailingWhitespaces is missing

[kmpm]

#751 and #844 would probably be helped by this as well.
At least #751 would be solvable if working with BOM were better and my issues would completely go away if there was a way to, per editor extension, set the encoding and other stuff.
So 👍

[jrieken]

• set/get encoding
• set/get charset

[vysker]

I also need this for my extension, as I explain here. Did not realize this issue already existed.

Definitely want this feature!

[tomasiser]

I believe that full .editorconfig support is blocked (editorconfig/editorconfig-vscode#35) until the API for buffer encoding is added. Is there any chance this fact could speed this up?

[sophodoros]

This feature was first requested back in 2015 when VSCode was just emerging from the primordial ooze. VSCode has come a long way since then, but still no encoding support in the extension API?

Perhaps a full-blown feature that allows the encoding to be modified is too far down the priority list. However, it seems like it shouldn't be too difficult to add a simple read-only encoding property to TextDocument. After all, the encoding is already displayed in VSCode's status bar alongside the language (TypeScript, etc), which IS supplied via the TextDocument.languageId property.

Even if the value shown in the status bar is a guess as seems to be suggested in earlier comments, it would be better than nothing. As an extension writer, I'd rather use VSCode's best guess at the encoding than try to figure it out on my own.

[gingerbeardman]

The most sane way to get features like this, or column select, is to use a different editor. IMHO

[lygstate]

ping for it

[Tarrowren]

I looked at the code and I thought the coding was tied to the document, but it's actually tied to the editor.

I spent some time exposing it to the TextEditor API

/**
 * Represents an editor that is attached to a {@link TextDocument document}.
 */
export interface TextEditor {
  ...

  getEncoding(): Promise<string | undefined>;

  setEncoding(encoding: string, mode: "encode" | "decode"): Promise<void>;
}

and then in the json extension using

export async function activate(context: ExtensionContext) {
  ...

  const editer = window.activeTextEditor;
  if (editer) {
    outputChannel.appendLine("ok");
    outputChannel.appendLine((await editer.getEncoding()) ?? "null");
    await editer.setEncoding("utf16be", "encode");
    outputChannel.appendLine((await editer.getEncoding()) ?? "null");
  }
}

It worked! When opening the file he changed the document from utf8 to utf16be.

[Tarrowren]

I have moved it to TextEdit and it now works like this

toDispose.push(
  workspace.onWillSaveTextDocument((e) => {
    e.waitUntil(Promise.resolve([TextEdit.setEncoding("utf8")]));
  })
);

// or

window.activeTextEditor.edit((textEdit) => {
  textEdit.setEncoding("utf8");
});

But encoding needs to be added to the UndoRedoSnapshot, and I haven't implemented it yet.

And I didn't follow the rules of eslint(local/code-import-patterns) and couldn't submit the code, so I had to return to the previous method.

If I want to provide encoding in the TextDocument, I tried it and it requires a lot of code changes, there may be other simpler ways to do it.

I've spent a lot of time on this and now I'm going back to work😥.

https://github.com/Tarrowren/vscode/tree/textedit-encoding

[lygstate]

@takekazuomi do you create a MR for this?

[helmut-hackl]

Add editor encoding api

[gingerbeardman]

🙄

[leaumar]

I was getting ready to file a bug against editorconfig for this. That the microsoft-affiliated editor does not let editorconfig safeguard the charset of windows-specific files that cannot be in utf8 in my project is too ironic and not funny in 2024. When merge?

[gingerbeardman]

• we're now up to 230K+ issues
• this is issue 824
• from pretty much day one, during preview release
• that's before v1.0 even shipped
• and it has a PR

... yet we are still waiting? 🙃

[mozhuanzuojing]

确实，我也从20年等到今天

[mozhuanzuojing]

把这个问题在里程碑下面讨论一下