Ensure that non-ASCII characters are escaped when emitting to KORE #3489
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
radumereuta
approved these changes
Jul 5, 2023
@radumereuta The backends and the KORE syntax specification already assume that all non-ASCII characters are given as escape sequences, so this is really just a bug fix in the frontend, and there're no further implications from this change alone. (There is still a lot of work to be done re #3344 to actually get the semantics of these escape sequences handled correctly in the backends though). |
6 tasks
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of #3344
Previously, any non-ASCII character present in a
StringorByteswas erroneously emitted as-is to the KORE (*.korefiles are presumed ASCII-only). This PR ensures that such characters are escaped instead.That is, a string like
"!µƩ🌍\\"used to be emitted as\dv{SortString{}}("!µƩ🌍\\")}but will now instead be emitted as\dv{SortString{}}("!\xb5\u01a9\U0001f30d\\")