Encoding: ISO-2022-JP encoder "SO/SI ESC" test #26158
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The test dates from 2014; expected behaviour changed in 2016
|
Thank you for contributing this! It seems you may have uncovered several issues by writing this test, which I hope to sort through in whatwg/url#557 before taking action here. |
Always happy to cause trouble. ;) I'd just finished writing an ISO-2022-JP encoder and was using Firefox to validate my results (since the tests are not very portable). Imagine my surprise when I discovered Firefox was wrong despite passing every test. I didn't expect to open a whole can of worms, though. |
|
@JKingweb would you mind duplicating the test you modified and add a non-ASCII code point to the input and the expectations? That will show some more variety in browsers. (I'm also happy to push such a test if that's okay with you.) I'll (also) work on the changes to the URL Standard that are needed. |
|
@hsivonen I went ahead and pushed a bunch more tests. If these look okay to you I suggest we go ahead and land these and then I'll work on the specification patches next. |
annevk
added a commit
to whatwg/encoding
that referenced
this pull request
Oct 21, 2020
This avoids the need for "prepend" in the "process" algorithm as is needed to fix #235. Additionally, this commit adds two clarifying asserts to the "process" algorithm documenting what error modes can be in effect when. Related tests: web-platform-tests/wpt#26158.
hsivonen
approved these changes
Oct 21, 2020
annevk
added a commit
to whatwg/url
that referenced
this pull request
Oct 21, 2020
Since the ISO-2022-JP encoder is stateful, percent-encoding needs to hold onto an instance of the encoder and manually perform error handling. This also requires the input to be the full string rather than individual code points as otherwise the callers of percent-encoding would need to be aware of this too. (As UTF-8 encoding cannot fail this problem does not affect those endpoints.) Depends on this Encoding PR: whatwg/encoding#238. Tests: web-platform-tests/wpt#26158. Fixes #557.
3 tasks
|
More bugs:
PRs: |
3 tasks
annevk
added a commit
to whatwg/url
that referenced
this pull request
Oct 28, 2020
Since the ISO-2022-JP encoder is stateful, percent-encoding needs to hold onto an instance of the encoder and manually perform error handling. This also requires the input to be the full string rather than individual code points as otherwise the callers of percent-encoding would need to be aware of this too. (As UTF-8 encoding cannot fail this problem does not affect those endpoints.) Builds on this Encoding PR: whatwg/encoding#238. Tests: web-platform-tests/wpt#26158. Fixes #557.
watilde
added a commit
to watilde/node
that referenced
this pull request
Oct 28, 2020
annevk
added a commit
to whatwg/url
that referenced
this pull request
Oct 28, 2020
Since the ISO-2022-JP encoder is stateful, percent-encoding needs to hold onto an instance of the encoder and manually perform error handling. This also requires the input to be the full string rather than individual code points as otherwise the callers of percent-encoding would need to be aware of this too. (As UTF-8 encoding cannot fail this problem does not affect those endpoints.) Builds on this Encoding PR: whatwg/encoding#238. Tests: web-platform-tests/wpt#26158. Fixes #557.
watilde
added a commit
to nodejs/node
that referenced
this pull request
Oct 29, 2020
Refs: web-platform-tests/wpt#25988 Refs: web-platform-tests/wpt#26158 PR-URL: #35794 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
annevk
added a commit
to whatwg/url
that referenced
this pull request
Nov 2, 2020
Since the ISO-2022-JP encoder is stateful, percent-encoding needs to hold onto an instance of the encoder and manually perform error handling. This also requires the input to be the full string rather than individual code points as otherwise the callers of percent-encoding would need to be aware of this too. (As UTF-8 encoding cannot fail this problem does not affect those endpoints.) Builds on this Encoding PR: whatwg/encoding#238. Tests: web-platform-tests/wpt#26158 and web-platform-tests/wpt#26317. Fixes #557.
annevk
added a commit
to whatwg/url
that referenced
this pull request
Nov 2, 2020
Since the ISO-2022-JP encoder is stateful, percent-encoding needs to hold onto an instance of the encoder and manually perform error handling. This also requires the input to be the full string rather than individual code points as otherwise the callers of percent-encoding would need to be aware of this too. (As UTF-8 encoding cannot fail this problem does not affect those endpoints.) Builds on this Encoding PR: whatwg/encoding#238. Tests: web-platform-tests/wpt#26158 and web-platform-tests/wpt#26317. Fixes #557.
targos
pushed a commit
to nodejs/node
that referenced
this pull request
Nov 3, 2020
Refs: web-platform-tests/wpt#25988 Refs: web-platform-tests/wpt#26158 PR-URL: #35794 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
BethGriggs
pushed a commit
to nodejs/node
that referenced
this pull request
Dec 8, 2020
Refs: web-platform-tests/wpt#25988 Refs: web-platform-tests/wpt#26158 PR-URL: #35794 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
BethGriggs
pushed a commit
to nodejs/node
that referenced
this pull request
Dec 10, 2020
Refs: web-platform-tests/wpt#25988 Refs: web-platform-tests/wpt#26158 PR-URL: #35794 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
BethGriggs
pushed a commit
to nodejs/node
that referenced
this pull request
Dec 15, 2020
Refs: web-platform-tests/wpt#25988 Refs: web-platform-tests/wpt#26158 PR-URL: #35794 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The test dates from 2014; expected behaviour changed in 2016 with whatwg/encoding@f9540e5.