-
Notifications
You must be signed in to change notification settings - Fork 186
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: StringIO#write transcodes strings with a different encoding #2927
fix: StringIO#write transcodes strings with a different encoding #2927
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the PR!
I'll address my own comments for convenience.
938aecb
to
a27f06b
Compare
Remaining test failures don't seem related to this PR. |
a27f06b
to
2d1822c
Compare
I found another edge case, revealed by this failing CRuby test:
It tries to write a UTF-8 String with non-ASCII characters into a StringIO which was I simplified it to:
So CRuby seems to magically change the StringIO encoding for this case. Waow. And I wondered if the BINARY case is handled similarly but no, that doesn't change the StringIO encoding:
So many hacks and inconsistencies (and maybe some bugs) for StringIO encoding :( |
The encoding logic for StringIO#write upstream seems to be https://github.com/ruby/stringio/blob/009896b97378311ecaa88c8fa9c32acd7656648a/ext/stringio/stringio.c#L1498-L1506. So special case both BINARY and US-ASCII for the String encoding, in case conversation fail, move on without converting. Yeah, that's a messy hack, but we can copy it. |
Thank you for the extra work needed to complete this work, @eregon. |
This is a potential fix for #2839. I've based this approach on the transcoding logic in
IO#write
.I'm not sure I've placed the new test in the best place; or if this fix is the best option. This is my first PR to TR! Feedback welcome. ❤️