Unmarshaled hash with non-ascii keys is corrupted

[deepj]

This issue is similar to jruby/jruby#5523 reported by @larskanis. But it behaves a bit differently in TruffleRuby. There was the same issue with arrays already fixed in TruffleRuby #1530

The used code is from @larskanis

TruffleRuby 1.0.0-RC13

irb(main):001:0> h={:€a=>nil, :€c=>nil, :€h=>nil}
=> {:€a=>nil, :€c=>nil, :€h=>nil}
irb(main):002:0> Marshal.load(Marshal.dump(h))
=> {:"€a"=>nil, :"\xE2\x82\xACc"=>nil, :"\xE2\x82\xACh"=>nil}
irb(main):003:0> Marshal.load(Marshal.dump(h)) == h
=> false

Ruby 2.5.3

irb(main):001:0> h={:€a=>nil, :€c=>nil, :€h=>nil}
=> {:€a=>nil, :€c=>nil, :€h=>nil}
irb(main):002:0> Marshal.load(Marshal.dump(h))
=> {:€a=>nil, :€c=>nil, :€h=>nil}
irb(main):003:0> Marshal.load(Marshal.dump(h)) == h
=> true

[dentarg]

Seems to happen when using truffleruby-head-ubuntu-18.04 with Addressable (build ran 2 days ago): https://github.com/sporkmonger/addressable/runs/2977469877?check_suite_focus=true#step:7:87

The Marshal dump is from 2012: sporkmonger/addressable@1eb715d

truffleruby-21.1.0-ubuntu-18.04 works fine though: https://github.com/sporkmonger/addressable/runs/2988483622?check_suite_focus=true#step:5:13

[eregon]

Thanks for the report! We'll look into that.

[eregon]

The problem in the original example is that some of the Symbols get a BINARY encoding when reloaded:

v = {:€a=>nil, :€c=>nil, :€h=>nil}
p v
r = Marshal.load(Marshal.dump(v))
p r
p r == v

p v.keys.map(&:encoding)
p r.keys.map(&:encoding)

{:"€a"=>nil, :"€c"=>nil, :"€h"=>nil}
{:"€a"=>nil, :"\xE2\x82\xACc"=>nil, :"\xE2\x82\xACh"=>nil}
false
[#<Encoding:UTF-8>, #<Encoding:UTF-8>, #<Encoding:UTF-8>]
[#<Encoding:UTF-8>, #<Encoding:ASCII-8BIT>, #<Encoding:ASCII-8BIT>]

21.1 has that issue too.

[eregon]

@dentarg I tried to reproduce but the addressable specs pass for me locally.
Unclear why they failed in that CI run.