Style/Encoding: false doesn't always work for ruby 2.0

[jkingdon]

If you have a ruby file without a # encoding: utf-8 line, and with a non-ASCII character, rubocop may blow up with "Invalid byte sequence in us-ascii". To make it happen every time, set LC_ALL to LC_ALL=en_US.US-ASCII (that's for MacOS Mavericks, exact value may depend on your system).

One workaround is to set LC_ALL to something like en_US.UTF-8 (exact value may vary depending on your system). However, since ruby 2.0+ defaults to UTF-8, rubocop should do the same (at least when being run with ruby 2.0+) rather than rely on a system default character set (which turns into the ruby Encoding.default_external which is what rubocop seems to currently be getting when reading files).

[bbatsov]

@jonas054 Would you look into this?

[jonas054]

OK.

[jonas054]

What happens is that we get an exception from Parser::Source::Buffer#source= when we call it in RuboCop::ProcessedSource#parse. If there's no encoding comment in the file, Parser falls back on the encoding that's set in the parsed string, and that depends on the external encoding set in environment variables, e.g., LC_ALL.

@bbatsov @yujinakayama I have two questions.

1. Should RuboCop's behavior depend in which Ruby version it's running under? See discussion in Enable Style/Encoding for Ruby >= 2.0 #1304.
2. Should this be solved in RuboCop or in Parser?

BTW, I was able to reproduce the problem in Linux with export LC_ALL=C.

[yujinakayama]

1. Should RuboCop's behavior depend in which Ruby version it's running under?

I think it should not.

1. Should this be solved in RuboCop or in Parser?

I'm not completely sure, but in this case I'd prefer handling it in RuboCop.

• Parser is a versatile library and it should be flexible. I think fixing encoding in Parser is somewhat inflexible (and doing so may be a breaking change).
• We already have control on it. We just need to always pass a UTF-8 encoded string to Parser.

[jonas054]

@yujinakayama Thanks!