StringIO mingles content with Unicode characters

[piotrmurach]

The tty-progressbar test suite passes fine on MRI from 2.0.0 to 3.0.0, as well as jruby-9.2.13.0 and jruby-head but fails on truffleruby-head. In particular, these two tests fail. Looking at the output comparison, this is the bit that differs:

"\e8\e[1G└── [===  ] two\n\e7\e[3A\e[1G┌" 

Gets to be gobbled up and shortened to:

"\e8\e[1G└─\e7\e[3A\e[1G┌"

The problem seems to be the StringIO class. It mingles, e.i. cuts chunk of content with Unicode chars like "└─ " making output comparison fail. Replacing the border characters with ASCII "|__ " equivalent gets the tests to pass again.

Equally, replacing StringIO with an OutputIO mock class fixes the issue completely:

class OutputIO
  def initialize(content = "")
    @content = content
  end

  def print(string)
    @content << string
  end

  def read
    @content
  end

  def flush; end

  def rewind; end

  def tty?
    true
  end
end

The print, read, flush and rewind are the only methods invoked. I'm not sure whether it is the reading or the writing to StringIO instance that causes the issue or maybe the rewind call is at fault. It seems that the first read is always fine, but subsequent printing and rewinding and the reading again mingles the content.

[eregon]

@aardvark179 Could you look at this since you were recently looking at StringIO?

[aardvark179]

I thought we had solved this, but I think last time I looked I'd run the tests with OutputIO. I found the cause of the problem, which was that StringIO was updating the position incorrectly after reading a string containing multi-byte characters. I've got a patch for this in our internal CI now.

[piotrmurach]

This is still an active bug. The recent run of tty-spinner test suite on truffleruby-head in GitHub CI confirms it.

[aardvark179]

You were about an hour too early for the merge (be96ee5) I'm not sure how long it takes truffleruby-head to be updated in CI.

[eregon]

truffleruby-head is updated now: https://github.com/ruby/truffleruby-dev-builder