Operation macro corrupts snippets with UTF-8 characters

[grolljak]

Hi, I have a case where I need to include a special character ä inside documentation.

Plugins/Dependencies I am using:

id("org.springframework.boot") version "2.3.3.RELEASE"
id("org.asciidoctor.jvm.convert") version "3.2.0"
implementation("org.springframework.restdocs:spring-restdocs-mockmvc")
implementation("org.springframework.restdocs:spring-restdocs-asciidoctor")
asciidoctorExtensions("org.springframework.restdocs:spring-restdocs-asciidoctor")

Problematic part of generated snippet response-fields.adoc
|The zip type, one of [grossempfänger, normal, postfach, alte plz].

Then I am including this snippet in HTML documentation via:
include::{path}/response-fields.adoc[]
and also via:
operation::{name_of_the_snippet}[snippets='response-fields']

In the first case I get correct result generated in HTML:
The zip type, one of [grossempfänger, normal, postfach, alte plz].

In the second case, if I use operations macro:
The zip type, one of [grossempfänger, normal, postfach, alte plz].

I am suspecting an issue with parsing snippets in this file, but I am not that familiar with Ruby language: https://github.com/spring-projects/spring-restdocs/blob/master/spring-restdocs-asciidoctor-support/src/main/resources/extensions/operation_block_macro.rb

[wilkinsona]

Thanks for the report. The problem only occurring with the operation macro definitely suggests that the Ruby code may be the problem. However, my understand is that with Ruby 2.x, the default string encoding is UTF-8. There are a few additional moving parts here, though, as the Ruby code is running on the JVM via JRuby.

What OS are you using and what is the default encoding set to?

[grolljak]

Hi, I am running Windows 10. Can't find default encoding, but since I never changed it, I am assuming its Windows 10 default.

We have generated same documentation on MacOS and the characters are printed correctly. So this is local problem on my computer. I will investigate whether I am able to set correct encoding on OS/program level and will try to update this thread with solution. Thanks for fast response.

[wilkinsona]

I've reproduced the problem on Windows by making some additions to REST Docs' test suite. Interestingly, the problem only appears to occur with more recent versions of AsciidoctorJ with 1.x being unaffected.