UTF-8 with BOM results in "Not a valid Kotlin file (expecting a top level declaration)"

[bagage]

When running ktlint (0.27.0) on some kotlin file, I get this error:

package demo 
 
fun main(args : Array<String>) { 
  println("Hello, world!") 
}

Error :

$ ktlint --color test.kt --debug --print-ast
[DEBUG] Analyzing /tmp/test.kt
Exception in thread "main" com.github.shyiko.ktlint.core.ParseException: Not a valid Kotlin file (expecting a top level declaration)
	at com.github.shyiko.ktlint.Main$printAST$1.invoke(Main.kt:495)
	at com.github.shyiko.ktlint.Main.printAST(Main.kt:504)
	at com.github.shyiko.ktlint.Main.main(Main.kt:262)

[shyiko]

Hey @bagage. I can't reproduce. Can you please double-check everything?

$ ktlint --version
0.27.0

[bagage]

It's odd: it is working fine on my PC (debian) but not on my working PC (ubuntu 18.04.1).

However the ktlint binary is the same, downloaded via wget https://github.com/shyiko/ktlint/releases/download/0.27.0/ktlint

$ md5sum ktlint
9f7990328c937b5ef5fb63da88e9b3db  ktlint
$ ktlint --version
0.27.0

What in environment could affect ktlint behavior? jdk version? something else? I'll try to investigate.

[bagage]

Hmm, after a bit of investigation, I found out that it was caused by the file encoding:

$ file /tmp/test-ok.kt # returns C source, ASCII text
$ file /tmp/test-broken.kt # returns C source, UTF-8 Unicode (with BOM) text

Here's a link to the broken file in case you want to reproduce the issue. For now I'll convert my files to ASCII instead :).

If UTF-8 Unicode files are not valid Kotlin file, it would be great to have a better hint about the error maybe?

[shyiko]

Huh. Apparently, https://github.com/shyiko/ktlint/blob/master/ktlint-core/src/main/kotlin/com/github/shyiko/ktlint/core/KtLint.kt#L140 chokes when BOM is present (UTF-8 without BOM is fine). Thanks for tracking it down, @bagage!