Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Commit

Permalink
add encoding format
Browse files Browse the repository at this point in the history
  • Loading branch information
lanking520 committed Jul 16, 2018
1 parent 065a3ea commit 771fc96
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ object BucketIo {
type ReadContent = String => String

def defaultReadContent(path: String): String = {
Source.fromFile(path).mkString.replaceAll("\\. |\n", " <eos> ")
Source.fromFile(path, "UTF-8").mkString.replaceAll("\\. |\n", " <eos> ")
}

def defaultBuildVocab(path: String): Map[String, Int] = {
Expand Down

0 comments on commit 771fc96

Please sign in to comment.