Allow non-English scripts for unquoted keys

CHANGELOG.md

@@ -4,6 +4,7 @@
 
 * Clarify where and how dotted keys define tables.
 * Add new `\e` shorthand for the escape character.
+* Allow non-English scripts in unquoted (bare) keys
 
 ## 1.0.0 / 2021-01-11
 

toml.abnf

@@ -46,19 +46,31 @@ comment = comment-start-symbol *non-eol
 ;; Key-Value pairs
 
 keyval = key keyval-sep val
-
 key = simple-key / dotted-key
+val = string / boolean / array / inline-table / date-time / float / integer
+
 simple-key = quoted-key / unquoted-key
 
-unquoted-key = 1*( ALPHA / DIGIT / %x2D / %x5F ) ; A-Z / a-z / 0-9 / - / _
+;; Unquoted key
+
+unquoted-key = 1*unquoted-key-char
+unquoted-key-char = ALPHA / DIGIT / %x2D / %x5F         ; a-z A-Z 0-9 - _
+unquoted-key-char =/ %xC0-D6 / %xD8-F6 / %xF8-37D       ; non-symbol chars in Latin block
+unquoted-key-char =/ %x37F-1FFF                         ; exclude GREEK QUESTION MARK, which is basically a semi-colon
+unquoted-key-char =/ %x200C-200D / %x203F-2040          ; include combining chars used in some languages
+unquoted-key-char =/ %x2070-218F / %x2C00-2FEF          ; this excludes arrows, blocks and the like
+unquoted-key-char =/ %x3001-D7FF                        ; skip 2FF0-3000 ideographic up/down markers and spaces
+unquoted-key-char =/ %xF900-FDCF / %xFDF0-FFFD          ; skip D800-D999 surrogate block, E000-F8FF Private Use area, FDD0-FDEF intended for process-internal use (unicode)
+unquoted-key-char =/ %x10000-EFFFF                      ; all chars outside BMP range, excluding Private Use planes
+
+;; Quoted and dotted key
+
 quoted-key = basic-string / literal-string
 dotted-key = simple-key 1*( dot-sep simple-key )
 
 dot-sep   = ws %x2E ws  ; . Period
 keyval-sep = ws %x3D ws ; =
 
-val = string / boolean / array / inline-table / date-time / float / integer
-
 ;; String
 
 string = ml-basic-string / basic-string / ml-literal-string / literal-string

toml.md

@@ -104,27 +104,39 @@ Keys
 
 A key may be either bare, quoted, or dotted.
 
-**Bare keys** may only contain ASCII letters, ASCII digits, underscores, and
-dashes (`A-Za-z0-9_-`). Note that bare keys are allowed to be composed of only
-ASCII digits, e.g. `1234`, but are always interpreted as strings.
+**Bare keys** may contain any letter-like Unicode character from any Unicode script,
+as well as ASCII digits, dashes and underscores. Punctuation, spaces, arrows, box drawing 
+and private use characters are not allowed.  Note that bare keys are allowed to be
+composed of only ASCII digits, e.g. `1234`, but are always interpreted as strings.
+
+* From the ASCII characters, only A-Z, a-z, 0-9, _ and - are allowed
+* From the rest of the first 256 characters, only U+0080-00BF, "×" (U+00D7) and "÷" (U+00F7) are disallowed
+* All of U+0100-1FFF are allowed, except ";" (U+037E)
+* Characters U+200C, U+200D, U+203F, U+2040, U+2070-218F, U+2C00 to U+2FEF are allowed
+* All characters from U+3001 and higher, except surrogates (U+D800 to U+D999), Private Use (U+E000 to U+F8FF, U+F0000 to U+100000) and process-internal use (U+FDD0 to U+FDEF)
 
 ```toml
 key = "value"
 bare_key = "value"
 bare-key = "value"
 1234 = "value"
+Fuß = "value"
+😂 = "value"
+汉语大字典 = "value"
+辭源 = "value"
+பெண்டிரேம் = "value"
 ```
 
 **Quoted keys** follow the exact same rules as either basic strings or literal
-strings and allow you to use a much broader set of key names. Best practice is
-to use bare keys except when absolutely necessary.
+strings and allow you to use any Unicode character in a key name, including spaces.
+Best practice is to use bare keys except when absolutely necessary.
 
 ```toml
 "127.0.0.1" = "value"
 "character encoding" = "value"
-"ʎǝʞ" = "value"
-'key2' = "value"
 'quoted "value"' = "value"
+"╠═╣" = "value"
+"⋰∫∬∭⋱" = "value"
 ```
 
 A bare key must be non-empty, but an empty quoted key is allowed (though
@@ -145,6 +157,7 @@ name = "Orange"
 physical.color = "orange"
 physical.shape = "round"
 site."google.com" = true
+பெண்.டிரேம் = "we are women"
 ```
 
 In JSON land, that would give you the following structure:
@@ -158,6 +171,9 @@ In JSON land, that would give you the following structure:
   },
   "site": {
     "google.com": true
+  },
+  "பெண்": {
+    "டிரேம்": "we are women"
   }
 }
 ```