Overhaul Unicode Common functionality

docs/feature_unicode.md

@@ -1,36 +1,36 @@
 # Unicode Support
 
-There are three Unicode keymap definition method available in QMK:
+There are three Unicode keymap definition methods available in QMK:
 
 ## UNICODE_ENABLE
 
-Supports Unicode input up to 0xFFFF. The keycode function is `UC(n)` in keymap file, where *n* is a 4 digit hexadecimal.
+Supports Unicode up to `0xFFFF`. The keycode function is `UC(n)` in the keymap file, where _n_ is a 4 digit hexadecimal number.
 
 ## UNICODEMAP_ENABLE
 
-Supports Unicode up to 0xFFFFFFFF. You need to maintain a separate mapping table `const uint32_t PROGMEM unicode_map[] = {...}` in your keymap file. The keycode function is `X(n)` where *n* is the array index of the mapping table.
+Supports Unicode up to `0x10FFFF` (all possible code points). You need to maintain a separate mapping table `const uint32_t PROGMEM unicode_map[] = {...}` in your keymap file. The keycode function is `X(n)`, where _n_ is an array index into the mapping table.
 
 And you may want to have an enum to make reference easier.  So you'd want to add something like this to your keymap:
 
 ```c
-enum unicode_name {
-  BANG, // ‽
-  IRONY, // ⸮
-  SNEK // snke 🐍
+enum unicode_names {
+  BANG,
+  IRONY,
+  SNEK,
 };
 
 const uint32_t PROGMEM unicode_map[] = {
-  [BANG]      = 0x0203D, // ‽
-  [IRONY]     = 0x02E2E, // ⸮
-  [SNEK]      = 0x1F40D // snke 🐍
+  [BANG]    = 0x203D,  // ‽
+  [IRONY]   = 0x2E2E,  // ⸮
+  [SNEK]    = 0x1F40D, // 🐍
 }:
 ```
 
 Make sure that the order for both matches.
 
 ## UCIS_ENABLE
 
-Supports Unicode up to 0xFFFFFFFF. As with `UNICODE_MAP`, you may want to maintain a mapping table in your keymap file.  However, there is no keycodes for this feature, you will have to add a keycode or function to call `qk_ucis_start()`. Once you've run that, you can just type the text for your unicode, and then hit space or enter to complete it, or ESC to cancel it. And if it matches an entry in your table, it will automatically "backspace" the trigger word (from your table) and then will input the unicode sequence.
+Supports Unicode up to `0x10FFFF` (all possible code points). As with `UNICODEMAP`, you may want to maintain a mapping table in your keymap file. However, there are no built-in keycodes for this feature — you will have to add a keycode or function that calls `qk_ucis_start()`. Once it's been called, you can type the mnemonic for your character, then hit Space or Enter to complete it or Esc to cancel. If the mnemonic matches an entry in your table, the typed text will automatically be erased and the corresponding Unicode sequence inserted.
 
 For instance, you would need to have a table like this in your keymap:
 
@@ -53,42 +53,103 @@ There are several functions that you can add to your keymap to customize the fun
 * `void qk_ucis_success(uint8_t symbol_index)` - This runs when the unicode input has matched something, and has completed. Default doesn't do anything.
 * `void qk_ucis_symbol_fallback (void)` - This runs if the input text doesn't match anything.  The default function falls back to trying that input as a unicode code.
 
-The default code for these are:
+You can find the default implementations of these functions in [`process_ucis.c`](https://github.com/qmk/qmk_firmware/blob/master/quantum/process_keycode/process_ucis.c).
+
+## Input Modes
+
+Unicode input in QMK works by inputting a sequence of characters to the OS, sort of like a macro. Unfortunately, the way this is done differs for each platform. Specifically, each platform requires a different combination of keys to trigger Unicode input. Therefore, a corresponding input mode has to be set in QMK.
+
+The following input modes are available:
+
+* **`UC_OSX`**: Mac OS X built-in Unicode hex input. Supports code points up to `0xFFFF` (`0x10FFFF` with `UNICODEMAP`).
+
+  To enable, go to _System Preferences > Keyboard > Input Sources_, add _Unicode Hex Input_ to the list (it's under _Other_), then activate it from the input dropdown in the Menu Bar.
+  By default, this mode uses the left Option key (`KC_LALT`), but this can be changed by defining [`UNICODE_OSX_KEY`](#input-key-configuration) with another keycode.
+
+* **`UC_LNX`**: Linux built-in IBus Unicode input. Supports all possible code points (`0x10FFFF`).
+
+  Enabled by default and works almost anywhere on IBus-enabled distros. Without IBus, this mode works under GTK apps, but rarely anywhere else.
+
+* **`UC_WIN`**: _(not recommended)_ Windows built-in hex numpad Unicode input. Supports code points up to `0xFFFF`.
+
+  To enable, create a registry key under `HKEY_CURRENT_USER\Control Panel\Input Method\EnableHexNumpad` of type `REG_SZ` called `EnableHexNumpad` and set its value to `1`. This can be done from the Command Prompt by running `reg add "HKCU\Control Panel\Input Method" -v EnableHexNumpad -t REG_SZ -d 1` with administrator privileges. Afterwards, reboot.
+  This mode is not recommended because of reliability and compatibility issues; use the `UC_WINC` mode instead.
+
+* **`UC_BSD`**: _(non implemented)_ Unicode input under BSD. Not implemented at this time. If you're a BSD user and want to help add support for it, please [open an issue on GitHub](https://github.com/qmk/qmk_firmware/issues).
+
+* **`UC_WINC`**: Windows Unicode input using [WinCompose](https://github.com/samhocevar/wincompose). As of v0.8.2, supports code points up to `0xFFFFF`.
+
+  To enable, install the [latest release](https://github.com/samhocevar/wincompose/releases/latest). Once installed, WinCompose will automatically run on startup. Works reliably under all version of Windows supported by the app.
+  By default, this mode uses the right Alt key (`KC_RALT`), but this can be changed in the WinCompose settings and by defining [`UNICODE_WINC_KEY`](#input-key-configuration) with another keycode.
+
+### Switching Input Modes
+
+There are two ways to set the input mode for Unicode: by keycode or by function. Keep in mind that both methods write to persistent storage (EEPROM), and are loaded each time the keyboard starts. So once you've set it the first time, you don't need to set it again unless you want to change it, or you've reset the EEPROM settings.
+
+You can switch the input mode at any time by using one of the following keycodes. The easiest way is to add the ones you use to your keymap.
+
+|Keycode                |Alias    |Input mode   |Description                              |
+|-----------------------|---------|-------------|-----------------------------------------|
+|`UNICODE_MODE_FORWARD` |`UC_MOD` |             |Cycles forwards through the available modes. [(Disabled by default)](#input-method-cycling)|
+|`UNICODE_MODE_REVERSE` |`UC_RMOD`|             |Cycles forwards through the available modes. [(Disabled by default)](#input-method-cycling)|
+|`UNICODE_MODE_OSX`     |`UC_M_OS`|`UC_OSX`     |Switch to Mac OS X input.                |
+|`UNICODE_MODE_LNX`     |`UC_M_LN`|`UC_LNX`     |Switch to Linux input.                   |
+|`UNICODE_MODE_WIN`     |`UC_M_WI`|`UC_WIN`     |Switch to Windows input.                 |
+|`UNICODE_MODE_BSD`     |`UC_M_BS`|`UC_BSD`     |Switch to BSD input (not implemented).   |
+|`UNICODE_MODE_WINC`    |`UC_M_WC`|`UC_WINC`    |Switch to Windows input using WinCompose.|
+
+You can also switch the input mode by calling `set_unicode_input_mode(x)` in your code, where _x_ is one of the above input mode constants (e.g. `UC_LNX`). Since the function only needs to be called once, it's recommended that you do it in `eeconfig_init_user` (or a similar function). For example:
 
 ```c
-void qk_ucis_start_user(void) { // outputs keyboard emoji
-  unicode_input_start();
-  register_hex(0x2328);
-  unicode_input_finish();
+void eeconfig_init_user(void) {
+  set_unicode_input_mode(UC_LNX);
 }
+```
 
-void qk_ucis_success(uint8_t symbol_index) {
-}
+### Audio Feedback
 
-void qk_ucis_symbol_fallback (void) { // falls back to manual unicode entry
-  for (uint8_t i = 0; i < qk_ucis_state.count - 1; i++) {
-    uint8_t code = qk_ucis_state.codes[i];
-    register_code(code);
-    unregister_code(code);
-    wait_ms(UNICODE_TYPE_DELAY);
-  }
-}
+If you have the [Audio feature](feature_audio.md) enabled on the board, you can set melodies to be played when you press the above keys. That way you can have some audio feedback when switching input modes.
+
+For instance, you can add these definitions to your `config.h` file:
+
+```c
+#define UNICODE_SONG_OSX  COIN_SOUND
+#define UNICODE_SONG_LNX  UNICODE_LINUX
+#define UNICODE_SONG_BSD  MARIO_GAMEOVER
+#define UNICODE_SONG_WIN  UNICODE_WINDOWS
+#define UNICODE_SONG_WINC UNICODE_WINDOWS
 ```
 
-## Unicode Input methods
+### Additional Customization
+
+Because Unicode is such a large and variable feature, there are a number of options that you can customize to work better on your system. 
+
+#### Start and Finish input functions
+
+The functions for starting and finishing Unicode input on your platform can be overridden locally. Possible uses include customizing input mode behavior if you don't use the default keys, or adding extra visual/audio feedback to Unicode input.
+
+* `void unicode_input_start(void)` – This sends the initial sequence that tells your platform to enter Unicode input mode. For example, it presses Ctrl+Shift+U on Linux and holds the Option key on Mac.
+* `void unicode_input_finish(void)` – This is called to exit Unicode input mode, for example by pressing Space or releasing the Option key.
 
-Unicode input in QMK works by inputting a sequence of characters to the OS,
-sort of like macro. Unfortunately, each OS has different ideas on how Unicode is input.
+You can find the default implementations of these functions in [`process_unicode_common.c`](https://github.com/qmk/qmk_firmware/blob/master/quantum/process_keycode/process_unicode_common.c).
 
-This is the current list of Unicode input method in QMK:
 
-* __UC_OSX__: MacOS Unicode Hex Input support. Works only up to 0xFFFF. Disabled by default. To enable: go to System Preferences -> Keyboard -> Input Sources, and enable Unicode Hex.
-* __UC_OSX_RALT__: Same as UC_OSX, but sends the Right Alt key for unicode input
-* __UC_LNX__: Unicode input method under Linux. Works up to 0xFFFFF. Should work almost anywhere on ibus enabled distros. Without ibus, this works under GTK apps, but rarely anywhere else.
-* __UC_WIN__: (not recommended) Windows built-in Unicode input. To enable: create registry key under `HKEY_CURRENT_USER\Control Panel\Input Method\EnableHexNumpad` of type `REG_SZ` called `EnableHexNumpad`, set its value to 1, and reboot. This method is not recommended because of reliability and compatibility issue, use WinCompose method below instead.
-* __UC_WINC__: Windows Unicode input using WinCompose. Requires [WinCompose](https://github.com/samhocevar/wincompose). Works reliably under many (all?) variations of Windows.
+#### Input Key Configuration
 
-At some point, you need to call `set_unicode_input_mode(x)` to set the correct unicode method.  This sets the method that is used to send the unicode, and stores it in EEPROM, so you only need to call this once.
+Additionally, you can customize the keys used to trigger the unicode input for macOS and WinCompose by adding defines to your `config.h`
+
+```c
+#define UNICODE_OSX_KEY  KC_LALT
+#define UNICODE_WINC_KEY KC_RALT
+```
+
+#### Input Method Cycling
+
+Also, you can choose which input methods are availble for cycling through.  By default, this is disabled. But if you want to enabled it, then limiting it to just those modes makes sense.  Note that `UNICODE_SELECTED_MODES` define is comma delimited.
+
+```c
+#define UNICODE_SELECTED_MODES UC_OSX, UC_LNX, UC_WIN, UC_BSD, UC_WINC
+```
 
 ## `send_unicode_hex_string`
 

keyboards/whitefox/keymaps/konstantin/config.h

@@ -21,3 +21,7 @@
 #define PERMISSIVE_HOLD
 #define TAPPING_TERM    200
 #define TAPPING_TOGGLE  2
+
+#define UNICODE_CYCLE_PERSIST   false
+#define UNICODE_SELECTED_MODES  UC_WINC, UC_LNX
+#define UNICODE_WINC_KEY        KC_RGUI

keyboards/whitefox/keymaps/konstantin/keymap.c

@@ -23,10 +23,6 @@
 #define DIVIDE  UC(0x00F7)
 #define MINUS   UC(0x2212)
 
-void eeconfig_init_user(void) {
-  set_unicode_input_mode(UC_WINC);
-}
-
 enum layers {
   L_BASE,
   L_FN,
@@ -162,7 +158,7 @@ const uint16_t PROGMEM keymaps[][MATRIX_ROWS][MATRIX_COLS] = {
    * ┌───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┬───┐
    * │   │F1 │F2 │F3 │F4 │F5 │F6 │F7 │F8 │F9 │F10│F11│F12│Num│Scr│Pau│
    * ├───┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴───┼───┤
-   * │ M4  │M2 │M↑ │M1 │M3 │M5 │   │   │   │Stp│Ply│Prv│Nxt│Clear│Ins│
+   * │ M4  │M2 │M↑ │M1 │M3 │M5 │   │UCM│   │Stp│Ply│Prv│Nxt│Clear│Ins│
    * ├─────┴┬──┴┬──┴┬──┴┬──┴┬──┴┬──┴┬──┴┬──┴┬──┴┬──┴┬──┴┬──┴─────┼───┤
    * │      │M← │M↓ │M→ │MW↑│   │   │   │   │   │   │   │        │Top│
    * ├──────┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴────┬───┼───┤
@@ -173,7 +169,7 @@ const uint16_t PROGMEM keymaps[][MATRIX_ROWS][MATRIX_COLS] = {
    */
   [L_FN] = LAYOUT_truefox( \
     _______, KC_F1,   KC_F2,   KC_F3,   KC_F4,   KC_F5,   KC_F6,   KC_F7,   KC_F8,   KC_F9,   KC_F10,  KC_F11,  KC_F12,  NUMPAD,  KC_SLCK, KC_PAUS, \
-    KC_BTN4, KC_BTN2, KC_MS_U, KC_BTN1, KC_BTN3, KC_BTN5, _______, _______, _______, KC_MSTP, KC_MPLY, KC_MPRV, KC_MNXT, CLEAR,            KC_INS,  \
+    KC_BTN4, KC_BTN2, KC_MS_U, KC_BTN1, KC_BTN3, KC_BTN5, _______, UC_MOD,  _______, KC_MSTP, KC_MPLY, KC_MPRV, KC_MNXT, CLEAR,            KC_INS,  \
     _______, KC_MS_L, KC_MS_D, KC_MS_R, KC_WH_U, _______, _______, _______, _______, _______, _______, _______,          _______,          TOP,     \
     _______,          KC_ACL0, KC_ACL2, KC_WH_L, KC_WH_R, _______, _______, _______, KC_VOLD, KC_VOLU, KC_MUTE, KC_APP,           KC_PGUP, BOTTOM,  \
     _______, DESKTOP, DSKTP_L,                            KC_WH_D,                   DSKTP_R, _______,                   KC_HOME, KC_PGDN, KC_END   \

quantum/process_keycode/process_ucis.c

@@ -95,8 +95,6 @@ void register_ucis(const char *hex) {
 bool process_ucis (uint16_t keycode, keyrecord_t *record) {
   uint8_t i;
 
-  unicode_input_mode_init();
-
   if (!qk_ucis_state.in_progress)
     return true;
 

quantum/process_keycode/process_ucis.h

@@ -14,8 +14,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef PROCESS_UCIS_H
-#define PROCESS_UCIS_H
+#pragma once
 
 #include "quantum.h"
 #include "process_unicode_common.h"
@@ -48,5 +47,3 @@ void qk_ucis_symbol_fallback (void);
 void qk_ucis_success(uint8_t symbol_index);
 void register_ucis(const char *hex);
 bool process_ucis (uint16_t keycode, keyrecord_t *record);
-
-#endif

quantum/process_keycode/process_unicode.c

@@ -20,11 +20,9 @@
 bool process_unicode(uint16_t keycode, keyrecord_t *record) {
   if (keycode > QK_UNICODE && record->event.pressed) {
     uint16_t unicode = keycode & 0x7FFF;
-    unicode_input_mode_init();
     unicode_input_start();
     register_hex(unicode);
     unicode_input_finish();
   }
   return true;
 }
-

quantum/process_keycode/process_unicode.h

@@ -13,12 +13,9 @@
  * You should have received a copy of the GNU General Public License
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
-#ifndef PROCESS_UNICODE_H
-#define PROCESS_UNICODE_H
+#pragma once
 
 #include "quantum.h"
 #include "process_unicode_common.h"
 
 bool process_unicode(uint16_t keycode, keyrecord_t *record);
-
-#endif