Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All ARM Keyboards do not work on Ubuntu 18.04 #5631

Closed
awkannan opened this issue Apr 16, 2019 · 45 comments · Fixed by #12472
Closed

All ARM Keyboards do not work on Ubuntu 18.04 #5631

awkannan opened this issue Apr 16, 2019 · 45 comments · Fixed by #12472

Comments

@awkannan
Copy link
Contributor

awkannan commented Apr 16, 2019

On Ubuntu 18.04, ARM powered keyboards are not recognized by default.

The OS sees the device, but no input shows up until you run hid_listen, eventually hid_listen picks up the device and then you can start typing.

The keyboard will intermittently stop working, and you will have to run hid_listen again to get it to work.

@mxgian reproduced this issue with a Planck rev 6, Instant60, and Ortho60 board. So it's not just limited to f303, but is also present on f103 and f072.

@mxgian
Copy link

mxgian commented Apr 16, 2019

hid_listen only opens the raw device to read input. i can also 'make the keyboard' work by cat'ting the raw device, in my case 'cat /dev/hidraw0' to start reading the raw device and then the keyboard starts working.

at least for me the keyboard continues to work until disconnect, but others have reported it will stop working after a few minutes.

update: so far this happens everytime with the instant60, but is inconsistent with the planck, sometimes the planck will start working after about 30 seconds. unclear why it works sometimes.

@yanfali
Copy link
Contributor

yanfali commented Apr 16, 2019

Looks like Xorg is crashing and taking down the system with it

https://gist.github.com/yanfali/a16bd5a7da633830962d127e81999caa

Apr 16 12:00:26 ptah systemd-coredump[1115]: Process 1056 (Xorg) of user 0 dumped core.
                                             
                                             Stack trace of thread 1056:
                                             #0  0x00007f1412572d7f raise (libc.so.6)
                                             #1  0x00007f141255d672 abort (libc.so.6)
                                             #2  0x0000563104dcab5a OsAbort (Xorg)
                                             #3  0x0000563104dc15cf FatalError (Xorg)
                                             #4  0x0000563104dcf3ee n/a (Xorg)
                                             #5  0x00007f1412572e00 __restore_rt (libc.so.6)
                                             #6  0x00007f14125f17c4 __wcslen_sse2 (libc.so.6)
                                             #7  0x00007f14125e0883 wcsrtombs (libc.so.6)
                                             #8  0x00007f141258de34 vfprintf (libc.so.6)
                                             #9  0x00007f1412645c5f __vsnprintf_chk (libc.so.6)
                                             #10 0x0000563104dbc901 Xvscnprintf (Xorg)
                                             #11 0x0000563104dc290f LogVMessageVerb (Xorg)
                                             #12 0x00007f140fe30044 n/a (libinput.so.10)
                                             #13 0x00007f140fe381ad n/a (libinput.so.10)
                                             #14 0x00007f140fe3a6d1 n/a (libinput.so.10)
                                             #15 0x00007f140fe1e944 n/a (libinput.so.10)
                                             #16 0x00007f140fe1eb29 n/a (libinput.so.10)
                                             #17 0x00007f140fe1ec87 libinput_path_add_device (libinput.so.10)
                                             #18 0x00007f140ffd20ff n/a (libinput_drv.so)
                                             #19 0x0000563104da2c27 n/a (Xorg)
                                             #20 0x0000563104d680c2 n/a (Xorg)
                                             #21 0x0000563104d689a3 config_init (Xorg)
                                             #22 0x0000563104daf137 InitInput (Xorg)
                                             #23 0x0000563104d55734 n/a (Xorg)
                                             #24 0x00007f141255f223 __libc_start_main (libc.so.6)
                                             #25 0x0000563104d5630e _start (Xorg)
                                             
                                             Stack trace of thread 1072:
                                             #0  0x00007f1411ba2afc pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0)
                                             #1  0x00007f140f3bc254 n/a (i965_dri.so)
                                             #2  0x00007f140f3bbf78 n/a (i965_dri.so)
                                             #3  0x00007f1411b9ca9d start_thread (libpthread.so.0)
                                             #4  0x00007f1412636b23 __clone (libc.so.6)
                                             
                                             Stack trace of thread 1068:
                                             #0  0x00007f1411ba2afc pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0)
                                             #1  0x00007f140f3bc254 n/a (i965_dri.so)
                                             #2  0x00007f140f3bbf78 n/a (i965_dri.so)
                                             #3  0x00007f1411b9ca9d start_thread (libpthread.so.0)
                                             #4  0x00007f1412636b23 __clone (libc.so.6)
Apr 16 12:00:26 ptah systemd[1]: systemd-coredump@0-1114-0.service: Succeeded.

I haven't done a headless test to see if this affects terminal.

@yanfali
Copy link
Contributor

yanfali commented Apr 16, 2019

This was with a 5.0.7 kernel on Arch Linux, so it's pretty new. Xorg was 1.20.4 so fairly new.

@yanfali
Copy link
Contributor

yanfali commented Apr 16, 2019

OK, I just tested this with just a tty and it works perfectly. This seems to be an xorg issue. They are crashing when they detect our keyboard firmware.

@yanfali
Copy link
Contributor

yanfali commented Apr 16, 2019

https://bugs.freedesktop.org/show_bug.cgi?id=110453 issue filed with Xorg. Not sure what'll happen.

@mxgian
Copy link

mxgian commented Apr 16, 2019

thanks for the quick turnaround. if you need to me generate a dump from my ubuntu system i can do that, just let me know how to generate the coredump.

@yanfali
Copy link
Contributor

yanfali commented Apr 16, 2019

It should be in the journald. I found it by using root and typing journalctl > afile.txt and then working backwards in the file. You could look for a REBOOT event as where to start searching.

@TellNoLies
Copy link

I have the same issue as mxgian on Gentoo Linux. My system:
Operating System: Gentoo Linux
KDE Plasma Version: 5.14.5
Qt Version: 5.11.3
KDE Frameworks Version: 5.54.0
Kernel Version: 4.20.10-gentoo
OS Type: 64-bit
Processors: 12 × Intel® Core™ i7-5820K CPU @ 3.30GHz
Memory: 15.6 GiB of RAM
Installed versions:
cross-arm-unknown-linux-gnueabi/gcc 7.3.0-r3, 8.2.0-r6, 8.3.0-r1
cross-avr/gcc 4.9.4, 7.3.0-r3
x11-base/xorg-drivers 1.20
cross-avr/avr-libc 2.0.0
dev-embedded/avrdude 6.3
sys-devel/crossdev 20190311

Hopefully that is enough information.

@yanfali
Copy link
Contributor

yanfali commented Apr 17, 2019

@TellNoLies feel free to chime in on the bug I filed.

@TellNoLies
Copy link

@yanfali Should I paste the same info on the Bugzilla?

@yanfali
Copy link
Contributor

yanfali commented Apr 17, 2019

@TellnlNoLies I think that would be helpful to have more than one user chime in, especially on different distros

@TellNoLies
Copy link

@yanfali how was that comment that I just left on the bug report? Will that info be helpful?

@yanfali
Copy link
Contributor

yanfali commented Apr 17, 2019

Very nice

@TellNoLies
Copy link

Okay, thanks.

@drashna
Copy link
Member

drashna commented Apr 18, 2019

I just booted up my laptop to test this.
Ubuntu 18.10 x64
Kernel is 4.18.0-17-generic.

@TellNoLies
Copy link

@drashna did it do the same for you on Ubuntu?

@drashna
Copy link
Member

drashna commented Apr 18, 2019

@drashna did it do the same for you on Ubuntu?

I'm assuming you mean 18.04, specifically. I don't know. I've been using 18.10 for a while, since the ARM GCC binaries are broken on 18.04. However, I will swap drives out and test this in the next day or so.

@drashna
Copy link
Member

drashna commented Apr 19, 2019

Installed 18.04. Can't reproduce. had system on for an hour, testing periodically.

@sfines
Copy link

sfines commented Apr 20, 2019

To date I've reproduced it on 18.04, 18.10, and 19.04, as well as on 3 different machines by 3 different manufacturers.

However, X does not dump core for me, it just doesn't recognize the device: I think it may have to do with the order udev enumerates... it sees it as a mouse first and as a keyboard second..

(II) Using input driver 'libinput' for 'QMK Ortho60 Mouse'
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) systemd-logind: got fd for /dev/input/event26 13:90 fd 76 paused 0
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Mouse: always reports core events
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "Device" "/dev/input/event26"
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "_source" "server/udev"
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event26 - QMK Ortho60 Mouse: is tagged by udev as: Mouse
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event26 - QMK Ortho60 Mouse: device is a pointer
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event26 - QMK Ortho60 Mouse: device removed
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "config_info" "udev:/sys/devices/pci0000:00/0000:00:14.0/usb1/1-3/1-3:1.1/0003:FEED:6464.0010/input
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) XINPUT: Adding extended input device "QMK Ortho60 Mouse" (type: MOUSE, id 28)
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "AccelerationScheme" "none"
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Mouse: (accel) selected scheme none/0
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Mouse: (accel) acceleration factor: 2.000
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Mouse: (accel) acceleration threshold: 4
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Mouse: (accel) selected scheme none/0
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Mouse: (accel) acceleration factor: 2.000
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Mouse: (accel) acceleration threshold: 4
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event26 - QMK Ortho60 Mouse: is tagged by udev as: Mouse
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event26 - QMK Ortho60 Mouse: device is a pointer
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) Using input driver 'libinput' for 'QMK Ortho60 Consumer Control'
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) systemd-logind: got fd for /dev/input/event28 13:92 fd 77 paused 0
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Consumer Control: always reports core events
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "Device" "/dev/input/event28"
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "_source" "server/udev"
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event28 - QMK Ortho60 Consumer Control: is tagged by udev as: Keyboard
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event28 - QMK Ortho60 Consumer Control: device is a keyboard
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event28 - QMK Ortho60 Consumer Control: device removed
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) libinput: QMK Ortho60 Consumer Control: needs a virtual subdevice
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "config_info" "udev:/sys/devices/pci0000:00/0000:00:14.0/usb1/1-3/1-3:1.1/0003:FEED:6464.0010/input
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) XINPUT: Adding extended input device "QMK Ortho60 Consumer Control" (type: MOUSE, id 29)
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "AccelerationScheme" "none"
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Consumer Control: (accel) selected scheme none/0
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Consumer Control: (accel) acceleration factor: 2.000
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Consumer Control: (accel) acceleration threshold: 4
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event28 - QMK Ortho60 Consumer Control: is tagged by udev as: Keyboard
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event28 - QMK Ortho60 Consumer Control: device is a keyboard
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) systemd-logind: got resume for 13:65
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event1  - Power Button: is tagged by udev as: Keyboard
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event1  - Power Button: device is a keyboard
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) systemd-logind: got resume for 13:82
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event18 - DELL0810:00 044E:120A UNKNOWN: is tagged by udev as: Keyboard
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event18 - DELL0810:00 044E:120A UNKNOWN: device is a keyboard
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) systemd-logind: got resume for 13:79
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event15 - Intel HID events: is tagged by udev as: Keyboard
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) event15 - Intel HID events: device is a keyboard
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Consumer Control: Applying InputClass "libinput keyboard catchall"
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) Using input driver 'libinput' for 'QMK Ortho60 Consumer Control'
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) systemd-logind: returning pre-existing fd for /dev/input/event28 13:92
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) QMK Ortho60 Consumer Control: always reports core events
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "Device" "/dev/input/event28"
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "_source" "_driver/libinput"
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) libinput: QMK Ortho60 Consumer Control: is a virtual subdevice
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "config_info" "udev:/sys/devices/pci0000:00/0000:00:14.0/usb1/1-3/1-3:1.1/0003:FEED:6464.0010/input
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (II) XINPUT: Adding extended input device "QMK Ortho60 Consumer Control" (type: KEYBOARD, id 30)
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "xkb_model" "pc105"
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (**) Option "xkb_layout" "us"
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (WW) Option "xkb_variant" requires a string value
Apr 19 22:42:39 slf-cons-lt /usr/lib/gdm3/gdm-x-session[2587]: (WW) Option "xkb_options" requires a string value

@drashna
Copy link
Member

drashna commented Apr 26, 2019

Well, if you have mousekeys enabled, it does add a USB HID Mouse device, as well.

@sfines
Copy link

sfines commented Apr 26, 2019

I wonder if that's the problem. I'll disable mousekeys (it's part of the cannonkeys ortho60 default image, if it is in fact on) and see what happens.

If it does, it's still a problem- just a different one.

@airyimbin
Copy link

For me the keyboard is only recognized if I boot with it plugged in from the start.
If I plug in the keyboard after Ubuntu boots it does not recognize it.
Also I if enable or disable LED it no longer recognizes it and I have to reboot.

I'm on Ubuntu 18.04.

@rycwo
Copy link

rycwo commented Jun 24, 2019

@sfines, having the exact same issue on Arch Linux/Centos 6.5 with the Ortho60. Did disabling mouse keys fix the issue for you?

With limited knowledge it looks like the following for me:

  • board is recognized;
  • bootloader loads the QMK firmware;
  • then I get usb_submit_urb(ctrl) failed: -1

In my case the keyboard is trying to be initialized before the "mouse".

The board used by the Ortho60 is the STM32F103C8.

@yanfali
Copy link
Contributor

yanfali commented Jun 24, 2019

We're not going any response for xorg, you might want to go nag them as it's an xinput issue and it's not clear how to resolve it.

@yanfali
Copy link
Contributor

yanfali commented Jun 24, 2019

Well you could try and do a debug build for xinput and then capture a coredump and see if you can find out where it's crashing.

@rycwo
Copy link

rycwo commented Jun 24, 2019 via email

@yanfali
Copy link
Contributor

yanfali commented Jun 25, 2019

I think it's all related. There are no issues if you just run it from a tty. Only from X. I'm running on Arch which has a modern kernel and xinput. Ubuntu is ancient in comparison.

@evamvid
Copy link

evamvid commented Sep 9, 2019

I've had this issue as well, but the keyboard behaves exactly the same in a tty as it does in x. In both cases, no keypresses are registered until I run hid_listen as root, at which point it starts working and continues even after I kill hid_listen.

This makes me think that it isn't an issue with X, but an issue with the USB drivers.

I'm using a BluePill-based Ortho48 from Cannonkeys on Arch Linux (technically AntergOS, which is now dead, but it's Arch for all intents and purposes).

My dmesg | tail looks like this:


[   81.808643] usb 1-1.2: Manufacturer: QMK
[   81.808646] usb 1-1.2: SerialNumber: 0
[   81.812865] input: QMK Ortho48 as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.2/1-1.2:1.0/0003:CA04:0248.0007/input/input34
[   81.869637] hid-generic 0003:CA04:0248.0007: input,hidraw3: USB HID v1.11 Keyboard [QMK Ortho48] on usb-0000:00:1a.0-1.2/input0
[   81.871417] input: QMK Ortho48 Mouse as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.2/1-1.2:1.1/0003:CA04:0248.0008/input/input35
[   81.871530] input: QMK Ortho48 System Control as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.2/1-1.2:1.1/0003:CA04:0248.0008/input/input36
[   81.926145] input: QMK Ortho48 Consumer Control as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.2/1-1.2:1.1/0003:CA04:0248.0008/input/input37
[   81.926382] input: QMK Ortho48 Keyboard as /devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.2/1-1.2:1.1/0003:CA04:0248.0008/input/input38
[   81.927574] hid-generic 0003:CA04:0248.0008: input,hidraw4: USB HID v1.11 Mouse [QMK Ortho48] on usb-0000:00:1a.0-1.2/input1
[   81.929691] hid-generic 0003:CA04:0248.0009: hiddev1,hidraw5: USB HID v1.11 Device [QMK Ortho48] on usb-0000:00:1a.0-1.2/input2

My boot log, grepped for usb (`journalctl -b | grep usb) is located here:

https://gist.github.com/evamvid/88f54b35faa04ca2aa53206e6299ed59

I think this might give some more insight into what the issue is. The keyboard is located on usb 1-1.2. For some reason it repeatedly "rejects the address" assigned by the computer, causing read errors.

Hopefully this is somewhat helpful!

Also, is it worth opening a separate issue for the STM boards? I don't have the original issue with crashing xorg. It seems like the STM boards might have a separate issue, because they don't seem to work in tty either.

@yanfali
Copy link
Contributor

yanfali commented Sep 11, 2019

So I've been trying to figure out what's going on. I have 1 linux box that crashes, it has Sandy Bridge hardware and it's a laptop. I have one desktop running a server based chipset and it does not crash and it's running exactly the same version of xorg 1.20.5. Still haven't figured out what is the difference. Can people chime in on what hardware they running linux on? Specifically intel chipset would be interesting to know.

I thought it might be a XHCI (USB3) vs EHCI (USB2) but I was unable to make it crash with that change. I've also disabled NKRO, SHARED_EP and MOUSE and it still crashes the problematic laptop.

@yanfali
Copy link
Contributor

yanfali commented Sep 11, 2019

OK folks, I just confirmed that after linux kernel 5.2.9 the problem is gone. I don't know what they fixed but it was still broken around 5.2.5, so in between those two releases something got fixed in the kernel. It's working for me now. I think it's just a kernel issue.

@yanfali
Copy link
Contributor

yanfali commented Sep 24, 2019

#6801 (review) please test this patch if you having Linux issues

@krnhotwings
Copy link

Tested the patch, and it does not fix the linux issue. The patch actually would have made no difference with my config settings (mouse buttons disabled, raw hid disabled, audio disabled, console enabled.) With only console enabled, the outcome of HID ordering is the same before and after the patch is applied.

I'm actually fairly confident that the problem has to do with the CONSOLE_ENABLE feature itself. If the rules.mk file has the feature turned off, the keyboard will work on linux.

I've also found that, at least on my system, if I have the console enabled, the keyboard will work on linux if I treat it like a PS/2 keyboard: keep the keyboard plugged in on boot. Once replugged or restarted, the keyboard will become unresponsive until next boot. Sometimes the keyboard will get into a funky "frozen" state during bootup, which would require me to restart the keyboard and then restart my system, but the instances that that had happened were pretty rare (meaning, I couldn't replicate the freezing behavior.)

iirc, my home desktop is running Pop OS 19.04 with kernel 5.0.0

@yanfali
Copy link
Contributor

yanfali commented Oct 2, 2019

@krnhotwings do you have any QMK keyboards that are AVR based with console turned on? Would like an a/b comparison. It looks like something chibios is doing is causing linux to puke for console output.

@krnhotwings
Copy link

Yup, I have a Tada68 (atmega32u4). Console enabled, keyboard works fine. Keyboard still works when replugged.

I added debug print statements to process_record_user and hid_listen prints as expected.

@sidcarter
Copy link
Contributor

thanks for that @krnhotwings - disabling console also fixed a similar issue with my alice pcb

@ridingqwerty
Copy link
Contributor

ridingqwerty commented Dec 31, 2019

Just want to add my experience to this issue.

I had been using a Proton C on Ubuntu 18.04 (kernel 4.15.0) for some time now with no issues. CONSOLE_ENABLE had been set to yes in rules.mk at least as far back as Oct 31 of this year.

Soon after I added the uprintf statement to process_record_user from testing and debugging docs a few days ago, I noticed the keyboard became intermittently spammy or unresponsive unless hid_listen was running as root.

As soon as I commented out the uprintf, but with CONSOLE_ENABLE still set to yes, the issue went away.

quick edit: actually it does appear to become unresponsive after a time, so I've set CONSOLE_ENABLE to no to see if that helps.

@stapelberg
Copy link
Contributor

I just ran into this issue myself.

The culprit is CONSOLE_ENABLE = yes in combination with anything that produces enough debug messages to the console, e.g. #define DEBUG_MATRIX_SCAN_RATE in config.h which produces one message every second.

The root cause is that the QMK ChibiOS USB driver does not drain the debug message queue until the hidraw device is opened, e.g. by running hid_listen.

You can clearly see this when attaching the gdb debugger and setting a watchpoint on ch.mainthread.state. Eventually, your main thread will transition into state=4 (CH_STATE_QUEUED, i.e. thread suspended on a queue operation) and never run again, which locks up your keyboard:

Program received signal SIGTRAP, Trace/breakpoint trap.
chSchGoSleepS (newstate=4 '\004') at lib/chibios/os/rt/src/chschd.c:300
(gdb) bt
#0  chSchGoSleepS (newstate=4 '\004') at lib/chibios/os/rt/src/chschd.c:300
#1  0x6000a35e in chSchGoSleepTimeoutS (newstate=4 '\004', timeout=4294967295) at lib/chibios/os/rt/src/chschd.c:389
#2  0x6000a7d8 in chThdEnqueueTimeoutS (tqp=0x20000d18 <drivers+268>, timeout=4294967295) at lib/chibios/os/rt/src/chthreads.c:872
#3  0x6000ac00 in osalThreadEnqueueTimeoutS (tqp=0x20000d18 <drivers+268>, timeout=4294967295) at ./lib/chibios/os/hal/osal/rt-nil/osal.h:831
#4  0x6000b112 in obqGetEmptyBufferTimeoutS (obqp=0x20000d18 <drivers+268>, timeout=4294967295) at lib/chibios/os/hal/src/hal_buffers.c:586
#5  0x6000b244 in obqWriteTimeout (obqp=0x20000d18 <drivers+268>, bp=0x20000aef "f4041\267\201", n=1, timeout=4294967295) at lib/chibios/os/hal/src/hal_buffers.c:737
#6  0x6000957c in _write (ip=0x20000cd4 <drivers+200>, bp=0x20000aef "f4041\267\201", n=1) at tmk_core/protocol/chibios/usb_driver.c:83
#7  0x6000902c in sendchar (c=102 'f') at tmk_core/protocol/chibios/usb_main.c:935
#8  0x600081b6 in _putchar (character=102 'f') at tmk_core/common/printf.c:27
#9  0x60007592 in _out_char (character=102 'f', buffer=0x20000b94, idx=12, maxlen=4294967295) at lib/printf/printf.c:153
#10 0x600079b8 in _vsnprintf (out=0x60007571 <_out_char>, buffer=0x20000b94 "\f\030", maxlen=4294967295, format=0x600146ac "frequency: %lu\n", va=...) at lib/printf/printf.c:592
#11 0x6000815c in printf_ (format=0x600146a0 "matrix scan frequency: %lu\n") at lib/printf/printf.c:867
#12 0x6000446c in matrix_scan_perf_task () at tmk_core/common/keyboard.c:121
#13 0x6000470c in keyboard_task () at tmk_core/common/keyboard.c:408
#14 0x600091a2 in main () at tmk_core/protocol/chibios/main.c:303

Now, what’s the most elegant way to fix this? I don’t know yet :) Looking at this now.

stapelberg added a commit to stapelberg/qmk_firmware that referenced this issue Apr 3, 2021
Before this commit, attaching an ARM-based (i.e. ChibiOS-based) keyboard that
uses CONSOLE_ENABLE = yes and produces debug messages would deadlock the
keyboard unless one was running hid_listen.

With this commit, dead-locking writes to the queue are detected and prevented.

fixes qmk#5631
zvecr pushed a commit that referenced this issue Apr 10, 2021
Before this commit, attaching an ARM-based (i.e. ChibiOS-based) keyboard that
uses CONSOLE_ENABLE = yes and produces debug messages would deadlock the
keyboard unless one was running hid_listen.

With this commit, dead-locking writes to the queue are detected and prevented.

fixes #5631
@purdeaandrei
Copy link
Contributor

Hello everyone, please review my improved fix for this issue here:
#12576
This current implementation has some issues which are described in my PR/commit messages.

makenova pushed a commit to makenova/qmk_firmware that referenced this issue Apr 26, 2021
…2472)

Before this commit, attaching an ARM-based (i.e. ChibiOS-based) keyboard that
uses CONSOLE_ENABLE = yes and produces debug messages would deadlock the
keyboard unless one was running hid_listen.

With this commit, dead-locking writes to the queue are detected and prevented.

fixes qmk#5631
rizo pushed a commit to rizo/qmk_firmware that referenced this issue May 10, 2021
…2472)

Before this commit, attaching an ARM-based (i.e. ChibiOS-based) keyboard that
uses CONSOLE_ENABLE = yes and produces debug messages would deadlock the
keyboard unless one was running hid_listen.

With this commit, dead-locking writes to the queue are detected and prevented.

fixes qmk#5631
toddyamakawa pushed a commit to toddyamakawa/qmk_firmware that referenced this issue May 19, 2021
…2472)

Before this commit, attaching an ARM-based (i.e. ChibiOS-based) keyboard that
uses CONSOLE_ENABLE = yes and produces debug messages would deadlock the
keyboard unless one was running hid_listen.

With this commit, dead-locking writes to the queue are detected and prevented.

fixes qmk#5631
HokieGeek pushed a commit to HokieGeek/qmk_firmware that referenced this issue Jul 11, 2021
…2472)

Before this commit, attaching an ARM-based (i.e. ChibiOS-based) keyboard that
uses CONSOLE_ENABLE = yes and produces debug messages would deadlock the
keyboard unless one was running hid_listen.

With this commit, dead-locking writes to the queue are detected and prevented.

fixes qmk#5631
BorisTestov pushed a commit to BorisTestov/qmk_firmware that referenced this issue May 23, 2024
…2472)

Before this commit, attaching an ARM-based (i.e. ChibiOS-based) keyboard that
uses CONSOLE_ENABLE = yes and produces debug messages would deadlock the
keyboard unless one was running hid_listen.

With this commit, dead-locking writes to the queue are detected and prevented.

fixes qmk#5631
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.