-
-
Notifications
You must be signed in to change notification settings - Fork 10.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Relay touchscreen / pen input from Android screen back to PC (turn Android device into a pen tablet for PC) #2312
Comments
Windows Digitizer Class Input Devices (drivers) are documented here: https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/windows-digitizer-class-input-devices |
E.g. for the case of Windows, Scrcpy could be loading a new Pen HID device driver (bundled with Scrcpy package), passing input captured from the Android screen / device and received from ADB, to the new HID driver. Because it is a standard Windows device driver dispatching input, applications like Inkscape, Adobe Illustrator, Microsoft Whiteboard, could then leverage these new input events. |
Hi, IMO, this is out of scope for scrcpy, which allows to control the device from the computer (not the other way around). Technically, it's also quite different (capturing device events, no video stream, injecting system-specific events to the computer, etc.), so there would be no benefit to implement it in scrcpy rather than a totally separate app. |
It is a very useful feature and Scrcpy has all the foundation in place to implement this as an additional extension. Please consider adding it. Windows Pen protocol implementation is documented here: https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/pen-protocol-implementation, and likely an implementation as a 3rd-party HID mini-port driver would be needed, that is to translate the received events from Android to the Windows' HID device class. |
E.g., using the getevent tool available on most Android devices, https://source.android.com/devices/input/getevent, can be executed on the Android target by Scrcpy
and events retrieved by ADB then translated to native events, e.g. on Windows platforms, using a new Pen HID mini-driver or using SendInput() Win32 API (could be easier). |
@matulek I really hope this gets implemented in scrcpy. |
@ilyesgouta after fiddling a little with scrcpy I realized that it doesn't allow to use android as the external monitor, so, after all, this feature unfortunately seems out of scope to me. |
Is your feature request related to a problem? Please describe.
The Android device (phone or tablet) screen could act as a pen or digitizing/display tablet (e.g. see wacom cintiq or intuos) to capture pen input (moves, press/release, strokes) to aid in drawing or sketching activities on the PC.
Describe the solution you'd like
Scrcpy to forward input events captured on the Android device to the Windows / Linux / Mac host. Input events here are gestures and interactions captured from the Android device touch screen. Ideally the solution wouldn't incur significant latency between the capture of the event and its restitution on the host computer. E.g. for a minimal latency implementation, a virtual Pen driver could be implemented for Windows host, so that input events from the Android device could become available to all user apps running on the Windows host (and uniformly handled by the Win32 API).
Describe alternatives you've considered
Other solutions include using a client/server approach, which could also work by injecting captured input from ADB device into the host PC (Win/Linux/Mac).
Additional context
Having the feature implemented in Scrcpy would be wonderful, as Scrcpy is an open source program, accessible to everyone on all major platforms.
The text was updated successfully, but these errors were encountered: