Request: Custom keyboard mapping

[BrockBrinkworth]

Really want custom mapping keyboard controls. I have seen another post about this and someone said it would be too much work but the thread is closed. Keyboard mapping would be amazing :)

[rom1v]

Ref: #149 #505

someone said it would be too much work but the thread is closed

Could you reference the thread?

[BrockBrinkworth]

#149 I THOUGHT the bit that said closed meant it was closed... its another thread u linked under it.
How hard is it to make custom buttons like emulators? Because scrcpy would be better than any emulator if it had this mapping. Emulators run like trash compared.

[BrockBrinkworth]

Is anyone still working on scrcpy? If so is this on the road map?
Or is there another program or a way to do this myself?

[rom1v]

Is anyone still working on scrcpy?

Yes, on my free time.

If so is this on the road map?

TBH, not really, unless someone wants to work on it (define how to configure key mapping and implement it…).

[BrockBrinkworth]

How the user would configure it: opens an overlay(i.e. steam overlay) and clicks where they want a button and assigns a letter to that button, making that letter click at that spot.

[BrockBrinkworth]

These 2 images show an overlay of Nox Player, pretty much what i am talking about.

[Mehardeep]

hi , am working keyboard mapping for pubg , but i stuck at camera movement (mouse movement)
like ctrl+alt to hold and release the mouse
anyone plz help??

[BrockBrinkworth]

On this image you can see the writing in the black box, this is what your talking about i am pretty sure. Pretty much i think it locks the mouse in that position and hides it so when you move the mouse only the screen moves and not the cursor. Message again with progress if you can.

[Zireael07]

For my use case, that is controlling a racing game, I was able to workaround this using a bit of Python scripting (I used pynput as pyautogui and keyboard both require running as sudo under Linux).

#!/usr/bin/python2.7

from pynput.mouse import Button, Controller
from pynput.keyboard import Key, Listener

global MOUSE 
MOUSE = Controller()

global LISTEN

global CENTER 
global OFFSET 
global LEFT
global RIGHT
global HEIGHT

# for scrcpy with -2M switch
#HEIGHT = 700 CENTER = 900 OFFSET = 500
# HEIGHT = 500 OFFSET 450 for 1024 -b 4M

# -m 768 -b 4M gives good enough quality while still working (-m 640 doesn't register clicks)
HEIGHT = 650 
CENTER = 950 
OFFSET = 300


LEFT  = CENTER - OFFSET
RIGHT = CENTER + OFFSET

def cust_click(x,y):
    MOUSE.position = (x,y)
    # MOUSE.click(Button.left)
    MOUSE.press(Button.left)
 
def space():
    cust_click(CENTER,HEIGHT+120) # 150 for -m 1024

def left():
    cust_click(LEFT, HEIGHT)

def right():
    cust_click(RIGHT, HEIGHT)

def on_press(key):
    #print('{0} pressed'.format(
    #    key))

    if key == Key.left:
        left()
    elif key == Key.right:
        right()
    elif key == Key.space:
        space()
    elif key == Key.tab:
        LISTEN.stop()


def on_release(key):
    #print('{0} release'.format(
    #    key))
    if key == Key.right or key == Key.left or key == Key.space:
        MOUSE.release(Button.left)
    # can't use esc because it is the phone 'back' hotkey, at least in snap version
    # if key == Key.esc:
    #     # Stop listener
    #     return False


# Collect events until released
with Listener(
        on_press=on_press,
        on_release=on_release) as LISTEN:
    LISTEN.join()

[anish6400]

Zireal please help me how can i use this code?

[Zireael07]

@AnishKumar0285: You need pynput installed (pip install pynput) and you need to run the script while having the scrcpy open. You'll need to adjust the keys and positions for the game you want to control, obviously, as well as the scrcpy screen's size.

[sukisux]

@AnishKumar0285: You need pynput installed (pip install pynput) and you need to run the script while having the scrcpy open. You'll need to adjust the keys and positions for the game you want to control, obviously, as well as the scrcpy screen's size.

Hello but i have a question i use your script but how can i add new keys and the key when im playing a shot game thet hide the cursor for use only the movement of the mouse, Thanks

[sukisux]

Anyone knows how to add a keys like the a or s key on pynput?

[BrockBrinkworth]

I would also like to know how to properly change and add to this code. I am trying to make it for a shooter game (pubg mobile).

[Zireael07]

Pynput has very good documentation, it should tell you everything you two want to know: https://pythonhosted.org/pynput/index.html

The mouse positions, as I said, have to be eyeballed.

[srevinsaju]

Fixed it in guiscrcpy, https://github.com/srevinsaju/guiscrcpy/blob/master/guiscrcpy/mapper.py
It is up on v2.0 pip3 install guiscrcpy
It uses pynput, but not as UI intuitive as nox

[BrockBrinkworth]

Thanks, i will try this.

[karousn]

I think, it will be better to have a file external yaml/json from the project (the project just give us an example for the format, like what guiscrcpy do for example) and externalize the work of input_manager.c to do the matching between the event and the correspond on SDL2 action.

[srevinsaju]

Yes @karousn, the only drawback for the guiscrcpy mapping is that it conflicts with the input of characters into a text box, using a mapper key such as Ctrl or Meta may enhance scrcpy 's mode of action. Guiscrcpy manually gives adb shell input and toggles the key position, which is slower than scrcpy's mastermind input method using a tunnel. Will try to implement that into guiscrcpy too :)

[Zireael07]

TC games is windows only, though.

[maximousblk]

If you want an easy implementation, I have a suggestion. I will be really hard for the user to configure, but you can build upon it.

Simplest thing to do is use a config file, preferably json, in which you can map clicks and movements simulation. Why would this be hard? because you have to map the keys and movements using pixels for measurement.

Now if you want a hard implementation but easy for users to configure, here's what you need to do.

1. create a GUI for the whole thing
2. In the gui have a seperate tab for mapping
3. In that tab user would take a screenshot (or live) of the situation which they want it for.
4. Then map the keys according to the screenshot
5. Launch scrcpy and press a special keystroke which would activate the mapping.

Just a suggestion.

this extends #1453

[inxomnyaa]

That repo gives a 404 for me - i wanted to suggest that you check for the name of the app, see https://stackoverflow.com/questions/15992342/how-to-detect-whether-a-process-or-application-is-running-and-terminate-the-prog/16000270#16000270

[renanwp2]

I have uploaded the QtScrcpy binaries on my page. You just need to clone my repository to have it working .This project is based upon scrcpy with "all" wanted features. It should work on all Debian-based systems (Compiled on Debian Buster). If it does not run, try running on terinal ldd QtScrcpy to track possible libraries lacking on your system. I will test this version to see if it's possible to play Free Fire, but supposedly it's possible, after this fix.

P.S.: It's working with Free Fire. I think the fix is fine. Your turn to test it I won't test it anymore so soon.

[nutlost]

I have uploaded the QtScrcpy binaries on my page. You just need to clone my repository to have it working .This project is based upon scrcpy with "all" wanted features. It should work on all Debian-based systems (Compiled on Debian Buster). If it does not run, try running on terinal ldd QtScrcpy to track possible libraries lacking on your system. I will test this version to see if it's possible to play Free Fire, but supposedly it's possible, after this fix.

P.S.: It's working with Free Fire. I think the fix is fine. Your turn to test it I won't test it anymore so soon.

"qtscrcpy" I can create all buttons myself except this one. I can't create move mouse button. I can't create more than one "Smalleyes" button. What should I do?

[MarceloMachadoxD]

guys sorry for asking here but i dont want create a new thread but... did you be able to run mapping keyboard to screenpress positions? i want to configure it to play genshin impact but i really dont understand well the suggestions with phyton theres an resumed area on how to configure it?

[guanzhangrtk]

Perhaps you guys can look at this project. It currently works with Bliss OS (Android-x86) however I don't see how it can't be adapted to real mobile devices...??

[renanwp2]

guys sorry for asking here but i dont want create a new thread but... did you be able to run mapping keyboard to screenpress positions? i want to configure it to play genshin impact but i really dont understand well the suggestions with phyton theres an resumed area on how to configure it?

Please, see the project QtScrcpy. Is it easy to install? Absolutely not. Good luck. On Linux, there is no easy way to play Android games.

[MarceloMachadoxD]

guys sorry for asking here but i dont want create a new thread but... did you be able to run mapping keyboard to screenpress positions? i want to configure it to play genshin impact but i really dont understand well the suggestions with phyton theres an resumed area on how to configure it?

Please, see the project QtScrcpy. Is it easy to install? Absolutely not. Good luck. On Linux, there is no easy way to play Android games.

Thanks so much friend, you're a friend

[king-om]

TBH, not really, unless someone wants to work on it (define how to configure key mapping and implement it…).

may be, we can make a script for the android device to listen to the input through the tunnel or something like that...!
I don't know much but I am a student and learning about development...

[hugonh]

do you guys playing watching by mobile screen or pc's monitor? for me, i cant handle input lag, i tried to manipulate screen resolution with qts and bitrate, but fps drops a lot

[rom1v]

with qts

What is qts?

fps drops a lot

That's not expected.

[luisalvarado]

Sorry for being late at the party here which I find very interesting for mimicking touch movements, dragging and such.

I am using Ubuntu 22.10 and 23.04 with scrcpy 2.0. Which method would be recommended to easily map for example the WASD keys to a touchscreen area for the up, down, left, right movements.

[luisalvarado]

If it helps for research purposes, the xdotool is the one I am using to more or less mimic this:

Currently it is able to drag the center of the virtual joystick (What I call the center position) and drag it to the corresponding X/Y coordinates for the UP | DOWN | LEFT | RIGHT positions. Depending if I hold the SHIFT, it will go longer to any direction to mimic the running speed, or move less to the sides to mimic the stealth/slower movement.

The only problems are the following:

1. If I change the sizes of the window, then I gotta redo the X Y positions for all combinations. I am looking for a way to make the X Y positions proportional to the window size. If genymobile does indeed include this, the window size would need to be taken into consideration when resizing or moving the window around. In the meantime I am thinking this out.

2. An option to drag the touch effect around. With this I mean, If I press the UP + LEFT key , it will click and hold on the center, then drag it to the Upper / Left side. But what happens if I want to change from Up to Right. I would want the effect to start where I left the touch event and then move to the Upper / Right corner of the center position, but without having to release first, then touch the center, then drag to the upper right side. This is because, in-game, the character would pause for a second to accomplish this, instead of naturally moving from left to right.

3. Use the mouse to "look around" without needing to use the mouse to click. So that, the keyboard would move the character around the map, while the mouse would turn the character in any angle.

4. Map the mouse buttons to the action buttons along with mapping keyboard keys to it as well (at the same time), so at any moment I can click the mouse, or press a key and it would execute the action button in game.

This is what I am working on with the xdotool and thinking my way through. The github mentioned above did not help at all btw.

[niksingh710]

If it helps for research purposes, the xdotool is the one I am using to more or less mimic this:

Currently it is able to drag the center of the virtual joystick (What I call the center position) and drag it to the corresponding X/Y coordinates for the UP | DOWN | LEFT | RIGHT positions. Depending if I hold the SHIFT, it will go longer to any direction to mimic the running speed, or move less to the sides to mimic the stealth/slower movement.

The only problems are the following:

1. If I change the sizes of the window, then I gotta redo the X  Y positions for all combinations. I am looking for a way to make the X Y positions proportional to the window size. If genymobile does indeed include this, the window size would need to be taken into consideration when resizing or moving the window around. In the meantime I am thinking this out.

2. An option to drag the touch effect around. With this I mean, If I press the UP + LEFT key , it will click and hold on the center, then drag it to the Upper / Left side. But what happens if I want to change from Up to Right. I would want the effect to start where I left the touch event and then move to the Upper / Right corner of the center position, but without having to release first, then touch the center, then drag to the upper right side. This is because, in-game, the character would pause for a second to accomplish this, instead of naturally moving from left to right.

3. Use the mouse to "look around" without needing to use the mouse to click. So that, the keyboard would move the character around the map, while the mouse would turn the character in any angle.

4. Map the mouse buttons to the action buttons along with mapping keyboard keys to it as well (at the same time), so at any moment I can click the mouse, or press a key and it would execute the action button in game.

This is what I am working on with the xdotool and thinking my way through. The github mentioned above did not help at all btw.

This seems promising can you do one with wtype and put it on git?

[luisalvarado]

Thanks, I will research what wtype is and if it can solve an issue with multiple keys being pressed.

[B4tiste]

How can I make scrcpy use ESCAPE for BACK instead of using the Right Click ?