[Feature] Augmented remote control using keyboard

[laurensvalk]

Update

If you're reading this in the future, we've now got an updated demo:

Click for updated demo

---

Is your feature request related to a problem? Please describe.

We frequently get feature requests related to remote control of your (partly) autonomous creations. For example, @GianCann and @JorgePe and others have requested this, and more recently @kueden in #153.

@kueden shared a few examples, such as a vehicle that does most control locally, but uses high level user input for direction. The idea was to make this possible for Powered Up hubs too, which is currently not possible:

Suggested solution

While hub-to-hub communication is perhaps still a long way off, I realized we can achieve augmented remote control in a much simpler way. I've just added a new function to try this out. We're curious to hear what you think.

Here's how it works:

from pybricks.experimental import getchar

while True:
    c = getchar()
    if c == ord('a'):
        print('You pressed a! Now drive forwards...')
    elif c == ord('b'):
        print('You pressed b! Self destruct in 3, 2, 1...')

You just "type" it the input window, at the bottom of this screenshot:

You can already try it!
You can try it with this Technic Hub firmware or this City Hub firmware using the instructions in the quick start guide under "saving a program permanently".

Additional details
getchar() is non-blocking. It returns None if no new character is available.

Unlike input() It does not require you to hit enter. All input is buffered (up to a certain size), and getchar gets the oldest character, if any. It also doesn't echo the input back, saving further delays.

Here's another example that makes it echo the input back if you want it:

from pybricks.experimental import getchar

while True:
    c = getchar()
    if c is not None:
        print(chr(c), end="")

The arrow keys also work, but they really consist of three different characters each. You can read them just fine, but it's easier and faster to use other keys. Common direction examples are WASD, or the 4862 keypad keys.

Your thoughts

Does this cover some of your use cases? Is there anything you'd rather see differently? If you tried it, did it work well?

[GianCann]

It would seem enough to be able to communicate some states to the hub
Very, very interesting ...
Unfortunately I will only be able to try it in the next days ...

Thank you @laurensvalk & @dlech !

[JorgePe]

It covers most of my needs for communicating with the hub... now that it doesn't block I can try some kind of coordination or hive.
I don't have my Control+ near me but wil surely try this night or tomorrow.

[laurensvalk]

try some kind of coordination or hive

Cool. For inspiration, we have a few initial examples of running multiple hubs and one central PC here, but we can probably update them now to use this getchar for better performance.

[JorgePe]

it works, it works!!! 👍
(Top Gear Rally Car)

[kueden]

Thanks for this! The eature will definitely take care of my needs. Adding the remote control feature through bluetooth damatically expands the possibilities of the C+.
All your help is highly appreciated.

[laurensvalk]

Thanks everyone for your responses. I think we will merge this.

It will only be available for all Powered Up hubs (not EV3). We'll also keep it in the experimental module for now. This means that we might change the function name or functionality one day, when more generic Bluetooth solutions are available.

[JorgePe]

Made a small program for the Top Gear Rally Car.
Q/A/O/P/0 keys
Using a small wireless keyboard and just the Chrome IDE

from pybricks.hubs import TechnicHub
from pybricks.pupdevices import Motor
from pybricks.parameters import Port, Stop
from pybricks.tools import wait

hub = TechnicHub()
from pybricks.experimental import getchar

mDrive = Motor(Port.D)
mSteer = Motor(Port.B)
mSteer.reset_angle()

SPEED_DRIVE = 100
TIME_DRIVE = 30
STEP_STEER = 15
SPEED_STEER = 720
MAX_STEER = 75  # attention - program seems to crash if STEER angle too large (motor stalls)

mSteer.run_target(SPEED_STEER,0, then=Stop.BRAKE)


while True:
    c = getchar()
    if c == ord('q'):
        mDrive.dc(SPEED_DRIVE)
        wait(TIME_DRIVE)
    elif c == ord('a'):
        mDrive.dc(-SPEED_DRIVE)
        wait(TIME_DRIVE)
    elif c == ord('o'):
        if mSteer.angle() > -MAX_STEER:
            mSteer.run_angle(SPEED_STEER,-STEP_STEER, then=Stop.BRAKE)
    elif c == ord('p'):
        if mSteer.angle() < MAX_STEER:
            mSteer.run_angle(SPEED_STEER,STEP_STEER, then=Stop.BRAKE)
    elif c == ord('r') or c == ord('0'):
        mSteer.run_target(SPEED_STEER,0, then=Stop.BRAKE)
    else:
        mDrive.stop()

[laurensvalk]

Awesome! Would you like to make a pull request over at pybricks-projects to share it?

You could place your script at:

pybricks-projects/sets/technic/42109_top_gear_rally_car/keyboard_remote/main.py

And maybe also add a README to briefly describe what it does and how to use it.

pybricks-projects/sets/technic/42109_top_gear_rally_car/keyboard_remote/README.md

Or if it's more convenient, share the above info in a new issue on pybricks-projects and we will merge it for you.

[JorgePe]

You speak like if I knew how to pull and merge code at github :)

I added two files but pretty sure not the proper way because I had to open 2 pull requests.

[dlech]

FYI, there is an option to upload more than one file at a time:

[laurensvalk]

Thanks everyone for the input and @JorgePe for the pull request. It's been merged.

This issue can be closed.

[kueden]

I am not as quick as Jorge, but I finally found the time to test it on my Time Machine

Indeed it works fine. My problem is softwarewise: if I keep the buttons pressed they are stacking and the motor continues running, possibly very long time. So far I could not figure out how to eliminate that.

If you have an idea, any suggestion is appreciated. It is not urgent. I can handle it for the moment.

Below my posssibly poor or wrong code

thanks again for all the help

while True:
    
    wait(1000)
    c = getchar()
    print(c)
    if c == ord('a'):
        earth.dc(30)
        wait(100)
       
    elif c == ord('d'):
        earth.dc(-30)
        wait(100)
        
        print('You pressed a')
    else: 
        earth.dc(0)

[laurensvalk]

Yeah, that is a good question. It can be done with a bit of extra code. If you're interested, try running the script below to see if it does what you need. This example empties the buffer by reading everything in it, avoiding the "stacking" (buffering) effect that you encountered.

But there is a bit of a limitation with how we do it currently. We don't directly read the keyboard state, but we read what the user "types". To see what I mean, open any text editor and hold one letter key. You'll see one letter appear right away, then a brief pause, and then a bunch more letters appear. So we have to wait at least for that pause to confirm that a key is no longer pressed. This example does that as well.

You'll notice that the R, G, B keys are responsive, but it takes a brief amount of time for the light to turn off after you stop pressing any keys. But you could press an arbitrary key (such as spacebar) to stop quickly.

from pybricks.hubs import TechnicHub
from pybricks.pupdevices import Motor, ColorDistanceSensor
from pybricks.parameters import Port, Direction, Stop, Color
from pybricks.tools import wait, StopWatch

from pybricks.experimental import getchar

hub = TechnicHub()

key_timer = StopWatch()
last_key = None

def get_key():
    global last_key
    
    # Read all available characters, keeping only the most recent.
    char = None
    while True:
        read = getchar()
        if read is None:
            break
        char = read
        
    # There was a new character, so update and reset the timer.
    if char is not None:
        key_timer.reset()
        last_key = char
        return char
        
    # There was no new character, but we'll keep returning
    # the last one for a little while longer, giving time
    # for new characters to come in
    if key_timer.time() < 600:
        return last_key

while True:
    char = get_key()
    
    if char == ord('r'):
        hub.light.on(Color.RED)
    elif char == ord('b'):
        hub.light.on(Color.BLUE)
    elif char == ord('g'):
        hub.light.on(Color.GREEN)
    else:
        hub.light.off()
    wait(10)
    

[kueden]

Thanks for this. It explains why my primitive attempts to empty the buffer failed. I will try to come up with a procedure that is as responsive as possible. I will keep you posted

[JorgePe]

I am making a serious of small demos of this feature using the Top Gear Rally Car.

I start with keyboard (Arrow keys and SPACE) and then will keep extending over it, replacing the keyboard with the Makey Makey (straightforward as it works like a keyboard) then a wireless gamepad (using Antimicro to map some buttons to the Arrows/SPACE keys) and a Xbox Adaptive Controller (same method) and probably something even more awkward that I might find in my gadgets drawer).
Will probably also add one or two examples on attaching directly to the NUS instead of the Pybricks IDE but not sure yet what (Nordic Toolbox UART most probably since it's also straightforward)

Arrow keys are not the best option since they generate 3 codes and break the example Laurens gave for keeping the buffer empty so I tried just a simplified version - most of the time it works but a few times not.

https://github.com/JorgePe/randomideas/tree/master/Pybricks_Getchar_Demo

https://youtu.be/yys5QrDj9A8
https://youtu.be/njr63D6O7Ow

[kueden]

Thanks JorgePe

I followed the suggestions of @JorgePe and @laurensvalk and got it working pretty nicely. I made the following obervations:
It is not possible (at least in my code) to have one key permanently presssed (e.g. for forward) and then press a second key (e.g. for turining right). They override each other.
This is not a problem, but it is not possible to operate it in the Power Functions mode, where the motor is running as long as you press the buttons. It is rather sending the forward command once and it will run until the stop command is received. This allows having multiple functions in parallel.

I had problems with the software crashing. I think this is related to the print command. When I am receiving too frequent print information from the hub it crashes. Taking that command out solves the problem.

This just for your information. I will come back to you as soon as I have a model up and running.

[laurensvalk]

Nice work both!

Yes, there seems to be an issue when printing a lot. I haven't been able to identify it consistently yet. If you can find a reproducable case with a small script, we'd appreciate if you want to share it in an issue.

[kueden]

I added the code that keeps crashing pretty reliably below. Removing the print command resolves the issue. Possibly there are other major coding short commings ;-)

from pybricks.pupdevices import Motor
from pybricks.parameters import Port, Stop
from pybricks.tools import wait, StopWatch
from pybricks.hubs import TechnicHub
from pybricks.parameters import Color
import math
from pybricks.experimental import getchar


mr = Motor(Port.D)
ml = Motor(Port.C)
watch = StopWatch()
hub=TechnicHub()
x=0
y=0
t0=0



while True:
    c = getchar()
    if c == ord('w'):
        x=x-1
    elif c == ord('r'):
        x=x+1
    elif c == ord('o'):
        y=y+1
    elif c == ord('l'):
        y=y-1
    
        

    lis=[x,y]
    print(lis)
    ml.dc(x)
    mr.dc(y)
    wait(1)
   
        

test.txt

[kueden]

Gents

I finally found the time to play with the remote control. I think the code below works well and is sufficient responsive. It requires indivicual key strokes and thus prevents any function blocking caused by continuously pressed keys. Continously pressed keys prevent the reading of new keys being pressed. The code can be used on any car with one motor for driving and one motor for steering. It has the following features:
Self calibration and centering of the steering upon startup of the software
a: drive backward
d: drive forward
s: brake
j: steer left
l: steer right
k: center steering
repeated pressing of the keys increases or decreases speed and steering angle. The software has some small augmentation functionality that adds more to the care than just simple remote control:
pressing q or e once causes the car to rotate on the spot for automatic turing. Keep the brake key (s) pressed to stop rotation. I will prepare a video and post it. Any feedback or advice for further improvements are more than welcome.

augmented RC_1.txt

[johnscary-ev3]

Hi all,

I am using the code below to read characters from keyboard for remote control on Robot Inventor.
In Pybricks code window it works great and is very useful ;0)

I also tried it under Visual Studio Code environment.
My same program runs with no errors, but it does not seem to get any characters from the terminal window keyboard.
Can this be made to work also?

---

from pybricks.experimental import getchar

# Read all available characters, keeping only the most recent.
    c = None
    while True:
        read = getchar()
        if read is None:
            break
        c = read

[laurensvalk]

Yes and no 😄 .

While pybricksdev is used as a command line tool in #167, it is really also a Python library. So you can make your computer send anything to the hub using e.g. this demo, and use getchar on the hub to process it.

But the convenient input/output window is only available in Pybricks Code right now.

[johnscary-ev3]

Hi, Thanks for the info.

I am trying to get that Demo software you mentioned but once again I came up against Poetry errors that I can't decode.
It seems to be looking for a file in this directory C:/Users/John/AppData/Local/pypoetry/ but directory does not exist on my PC.
Any clues for me?

PS C:\users\john> cd pybricksdev-demo
PS C:\users\john\pybricksdev-demo> poetry install
Installing dependencies from lock file

Package operations: 84 installs, 0 updates, 0 removals

  • Installing ipython-genutils (0.2.0)

  EnvCommandError

  Command C:\users\john\pybricksdev-demo\.venv\Scripts\pip.exe install --no-deps file:///C:/Users/John/AppData/Local/pypoetry/Cache/artifacts/e6/8e/a3/8f37e14310c0072b3fcc4240490bcb42630aa695d069aee89953ebd9f8/ipython_genutils-0.2.0-py2.py3-none-any.whl errored with the following return code 1, and output:


  at ~\.poetry\lib\poetry\utils\env.py:1074 in _run
      1070│                 output = subprocess.check_output(
      1071│                     cmd, stderr=subprocess.STDOUT, **kwargs
      1072│                 )
      1073│         except CalledProcessError as e:
    → 1074│             raise EnvCommandError(e, input=input_)
      1075│
      1076│         return decode(output)
      1077│
      1078│     def execute(self, bin, *args, **kwargs):

PS C:\users\john\pybricksdev-demo>

[laurensvalk]

This might be better as a new issue, or perhaps here (#177), thanks!