[Feature] Powered Up Remote Control

[laurensvalk]

We are considering adding support for this device. The proposed API is published here. Please let us know what you think!

In essence, you would initialize and use it just like most sensors. Pairing happens in your script; no manual steps required. Specifying the bluetooth address is optional.

from pybricks.pupdevices import Remote
from pybricks.parameters import Button, Color

remote = Remote()

while True:

    if Button.LEFT_UP in remote.buttons.pressed():
         remote.light.on(Color.RED)
    else:
        remote.light.off()

Does this meet your use case? Are there other features you would like to see?

As always, upvoting this post (:+1:) will help us prioritize the features you want most, so spread the word!

[laurensvalk]

Tagging @Nelvarion, @ZPhilo, @Patlepirate01, @JorgePe, @kueden for ideas. Thank you!

[JorgePe]

"Bluetooth address of the remote. If no address is given, it connects to the first available remote that it finds"
For us, knowing the address is easy, but for neophytes it would help to have something that shows all devices nearby (maybe a button in Pybricks IDE is enough but having a "scan" on the API would help on the long run).

Also is there a way to "unpair"? So I can pass my robot from "remote1" to "remote2" if I want to make a game with a team, each player controlling until completing his/her part then passing the "token" to the next player?

[laurensvalk]

Thanks for your response!

Regarding scanning, how about this?

remote = Remote()
print(remote.address)

Now you know the address. So if you have more than one you could then do:

remote = Remote("01:23:45:67:89:AB")
my_other_remote =  Remote("00:11:22:33:44:55")

Also is there a way to "unpair"? So I can pass my robot from "remote1" to "remote2" if I want to make a game with a team, each player controlling until completing his/her part then passing the "token" to the next player?

Thanks for sharing, this is a nice use case. Yes, a disconnect() method would be good.

[Nelvarion]

Also is there a way to "unpair"? So I can pass my robot from "remote1" to "remote2" if I want to make a game with a team, each player controlling until completing his/her part then passing the "token" to the next player?

Wouldn't it be better to handle the turns in the software to avoid the time it takes to reconnect?

[ZPhilo]

What happens with several remotes connected to the same hub? Does it work at all? is it possible for the remotes to be detected in the order they are powered, without messing with the addresses?

[JorgePe]

Also is there a way to "unpair"? So I can pass my robot from "remote1" to "remote2" if I want to make a game with a team, each player controlling until completing his/her part then passing the "token" to the next player?

Wouldn't it be better to handle the turns in the software to avoid the time it takes to reconnect?

Probably better in this quick example but I'm sure there will be cases where one might need to make the switch, Also instead of "one robot many remotes" it could also be useful for "many robots one remote" (need to think a good particular case but right now can only think on a robot controlling other robots without a proper messaging protocol... yes too clumsy)

[laurensvalk]

is it possible for the remotes to be detected in the order they are powered, without messing with the addresses?

Sure, if you turn on one remote at a time, you could do it like this. You could set the light as a way to tell you when to turn on the next one.

from pybricks.pupdevices import Remote
from pybricks.parameters import Color

philo1 = Remote()
philo1.light.on(Color.GREEN)

philo2 = Remote()
philo2.light.on(Color.GREEN)

philo3 = Remote()
philo3.light.on(Color.GREEN)

What happens with several remotes connected to the same hub? Does it work at all?

No promises yet, but technically that should be possible. Usage could look like the script above.

[laurensvalk]

many robots one remote

This could be interesting for the Liebherr, but we don't know yet if this can technically be done.

But indeed as you say, this could work: remote <---> hub1 <---> hub2

[laurensvalk]

Good news. I was doing some experiments, and all buttons can be independently read.

Also "many robots one remote" may actually be possible. It does not seem to refuse a new connection when a connection already exists.

I suppose @Nelvarion can already use it --- I'm connecting to the remote with a Jupyter Notebook for now to see what messages it sends 😄 Should I push it?

[JorgePe]

Should I push it?

Any doubts? :D

A little out of scope: if you can connect to the Remote... you can also connect to pretty anything BT right? Like the EV3 or even the NXT... or even our own gadget as long as we offer the proper BT service. Not for now... but can a GeneralBTDevice be considered in the roadmap?

[laurensvalk]

Right now I'm still connecting from my PC to the remote, to figure out what the remote sends when you push a button.

Next step is to make the hub do it, but @dlech said that part was easy (:sweat_smile:).

if you can connect to the Remote... you can also connect to pretty anything BT right?

Yes, though most hubs speak only BLE. If we ever enabled classic on SPIKE you can use it as a bridge to talk to EV3. 🤔

[laurensvalk]

Should I push it?

Any doubts? :D

Here you go.

[JorgePe]

don't forget that ev3dev can talk pure BLE - on EV3 with a USB dongle, on a BrickPi directly with the onboard BT chipset... until now I only used ev3dev as a GATT client but a couple of years ago I read something about creating our own BLE GATT services on a Raspberry Pi (doing it on the EV3 will probably be pushing it too much)

[littldr]

Huge upvote for this from my side! I'm super excited to see this implemented!

It would allow me to not only build models together with my son (4), but also to allow him to easily interact with these models. :)

After scanning through the API docs I have the following questions:

• Has Button.LEFT_UP the same value as Button. LEFT_PLUS? The remote has a plus and minus button on each side, and if one is up or down can be changed.
• What about the red button in the middle of each side? I would assume it would be called Button.LEFT_CENTER or something like that? Or I am missing here something?
• Can the green button in the middle also be read? Or is changing the color (like in the original firmware) something the remote does on it's own and not via the hub?
• What would happen if the timeout is exceeded? Is the whole script stopped or a catchable exception raised? Or is the remote.address simply empty?

You do not need to answer these questions, but maybe it helps to find blindspots in the api docs or design. :)

[laurensvalk]

Huge upvote for this from my side! I'm super excited to see this implemented!

That's good to hear --- makes it more interesting for us to implement it too :)

Has Button.LEFT_UP the same value as Button. LEFT_PLUS?

The Button constants do not really have a value; Button.LEFT_PLUS is just the "entity" that is that button. There's a few numbers in the fake API, but they are not shown in the docs since they don't really mean anything. We should probably remove these.

What about the red button in the middle of each side? I would assume it would be called Button.LEFT_CENTER or something like that? Or I am missing here something?

I was thinking to just keep those as Button.LEFT and Button.RIGHT so we didn't have to introduce more. In any case, I'm planning to add a diagram like this to remove all doubt.

Can the green button in the middle also be read?

Yes 🎉 The only catch is that if you press it too long (> 3--4 seconds) it turns off.

Or is changing the color (like in the original firmware) something the remote does on it's own and not via the hub?

The color can also be set, see the code snippet in the original post for the suggested API.

What would happen if the timeout is exceeded? Is the whole script stopped or a catchable exception raised? Or is the remote.address simply empty?

I think we will raise a TimeoutException, which we can add to the docs indeed, thanks!

[kueden]

I fully support this. Most users are into remote control. It will give them easy access to pybricks if you provide a simple way of remotely control. Personally I prefer physical buttons to virtual ones on a mobile phone. So this tool will allow to combine any model with physical tools that are even on a Lego controller (rather than ps4) keeping the purists on board.

I would really appreciate to see this implemented.

[joosbuijsNL]

I would also really appreciate to see this implemented. This would allow builders to get from behind their laptops and control their builds physically and interact with it more. I can already see several applications!

Please let me/us know how the community can help, other than cheering on the side of course :D

[laurensvalk]

Please let me/us know how the community can help, other than cheering on the side of course :D

Cheering is good, sponsoring is even better 😄.

[mwinkler]

sponsoring is even better 😄.

Done :)

Really keen to see this feature running on the hub!

[laurensvalk]

Thank you, much appreciated!

[laurensvalk]

And @joosbuijsNL and others, another great way to help is to test the latest releases to help us find (and maybe even fix) bugs. We just released a new firmware yesterday, which you can get via Pybricks Code. If you still have something built, running your existing code with new firmware is a nice way to verify that we didn't break anything.

Another great help is to share your example code for official sets (e.g. Technic) at Pybricks Projects so others can try it too.

[TheVinhLuong102]

@laurensvalk, do you mean the support for this particular Remote Control is in the latest firmware build already? If so, Antoni and I can test its working with the SPIKE/Inventor Hub. :)

[joosbuijsNL]

@laurensvalk Of course, would be more than happy to help. Although currently I'm in the state where I can mainly test the documentation by trying to follow it ;)

Another point, as indicated by @TheVinhLuong102 's question: how can we know when a new firmware version has been released, what has been changed or added, and how can we signal that everything seems ok (because no bug reports could also mean we just didn't test).

[laurensvalk]

@joosbuijsNL

how can we know when a new firmware version has been released, what has been changed or added

There is a pinned issue #48 with release announcements. I like your idea of keeping tracking that you’ve tested it. Not sure what the best format is to do that. Maybe give the post in #48 a 🚀 ? Or a new issue?

@TheVinhLuong102, no, not yet. This post was to gather ideas from users for features, and to get an idea of how important this feature is to all of you. We might be doing more posts like these to help us prioritize new features.

But it is probably fair to say that this one is very popular.

[Vinz1911]

Also from my side, the remote works great with the Technic hub :) For me one of the best Pybricks Features!! It makes big fun to play around

[dlech]

@JorgePe we should start a new issue for this since it isn't directly related to the remote control.

[laurensvalk]

Should we consider one hub - multiple remote controls scenario? What are real-world used cases? How many people would actually do this?

A single remote might be fine, especially for now. If we add ever some form of simplified, generic LWP3 support, people can always add more remotes that way.

How should we handle disconnection (out of range) and reconnection? How often does this actually happen in real life?

Additionally, there is the use case of wanting to disconnect explicitly, which would be useful.

That could provide an easy way to operate machines like the Technic Liebherr excavator without having to bother with hub-to-hub communication.

For example, you could program it to disconnect when you click the center button. The other hub could be scanning the whole time, and then take over when the remote is available. And vice versa.

[pqhf5kd]

I'm using https://beta.pybricks.com/ and I've installed the (I assume) latest firmware though the interface.

When trying the Remote example code, the hub LED begins to pulse and the remote LED blinks white, but it doesn't appear to connect and then I get
Traceback (most recent call last): File "main.py", line 6, in <module> OSError: [Errno 116] ETIMEDOUT

I'm using the City hub by the way

[laurensvalk]

The hub begins searching for the remote when your script creates the remote object:

my_remote = Remote()

If it does not see the remote within 10 seconds, you get the time out error. So, make sure the remote is on during that time.

We're making the timeout configurable, so you can choose to wait more than 10 seconds.

[pqhf5kd]

I press the "Download and run this program" button and the green button on my remote at the same time. The remote blinks white and the hub pulses but it doesn't appear to connect. The remote stops blinking roughly the same time as the timeout.

[TheVinhLuong102]

@dlech @laurensvalk: @AntoniLuongPham and I are super thrilled to report that the PUP Remote works marvelously well and enabled us to build & perform these for my wife's birthday this very weekend! <3

Thank you a lot for the brilliant work!!

[laurensvalk]

Amazing!

[JorgePe]

I have an automatic lighter using EV3 and plain python... I think I will convert it to pybricks and add some wheels to complete the set... :D

[ZPhilo]

Another one...
https://www.youtube.com/watch?v=igGqzSjl0U0
Tested with an alternate model of 42114, but since LEGO app is compatible with both, the program should work with official model too.
Uses manual gear shift (red buttons), and toggle between driving and blade lifting modes using green button.
I'll share program after a bit of cleaning...

[Tcm0]

I made simple programs for most of the official lego models: https://github.com/Tcm0/PybricksRemoteLayouts. Sadly I can't test some of the programs because I don't have the sets.

[dlech]

Suggested API change: separate connect() method.

Laurens and I have both gone back and forth on this quite a few times because there is not an obvious "this is the right way to do it", but having thought about it a bit, here is one last case for having a separate method:

1. PRO: It allows the possibility of adding wait=False (or async/await) in the future. Connecting is currently blocking, so if you tried to (re)connect the remote of a balancing robot, for example, it would most likely fall over.

   This is probably only important if we care about allowing the user program to disconnect/connect while a program is running. Advanced users might like to do this, but for beginning users, it might make more sense to just automatically reconnect in the background (much easier said than done, though).

2. OPINION: Having the constructor connect to the device breaks our coding pattern of # create your objects here.

   # create your objects here
   hub = SomeHub()
   sensor = SomeSensor()
   remote = Remote()

   This works, however, we usually want to give feedback to the user that the program is waiting for the remote to connect.

   # OK
   
   hub = SomeHub()
   sensor = SomeSensor()
   
   print("connecting...")
   hub.light.blink(...)
   remote = Remote()
   print("connected.")
   hub.light.on(...)

   # Better (IMHO)
   
   hub = SomeHub()
   sensor = SomeSensor()
   remote = Remote()
   
   def connect_remote():
       print("connecting...")
       hub.light.blink(...)
       remote.connect()
       print("connected.")
       hub.light.on(...)

   # Yikes! (IMHO)
   
   hub = SomeHub()
   sensor = SomeSensor()
   remote = None
   
   def connect_remote():
       global remote
       print("connecting...")
       hub.light.blink(...)
       remote = Remote()
       print("connected.")
       hub.light.on(...)

3. CON: Adding an extra method increases the firmware size. Even more so if we need an is_connected() method.

[dlech]

On a related note, we are pretty sure we can live without a disconnect method (as discussed in #404) since it appears to be somehow possible to connect one remote to two hubs (we haven't figured out how this works yet but someone sent us a YouTube link showing this with the official LEGO firmware - don't remember where that was).

[Vinz1911]

I think having a dedicated connect() method is absolutely fine. I mostly write code object oriented and i used your Yikes! solution for my prime hub control class 😆. Does it increase the firmware in a way which is relevant?

And yes I'm also absolutely fine with not having a disconnect() method 👍

[dlech]

Does it increase the firmware in a way which is relevant?

I'm not sure I understand the question. Are you asking about the proposed connect() method or the Yikes! example?

[Vinz1911]

Sorry, it was a bit unlucky 😅.

The question was about your contra point. The Firmware size increase for the dedicated connect method

[dlech]

The question was about your contra point. The Firmware size increase for the dedicated connect method

Each python function typically requires 50 to 100 bytes of flash space. It really wouldn't be a concern but the Move hub only has a few kB free which would be nice to reserve for user programs.

On a related note, we are pretty sure we can live without a disconnect method (as discussed in #404) since it appears to be somehow possible to connect one remote to two hubs (we haven't figured out how this works yet but someone sent us a YouTube link showing this with the official LEGO firmware - don't remember where that was).

I found https://bricks.stackexchange.com/a/10745/3498 that explains how the process works a bit better. With the help of a Bluetooth sniffer, we can see that hub 1 (central) connects to the remote (peripheral), then hub 1 (central) connects to hub 2 (peripheral). So the remote is only connected to one hub and the message is relayed that way.

So there doesn't seem to be any special feature that we missed.