Speaker relative path not working with external controller

[gabryelreyes]

Describe the Bug

Using a external controller: using the Speaker to play a WAV file that is found relative to the controller executable results in the file not to being found.

This should be possible according to the documentation: https://cyberbotics.com/doc/reference/speaker#description

├── ExternalController
│   ├── ExternalController.exe
│   └── Sounds
│       ├── CoolSound.wav

Root cause

robot->controllerDir(); returns an empty string, as mControllerDir is not updated if the controller is <extern>.
This results in QFile::exists(mControllerDir + filename) returning false for the WAV file.

Steps to Reproduce

1. Create a world with robot
   • It shall have an <extern> controller.
   • It shall have a Speaker device.
2. Write an external controller for the robot.
3. Have the controller play a WAV file with a relative path to the file.
   • Example: pathToFile = "sounds/CoolSound.wav"

Expected behavior

Expected to hear the sound.

Observed behavior

WARNING: DEF ROBOT robot > Speaker "speaker": Sound file 'Sounds/CoolSound.wav' not found. The sound file should be defined relatively to the controller, or absolutely.`

System

• Operating System: Windows 11
• Graphics Card: Nvidia GeForce Mx550

[gabryelreyes]

mControllerDir cannot be updated in the case of external controllers given that it may be located in a remote machine.
The only possibilty for the sounds to reliably work would be for them to be located relative to the world file, as this would ensure the files are in the same machine as the simulation or as URLs to be downloaded.

[omichel]

Otherwise, we can also modify the protocol to send the sound file to the server together with the speaker sound play command.

[gabryelreyes]

I do not know how easy/difficult it is to implement such a feature to handle files that are a couple hundred of kBytes. Caching would be required, as if it was a web resource.

[omichel]

Yes, if implementing such a feature, we should also implement some caching functionality.

[nhjschulz]

Proposal for external controller sound support:

Add a new speaker API to upload sound data to speaker devices from external controller. Speakers will then cache the data and a later call to play the file will work, regardless of what the path was (as long it is the same string used during the upload).

On controller side it will look like this:

... 
const char[] LOCAL_SOUND_FILE="c:/foo/bar/x.wav";

wb_speaker_upload_sound( left, right, LOCAL_SOUND_FILE, balance);
....
wb_speaker_play_sound(... LOCAL_SOUND_FILE, ....);

The upload sound API will stream the data to the Webots instance together with the filename for caching.
This may result in large messages if we embed the content of eventually several wav files inside it.

Will this work or is there a message length limit to consider ?
Is backward compatibiliy between controllers and Webots version an issue if we extend the speaker protocol ?

[omichel]

Yes, that would work. However, I believe we can probably implement this without modifying the API: When passing a sound file to wb_speaker_play_sound, we could simply upload this sound file to the Webots server if it was not already uploaded and play it. This means that we have to maintain, in the libController, a list of sound files which have already been uploaded to avoid uploading them several times.

[nhjschulz]

@omichel Ok, I'll embed the sound data upload logic into existing wb_speaker_play_sound() as suggested above..