Esteso: Interactive AI Music Duet Based on Player-Idiosyncratic Extended Double Bass Techniques

Additional material accompanying the paper "Esteso: Interactive AI Music Duet Based on Player-Idiosyncratic Extended Double Bass Techniques" at the 2024 International Conference on New Interfaces for Musical Expression (NIME 2024).

Domenico Stefani, Matteo Tomasetti, Filippo Angeloni, Luca Turchet

Picture by Alberto Boem

• Performance at the International Conference on New Interfaces for Musical Expression (NIME 2024) at time mark 1:31:00 :
  www.youtube.com/live/Vrywo3fpALw?si=Ztve_z-ukKH0j1zy&t=5454
• Short demo video at https://youtu.be/HEhJXAgFiXM
• Longer demo video here

Repository Structure

• maxpatch/ contains the MaxMSP patches used for the project. More details in maxpatch/README/md.
• SoundDemos/ contains audio samples of the extended double bass techniques and the timbre transfer model.

MAX/MSP Patch

The main patch is maxpatch/main.maxpat

Requirements

• FluCoMa 1.0.6 (download)
• Petra 0.2.2_beta (download)
• nn_tilde 1.5.6 (download)
  To properly install nn_tilde (at least on Windows) you should copy the DLLs from the package folder to the directory where the Max executable is (probably C:\Program Files\Cycling '74\Max 8). Video Instructions: www.youtube.com/watch?v=zzSxvyVvEx8

Max Patch - Usage Instructions

1. Unzip the archive maxpatch/contrabassmoretech.zip and add the folder to Max's paths (Options > File Preferences)
   or copy the extracted contrabassmoretech.ts to the nn_tilde packages folder (C:\Users\cimil\Documents\Max 8\Packages\nn_tilde).
   contrabassmoretech.ts contains the rave model trained on double bass.

2. Prepare the setup:

   1. Plug your computer in for better performance,
   2. Set your interface's buffer size to the highest value (we used 2048),
   3. Set the samplerate to 48000 Hz (Very important as that's how the Rave model was trained),
   4. Use a performant interface.
      We tried Behringers and Focusrites but for some reason, we got severe glitches in the patch output. We ended up using an RME Fireface UFX and later an RME Fireface UFX+, which worked flawlessly.

3. Open main.maxpat and verify that the only errors showing correspond to missing audio files in the two playlists. You can correct these by unzipping maxpatch/knn-wav-dataset/knn-wav-dataset.zip and dragging the unpacked audio files one-by-one to the corresponding names in the two playlists showing in the main patch.
   

4. Activate the green toggle to start the performance mode of the patch.

5. Choose mode:

   1. If you want to test the system with recordings, click play on any playlist recording and pause after a few seconds. The orange mode display should go first from Idle to recording, and after stopping it should change to Playback (given about a second of silence with dB under the selected silence threshold).
   2. If you want to use the system with a real double bass you should close presentation mode, unlock the patch and look for the adc~ object. Once found, change the input channel to the desired in. The default is 9 because we used an RME UFX audio interface.

Contact domenico [dot] stefani [at] unitn [dot] it if you have issues with the patch. Alternatively, find my contacts at domenicostefani.com.

Additional Material

Esteso's System Architecture

Extended Double Bass Techniques

• "Brushed" Jeté: Jeté (or Ricochet technique) with brushed characteristics, introduced with careful use of bow actions and angle.
• Sfregato con legno: Open strings are struck and the wood part of the bow is slid along the string from the bridge, barely making contact with it. It introduces a peculiar buzzing sound.
• Percussive: Collection of different percussive hits on the instrument's body. The hand parts used are fingertips, palms, knuckles, and closed fists. The parts hit are the wooden body, plastic tailpiece, and fingerboard.

High-quality samples of the double bass extended techniques chosen from the personal vocabulary of the musician can be found in the SoundDemos folder.
Here are some audio extracts of the musician playing with the three techniques:

Brushed Jetè

jete-brushed-dry.mp4

Sfregato con Legno

sfregato-con-legno-dry.mp4

Percussive

percussive-dry.mp4

Timbre-transfer model

We employed a RAVE timbre transfer model trained on double-bass sounds from the OrchideaSOL dataset.
We precisely chose to use it as it produced a peculiar hybrid double-bass sound when processing real double-bass recordings.
Samples of dry and processed sounds (only through the custom RAVE model) for each technique are present in the SoundDemos\timbre_transfer_demo_extended_techniques folder.
Quick lower quality versions are available below:

Dry Brushed Jetè

jete-brushed-dry.mp4

Brushed Jetè through RAVE

jete-brushed-wet.mp4

Dry Sfregato con Legno

sfregato-con-legno-dry.mp4

Sfregato con Legno through RAVE

sfregato-con-legno-wet.mp4

Dry Percussive

percussive-dry.mp4

Percussive through RAVE

percussive-wet.mp4

Note that the sounds heard on other demo videos are not only the result of manipulation using the RAVE model, but also the use of granular synthesis and reverb, as described in the paper.

---

Domenico Stefani, Matteo Tomasetti, Filippo Angeloni, Luca Turchet