Multithreaded rendering

[psuskin]

I have 4 rendering processes that I would like to run at the same time. However, when I try this with python's threading module, only one render process is completed, after which my script abruptly comes to an end. Is there any reason why multithreading wouldn't be possible? Has someone implemented it before?

[DBraun]

I’m going to test this more once I update libfaust, which has been taking a while. Does your code work if the number of threads is limited to just one? Could you share an MRE to help us get started debugging? I’m also curious if it helps to use the multiprocessing module with Queues. I think there are subtle differences with the threading module.

[psuskin]

Thanks for the quick reply. I created a (not-so-minimal) MRE that I'd like to share. In this example, I'm trying to render a synthesized midi for drums and render a loop with effects for the melody asynchronously. However, the script gets stuck after only having printed "Rendering synth...".
For reference, the code runs fine if only one of the threads executes functions pertaining to rendering (for example, the commented out t2 thread that operates on dummyFunc). Additionally, I tried an implementation using concurrent.futures, but the result is exactly the same. I'll gladly try out the multiprocessing module with Queues next, thanks for the idea.

import dawdreamer as daw
from scipy.io import wavfile

import numpy as np

import threading
import concurrent.futures

# GLOBAL PARAMETERS
BPM = 160

# Static parameters
SAMPLE_RATE = 44100
BLOCK_SIZE = 512

Sitala = "C:/Program Files/Steinberg/VSTPlugins/Sitala.dll"
Kotelnikov = "C:/Program Files/Steinberg/VSTPlugins/TDR Kotelnikov.dll"

stems = {}

def render(synthPlugin, midi, name):
    engine = daw.RenderEngine(SAMPLE_RATE, BLOCK_SIZE)
    engine.set_bpm(BPM)

    synth = engine.make_plugin_processor("synth", synthPlugin)

    synth.load_midi(midi)

    graph = [(synth, [])]
    engine.load_graph(graph)

    print("Rendering synth...")
    engine.render(10)
    print("Finished rendering synth.")

    output = engine.get_audio()

    stems[name] = output

def effect(plugin, audio, name):
    engine = daw.RenderEngine(SAMPLE_RATE, BLOCK_SIZE)
    engine.set_bpm(BPM)

    playbackProcessor = engine.make_playback_processor("playback", audio)
    
    effect = engine.make_plugin_processor("effect", plugin)
    effect.set_parameter(0, 0.2)

    graph = [(playbackProcessor, []), (effect, ["playback"])]
    engine.load_graph(graph)

    print("Rendering effect...")
    engine.render(10)
    print("Finished rendering effect.")

    output = engine.get_audio()

    stems[name] = output

def dummyFunc():
    print("I'm just a dummy.")

def multithread():
    drumsMidiFile = "MIDIs/THH - 160 BPM - 4 Bars - Bass Pattern 1 - Main 1a - Abmaj_Fmin - 6666666666666666.mid"

    _, melodyData = wavfile.read("Samples (Time Based)/Melodies/THH - 160 BPM - 4 Bars - Melody Loop 1a - Cmaj_Aminor - 6666666644444444.wav")
    melodyData = np.array(melodyData, dtype=np.float32).T / 2 ** (16 - 1)
    
    t1 = threading.Thread(target=render, args=(Sitala, drumsMidiFile, "drums"))
    t2 = threading.Thread(target=effect, args=(Kotelnikov, melodyData, "melody"))
    #t2 = threading.Thread(target=dummyFunc)

    t1.start()
    t2.start()

    t1.join()
    t2.join()
    
    
    """
    pool = concurrent.futures.ThreadPoolExecutor(max_workers=4)
    pool.submit(render, Sitala, drumsMidiFile, "drums")
    pool.submit(effect, Kotelnikov, melodyData, "melody")

    pool.shutdown(wait=True)
    """

    if "drums" in stems:
        wavfile.write("multithreadDrums.wav", SAMPLE_RATE, stems["drums"].T)
    if "melody" in stems:
        wavfile.write("multithreadMelody.wav", SAMPLE_RATE, stems["melody"].T)

multithread()

[psuskin]

I just wanted to update you with my current workaround. Even after making changes to the source code to allow asynchronous rendering, python multithreading was just as fast as no multithreading at all. For this reason, I created a simple multithreaded render function in C++ and bound it to python. The function accepts the engines to render, as well as the corresponding durations, and renders each engine asynchronously via C++ multithreading. This worked like a charm, but I wouldn't portray it as the cleanest possible solution, to say the least. Just thought I'd let you know.

[DBraun]

Thanks. It's good to know that's a solution. I think the next step would be to have something that doesn't change the API at all. And the next after that would be to locally hear a graph while the editor is open.

[bnku]

Hello, @psuskin! Could you please share a more detailed description of your implementation? I am struggling with the same problem right now. I would greatly appreciate it!

[psuskin]

Gladly @bnku. I’m a bit busy at the moment, but if you want to get started, you can take a look at my fork. The commits are pretty telling with respect to the multithreading implementation.

[psuskin]

Did the commits clear everything up @bnku? Just in case they didn't, these two commits should do the trick:
Fix for multithreaded rendering in general: psuskin@2d3aaa8
Custom multithread class implementation: psuskin@5bdb09e

[DBraun]

For those subscribed here, we now have two examples for multiprocessing synthesizers:

https://github.com/DBraun/DawDreamer/tree/main/examples/multiprocessing_plugins (made for .fxp presets such as Serum)
https://github.com/DBraun/DawDreamer/tree/main/examples/xml_synth_sound_rendering (made for XML presets for TAL-U-No-LX)