Real-time signal processing

[rob-luke]

I would like to do real time signal processing in Julia. By this I mean a replacement for simulink (MATLAB) or LabView (NI). I would like to be able to take measurements, set up a processing chain and output the result.

Reactive looks like a nice framework for this processing. With the previous() command most filtering configurations can be implemented. Ideally this could be integrated with DSP.jl (@simonster, @JayKickliter). Combined with AudioIO.jl (@ssfrr, @wherrera10) for input and output, I have proposed a basic acquisition example in AudioIO issues.

For my uses I need to process data at a minimum of a 1kHz rate. I know the processing time will depending on the complexity. But will Reactive.jl be a limiting factor?

My questions are

1. Has anyone used Reactive.jl in this way successfully? If so, have they documented it?
2. Will Reactive be fast enough for real time processing? I am starting to test this myself, but have no intuitive feeling for the code base yet.

(edit: added processing rate requirements)

[rob-luke]

Here is a basic filtering example using Reactive.jl. It is a low pass filter on some MarketData values. I would like to extend this idea to real-time measurements.

(edit: obviously the backward filter pass wouldn't work in real time)

using MarketData, Reactive, Plots

#
# Set up filter
#

# feed forward coefficients = numerator
b0 = 0.1201
b1 = 0.2403
b2 = 0.1201

# feed backward coefficients = denominator
a0 = 1
a1 = -0.8093
a2 = 0.2898

# Delay line
x_0 = Signal(convert(Float32, BA.values[1, 1]))
x_1 = previous(x_0)
x_2 = previous(x_1)

# Coefficients
x_0o = map(a -> a * b0, x_0)
x_1o = map(a -> a * b1, x_1)
x_2o = map(a -> a * b2, x_2)

z_0 = map(+, x_0o, x_1o, x_2o; init = BA.values[1, 1])  # temp requirement

# Let output settle
for n = 1:10

    # Delay line
    z_1 = previous(z_0)
    z_2 = previous(z_1)

    # Coefficients
    z_1o = map(a -> a * -a1, z_1)
    z_2o = map(a -> a * -a2, z_2)

    # Output
    z_0 = map(+, x_0o, x_1o, x_2o, z_1o, z_2o; init = BA.values[1, 1])
end

#
# Run filter
#

# Forward pass
res = Float32[]
for v in BA.values[:, 1]
    push!(x_0, v)
    sleep(0.0000001)
    push!(res, value(z_0))
end

# Backward pass
res2 = Float32[]
for v in reverse(res)
    push!(x_0, v)
    sleep(0.0000001)
    push!(res2, value(z_0))
end
res2 = reverse(res2)

#
# Plot results
#

#= unicodeplots() =#
gadfly()

o = plot(1:length(BA), BA.values[:, 1],lab="Original")
o = plot!(o, 1:length(res2), res2, c=:orange, lab="Filtered")

ylims!(32, 57)
xlims!(0, 500)
title!("Reactive Based Filtering")

[ssfrr]

You might want to take a look at @shashi's AudioIO PR: ssfrr/AudioIO.jl#28 and the ensuing discussion where we talk about a more Reactive-like API for AudioIO. There are some issues with how to represent stateful processing nodes that neither of us really nailed down solutions to, but I'd definitely welcome new eyes on the problem.

One question is whether latency is a factor for you. For real-time audio processing I'd like to get the latency down below 10ms at 48kHz samplerate, which requires careful design to avoid memory allocations that put pressure on the GC. If worst-case latency isn't as much of a factor for your needs and all you care about is throughput, it's definitely easier to get things working.

My guess is that the requirements for real-time DSP stuff are different enough from the focus of Reactive.jl that they'll remain somewhat separate, but API design cross-pollination is very welcome, as Reactive is super cool.

[shashi]

The main intention of Reactive was for use in GUI libraries for say 60fps updates. Reactive's performance has not been the bottleneck in this use case.

Will Reactive be fast enough for real time processing? I am starting to test this myself, but have no intuitive feeling for the code base yet.

I have not looked into optimizing Reactive for this kind of application, but it is certainly a thing we should try to do. If nothing, we will end up with some optimizations.

@ssfrr If we parameterize audio nodes with phase, maybe we can use Refs to get at a nice solution to this problem.

I believe what @codles is referring to is being able to generate all the 48kHz samples in a Reactive pipeline. The master branch uses closures copiously, which will perform better once Jeff's jb/functions branch gets merged to master Julia.

@codles the global variables in your code will have huge performance impact, so if you enclose the whole snippet in a function, it will perform better

[rob-luke]

Thanks for the feedback. From your comments it sounds like AudioIO is a more a appropriate place for this discussion, I will think on it a bit more then open any issues in AudioIO.jl