Streaming mode

[superbock]

This is a stripped down version of @amaurydurand's work (PR #184).

TODO:

• move buffer initialisation to subclassed of BufferProcessor
• reduce CPU load (re-use Filterbank)
• rewrite bin_frequencies as an property
• use float32 instead of int16 dtype
• rewrite all features as processors (mostly madmom.features.onsets.*) (separate issue/PR)
• init Stream with the given arguments if one is given check if the arguments match the ones of the processor
• float hop_size not possible, raise error? (see issue Allow float hop_size for Stream? #249)
• difficult to get diff_ratio right in SpectrogramDifference, find a solution (maybe only allow diff_frames in online mode?)
• check memory consumption
• make block_size configurable (e.g. it could be advantageous to process everything in blocks of 2048 frames instead of whole length pieces) (see separate issue Allow processing in blocks #250)
• fix output writers to work with continuous/steady data (flush more often?)
• recheck cached attributes/variables if they change
• add option to reset processors (e.g. for processing of next song)
• rewrite FilterbankProcessor to process the input and create a Filterbank when called the first time and cache it for consecutive calls (like a buffer) (see separate issue Rewrite FilterbankProcessor #248)
• drop the combined spectrogram processors (FilteredSpectrogramProcessor, LogarithmicSpectrogramProcessor and LogarithmicFilteredSpectrogramProcessor) altogether? (see separate issue Rewrite FilterbankProcessor #248)
• add support to select mono/stereo/left/right channel in Stream (should be handled by Signal, see separate issue add channel selection mechanism to Signal #211)
• check if passing **kwargs to process() methods is needed (related: add **kwargs to process() #33)

[winchell]

Will this make the existing algorithms work with RT audio input?

[superbock]

Yes, exactly. We hope to get this done by end of the year.

[superbock]

Besides the open TODOs below, this PR is more or less ready.

As decided/merged in #244, process() is called the exact same way as before, independently whether a sequence gets processed as a whole or frame-by-frame. Stateful processors accept an additional reset argument, which defaults to True and thus does not change existing behaviour.

As discussed in #246, an additional online processing mode is added which processes live microphone input or files always on a frame by frame basis. The existing single now has the additional --online argument to switch between offline (default) and online parameters. Since single mode always loads the processors, there's no need to reset during operation. On the other hand, this enables us to allow block-wise processing later on (#250).

Open questions / TODOs:

• add option to define length of the Stream to retrieve only a certain number of frames (or specify a length, maybe via stop)
• some processors can not work online, raise error?
• how to signal buffer under-/overflow?
• how to terminate a Stream?
• start and stop attributes of Signal are not used right now, remove them again?

Maybe these can remain open or don't need to be addressed (right now), as this PR handles only the common foundation for various online processing methods (e.g. #238 for online beat tracking).

[superbock]

One more thing to consider: at the moment there is no way to determine whether a processor is online-capable or not.

A previous attempt to solve this used different process methods. E.g. process_sequence() and process_step() for offline/online mode, respectively. However, since normal functions must be usable as processors also, Processors are simply called (like functions), thus there is not really a uniform way to distinguish between online/offline mode for processors.

There's no need to address this within this PR, since none of the added functionality affects this. We should just keep this in mind.

[fdlm]

Great work!

Besides the minor comments, I think the documentation needs to be updated to reflect the possibilities and caveats when doing online processing.

EDIT
Oh, and of course we should create an issue with the open ToDos...

[fdlm]

Is there a reason for this change? I actually think the previous code was more direct and thus easier to understand.

[superbock]

Previously it was not possible to overwrite the parameters, only to pass additional ones to FramedSignal (which in turn got passed to Signal).

This change was necessary in order to be able to pass arguments to process(), which is needed to change the behaviour of processors, e.g. not reset stateful processors when being processed framewise.

[fdlm]

Is this necessary? As far as I understand, the self.spectrogram is a FilteredSpectrogram (and thus has a filterbank), then self.spectrogram.bin_frequencies will return the center_frequencies of the filterbank anyways.

[superbock]

You are right. I think you meant: "if self.spectrogram is a FilteredSpectrogram", though.

[superbock]

I will update the branch to include the suggested corrections and add more documentation before merging.

[superbock]

Previously it was not possible to overwrite the parameters, only to pass additional ones to FramedSignal (which in turn got passed to Signal).

This change was necessary in order to be able to pass arguments to process(), which is needed to change the behaviour of processors, e.g. not reset stateful processors when being processed framewise.

[superbock]

You are right. I think you meant: "if self.spectrogram is a FilteredSpectrogram", though.