A couple small observations and questions regarding forward compatibility

[robbert-vdh]

Hi!

I was looking through the APIs (lots of changes since the last time I did that), and everything looks very clean. I just had a couple small notes I wanted to mention. Because who knows, maybe one day I'll have to implement clap support in yabridge. ;) Feel free to not do anything with this if it's not useful or of it goes against clap's design goal.

For future reference, this was written for clap 0.15.0/commit 24a3df6.

• The channel maps currently only support predefined sets of channel layouts. I actually quite like this because 99.9% of the use cases will involve processing either mono, stereo, 5.1, or 7.1 surround audio, but a long-standing complaint for VST3 (and one of the few places where VST2 still offers more functionality) is the inability to support exotic ambisonic setups or other non-standard channel layouts. See VST3 SDK issue #28 and this thread on their forum. I'm not actually sure what the best approach would be where you can have both clap's current simple model and a way for plugins to support exotic channel layouts, but I thought it might be worth mentioning since this is one of the complaints against VST3 clap has not solved yet.

• The second thing I wanted to ask about is forward compatibility. This is a tough one because forward and backward compatibility is often where APIs get most of their complexity from. What is the current plan for extending functionality to an existing interface? For instance, if at some point after clap 1.0's release it's deemed necessary to add another flag or more information to the audio port info, then how will this work? The usual options are to either completely replace the extension by a _v2 version of the extension, to add padding at the end of the structs in the hope that any new fields will fit in there, or to begin every struct with a version field so you can cast the pointer to that struct to a newer version of the struct depending on which version you support. But none of those options are ideal. So I was just wondering, are there already plans for how clap will handle this?

Anyways, I'm excited for clap's future, and I was very happy to see initial clap support added in Bitwig 4.1's betas. The polyphonic automation and modulation alone is going to be a game changer for sure. So let me know what you think!

[mkschulze]

Proper multichannel handling would be nice yes, like 7th order Ambisonics with up to 64 channels. In recent times of 3D Audio and VR/AR are such formats gaining popularity again.

[abique]

Hello!

Thank you very much for this issue.

If we want to complement an interface we can create a new interface which just complements the existing one, like it is working with gui, win32, cocoa, x11, ...

If we want to completely replace an extension, we can also do that. So we have all the options and I think the correct approach needs to be thought for every different case.

Then regarding the channel mapping, there are not a lot of options because I do not have a strong knowledge with all the different standard layouts.

I have no idea how Ambisonics works, but if there are a thousand of possible combinations then it needs another approach.

I imagine something like this:

chmap = CLAP_CHMAP_AMBISONIC;

struct clap_plugin_ambisonic_ports {
   bool (*get_info)(const clap_plugin* plugin, bool is_input, uint32_t port_index, clap_ambisonic_port_info *info);
};

If you can give me more info about ambisonic, maybe we can tackle this.

Cheers!

[robbert-vdh]

I'll try to ask some other people that have actually used ambisonics to chime in, since I too can only make assumptions on what would be a good workflow for custom speaker layouts.

[mkschulze]

I'm not sure if the layouts need to be predifined actually, in the post here the developer just asks for a way to define discrete channels, so that could be enough for the developer to make the plugin compatible to any layout and put it into a config, couldn't it?

[abique]

I'm not sure if the layouts need to be predifined actually, in the post here the developer just asks for a way to define discrete channels, so that could be enough for the developer to make the plugin compatible to any layout and put it into a config, couldn't it?

Then isn't CLAP_CHMAP_UNSPECIFIED already working?
The host knows the channel count but doesn't know the semantic of each channel.

[mkschulze]

If it enables the developer to use this and define own layouts, it could be enough probably. I can also ask around and get some input from more experienced people.

[m-hilgendorf]

There are two problems with Ambisonics, the first is that the channel count may overlap with other exotic setups (eg: quadraphonic and 3.1 with first order, 7.0.2 surround with 2nd order) and the second is that the number of channels doesn't distinguish the Ambisonic encoding. There are multiple Ambisonic encodings of the same order with different channel orderings.

The two most common are ambiX and FuMa. Facebook 360 has a decent description.

The common case isn't really stereo though - it's what some call "multi-mono" (number of inputs == number of outputs, don't care about their semantics). I don't think this API can express that case, currently.

What would be very helpful is not encoding speakers as bitflags like VST3, rather an array of strings describing the channels. Most people do not care about it, for the cases that you do it is a major pain to try and figure out how to communicate the right encoding and incantation to have the host instantiate your plugin.

[abique]

Yeah I was thinking about something similar.
So CLAP_CHMAP_SURROUND and CLAP_CHMAP_AMBISONIC would be opaque; you would know the channel count but not the semantic of the channels themselves.

Then would come two additional extensions, one for surround and one for ambisonic so the host can query the semantic info for each channels.

Would that sound good?

[abique]

This commit 6c158d5 addresses the surround case properly I think.

[m-hilgendorf]

I think for ambisonics you could just assume ACN ordering (what ambiX uses). It would still be possible for a plugin to have a to reinterpret the channel semantics independent of the host, since in practice I don't think the host is going to care that much about it (host authors should probably chime in, but very few of them support Ambisonics at all)

This commit 6c158d5 addresses the surround case properly I think.

Just curious about the preferred idiom - why port index instead of port id?

How should hosts handle plugins that report a different channel ordering than they expect for their i/o, or request if a plugin supports a given channel set?

[abique]

Just curious about the preferred idiom - why port index instead of port id?

We address the port using is_input and index quite often, so it makes it convenient I think.

[abique]

For ambisonic we could get away with a few enums right?

• CLAP_CHMAP_AMBISONIC_FUMA
• CLAP_CHMAP_AMBISONIC_AMBIX

[m-hilgendorf]

That will probably be perfect! But just a small change :

CLAP_CHMAP_AMBISONIC_FUMA, // FuMa channel ordering
CLAP_CHMAP_AMBISONIC_ACN,   // ACN channel ordering

Sorry for the bad jargon - ambiX is a file format that uses the ACN channel ordering. For prior art here's a doc from Wwise (game engine audio/plugins) which supports both

[abique]

Thank you for opening this ticket!
I hope the fix works for you :)

[mkschulze]

next step would be Bitwig gonna approach this somehow, with CLAP plugins, something like this in live:
https://www.envelop.us/software/
https://www.ableton.com/en/blog/free-tools-live-unlock-3d-spatial-audio-vr-ar/
☺️
just to test if it works correctly I mean..