Add UBO support to WebGL mode

[RandomGamingDev]

Increasing access

This would allow p5.js to move further in terms of graphical capabilities by allowing for the usage of higher capacity UBOs, which also by design provide better performance,

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build process
• Unit testing
• Internationalization
• Friendly errors
• Other (specify if possible)

Feature request details

Add support within .setUniform() or a new function for UBOs, and preferrably, if possible, for sub buffer updating.

[MuktiMishra]

If we make a function to set setUniform() function to handle UBOs via a new case (gl.UNIFORM_BUFFER) and add a new method setUniformBuffer() to support uniform buffer binding and sub-buffer updates, enabling more efficient updates for large datasets.
The setUniformBuffer() function might use gl.bufferData or gl.bufferSubData to update the buffer data.
We can bound the buffer to the specified uniform block using gl.bindBufferBase ... What's the opinion on this?
@davepagurek can you please guide on this?

[davepagurek]

@MuktiMishra let me know if I'm interpreting your suggestion correctly, it sounds like we'd maybe want to expose helpers to get offsets for properties, make users make their own buffer objects, and allow passing raw GL UBOs into setUniform(blockName, yourUBO). If that's simple to do internally, that could be most worth doing to unblock library authors, since for more normal users getting into shaders, blocks are not going to be necessary for them for a while, and it'd be good to give them a more simplified view of what they can do at first to avoid overwhelm.

Just to think through the scenario where we do try to make this more user-friendly though: Since currently you can't setUniform with an object as the data, maybe we can introduce that as a way to set blocks?

// e.g. uniform myBlock { float prop1; float prop2; }

myShader.setUniform('myBlock', { prop1: random(), prop2: random() })

I think that would mean that under the hood we'd be creating the UBO for you, which is nice if you update everything at once, but less nice if you are regularly only updating bits. An alternative that's a tad heavier to use would help support sub-data would be:

// e.g. uniform myBlock { float prop1; float prop2; }

const block = createUniformBlock({ prop1: random(), prop2: random() })
myShader.setUniform('myBlock', block)

block.setValue('prop1', random()) // Internally marks prop1 as dirty
myShader.setUniform('myBlock', block) // internally uses bufferSubData now

...or one more idea, currently if you have a uniform struct you have to set each property individually. We could use the same API for blocks, too:

// e.g. uniform myBlock { float prop1; float prop2; }

myShader.setUniform('myBlock.prop1', random())
myShader.setUniform('myBlock.prop2', random())
plane() // Buffers the whole block since everything is dirty the first time

myShader.setUniform('prop1', random()) // Internally marks prop1 as dirty
plane() // internally uses bufferSubData now

I'm not 100% sure how to think of that yet, because again it's probably not the bottleneck for most people's shaders anyway. But also depending on where we go with #7188, this could be something we maybe do automatically for you?

@RandomGamingDev maybe you can help add some context on the cases you were imagining to see what direction to go?

[RandomGamingDev]

@MuktiMishra let me know if I'm interpreting your suggestion correctly, it sounds like we'd maybe want to expose helpers to get offsets for properties, make users make their own buffer objects, and allow passing raw GL UBOs into setUniform(blockName, yourUBO). If that's simple to do internally, that could be most worth doing to unblock library authors, since for more normal users getting into shaders, blocks are not going to be necessary for them for a while, and it'd be good to give them a more simplified view of what they can do at first to avoid overwhelm.

Just to think through the scenario where we do try to make this more user-friendly though: Since currently you can't setUniform with an object as the data, maybe we can introduce that as a way to set blocks?

// e.g. uniform myBlock { float prop1; float prop2; }

myShader.setUniform('myBlock', { prop1: random(), prop2: random() })

I think that would mean that under the hood we'd be creating the UBO for you, which is nice if you update everything at once, but less nice if you are regularly only updating bits. An alternative that's a tad heavier to use would help support sub-data would be:

// e.g. uniform myBlock { float prop1; float prop2; }

const block = createUniformBlock({ prop1: random(), prop2: random() })
myShader.setUniform('myBlock', block)

block.setValue('prop1', random()) // Internally marks prop1 as dirty
myShader.setUniform('myBlock', block) // internally uses bufferSubData now

...or one more idea, currently if you have a uniform struct you have to set each property individually. We could use the same API for blocks, too:

// e.g. uniform myBlock { float prop1; float prop2; }

myShader.setUniform('myBlock.prop1', random())
myShader.setUniform('myBlock.prop2', random())
plane() // Buffers the whole block since everything is dirty the first time

myShader.setUniform('prop1', random()) // Internally marks prop1 as dirty
plane() // internally uses bufferSubData now

I'm not 100% sure how to think of that yet, because again it's probably not the bottleneck for most people's shaders anyway. But also depending on where we go with #7188, this could be something we maybe do automatically for you?

@RandomGamingDev maybe you can help add some context on the cases you were imagining to see what direction to go?

Since p5.js is dedicating a lot of its efforts to other things right now (primarily the p5.js 2.0 RFC) and since UBOs are a feature that only complex and really performance hungry shaders require, which really benefit from or even require low level control, I was thinking of a p5.js API that just maps onto the gl.bufferData() and gl.bufferSubData() functions for UBOs specifically (maybe if it expands past that we could even get other low level features like TBOs :D). That way p5.js doesn't spend too much time on something that most people won't use and the people who want a stable and a little simpler API for low level control of UBOs get it.

[MuktiMishra]

@MuktiMishra let me know if I'm interpreting your suggestion correctly, it sounds like we'd maybe want to expose helpers to get offsets for properties, make users make their own buffer objects, and allow passing raw GL UBOs into setUniform(blockName, yourUBO). If that's simple to do internally, that could be most worth doing to unblock library authors, since for more normal users getting into shaders, blocks are not going to be necessary for them for a while, and it'd be good to give them a more simplified view of what they can do at first to avoid overwhelm.

Just to think through the scenario where we do try to make this more user-friendly though: Since currently you can't setUniform with an object as the data, maybe we can introduce that as a way to set blocks?

// e.g. uniform myBlock { float prop1; float prop2; }

myShader.setUniform('myBlock', { prop1: random(), prop2: random() })

I think that would mean that under the hood we'd be creating the UBO for you, which is nice if you update everything at once, but less nice if you are regularly only updating bits. An alternative that's a tad heavier to use would help support sub-data would be:

// e.g. uniform myBlock { float prop1; float prop2; }

const block = createUniformBlock({ prop1: random(), prop2: random() })
myShader.setUniform('myBlock', block)

block.setValue('prop1', random()) // Internally marks prop1 as dirty
myShader.setUniform('myBlock', block) // internally uses bufferSubData now

...or one more idea, currently if you have a uniform struct you have to set each property individually. We could use the same API for blocks, too:

// e.g. uniform myBlock { float prop1; float prop2; }

myShader.setUniform('myBlock.prop1', random())
myShader.setUniform('myBlock.prop2', random())
plane() // Buffers the whole block since everything is dirty the first time

myShader.setUniform('prop1', random()) // Internally marks prop1 as dirty
plane() // internally uses bufferSubData now

I'm not 100% sure how to think of that yet, because again it's probably not the bottleneck for most people's shaders anyway. But also depending on where we go with #7188, this could be something we maybe do automatically for you?

@RandomGamingDev maybe you can help add some context on the cases you were imagining to see what direction to go?

@davepagurek yea you're interpreting it right that we can add helpers functions to get offsets for properties, we'd be creating the UBO , which is quite good if we update everything at once, rather than if we are regularly only updating bits.
But then I think suggestion of @RandomGamingDev sounds pretty cool that p5.js doesn't spend too much time on something that most people won't use and the those who want a stable and a little simpler API for low level control of UBOs can help that out.
@davepagurek what do you think on this?

[davepagurek]

I'm open to that. If anyone is interested in adding this functionality, since we're shifting a bunch of things around, I'd recommend making the PR into the dev-2.0 branch.

Also I wonder if there's a good way in the docs for us to mark things that are useful primarily for library authors but maybe not for everyday users. Looking through the available jsdoc tags, maybe @package (which seems to mean "private except for things operating at the same level as the package itself") is the closest thing to "for libraries"?

[MuktiMishra]

@davepagurek I can try making a PR for this , could you please assign it to me?

And I agree that distinguishing between features for everyday users and library authors in the docs is important and using the @package tag for this purpose seems like a good approach, especially since it fits the need for marking things internal to the package , other than that we can also use @private or custom tags for the same.

[davepagurek]

Thanks @MuktiMishra!