Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebGPURenderer: StorageBufferNode Support reading external elements in the WebGL Backend #27661

Merged
merged 37 commits into from
Feb 6, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
04c41e8
init pbo
RenaudRohlinger Feb 1, 2024
b4b82eb
remove flag
RenaudRohlinger Feb 1, 2024
3a0b66a
regenerate live example
RenaudRohlinger Feb 1, 2024
62238bc
single buffer and alternate for example
RenaudRohlinger Feb 1, 2024
64dee72
cleanup, no idea about circ dep
RenaudRohlinger Feb 1, 2024
9d32e79
test fix circular
RenaudRohlinger Feb 1, 2024
e089954
cleanup
RenaudRohlinger Feb 1, 2024
9738dd4
moved pbo management to GLSLNodeBuilder
RenaudRohlinger Feb 2, 2024
9cbafeb
fix screenshots
RenaudRohlinger Feb 2, 2024
8072cc5
Merge remote-tracking branch 'upstream/dev' into utsubo/feat/storage_pbo
RenaudRohlinger Feb 2, 2024
90bf43a
support more ranges
RenaudRohlinger Feb 2, 2024
0f5d614
format dynamically in arrayelementnode
RenaudRohlinger Feb 3, 2024
b1c595b
init vertex buffer allocation to prevent issues with some backends
RenaudRohlinger Feb 3, 2024
71a5311
support different type size and update example
RenaudRohlinger Feb 3, 2024
a6c5b64
fix files.json
RenaudRohlinger Feb 3, 2024
0b6b1a8
puppeteer need webgpu support
RenaudRohlinger Feb 3, 2024
fbb4d63
unbind post pbo
RenaudRohlinger Feb 3, 2024
15033d7
add TODO
RenaudRohlinger Feb 3, 2024
affd3bc
Merge branch 'dev' into pr/27661
sunag Feb 4, 2024
cbb95e2
cleanup
sunag Feb 4, 2024
80f9766
Move to StorageArrayElementNode
sunag Feb 4, 2024
daf8f38
cleanup
sunag Feb 4, 2024
05c53b2
rev
sunag Feb 4, 2024
cf55995
add increaseUsage
sunag Feb 4, 2024
17ac24c
Update StorageArrayElementNode.js
sunag Feb 4, 2024
356b712
fixes optmization and revisions
sunag Feb 4, 2024
1c89f4d
revision
sunag Feb 4, 2024
74c6ac4
improved example for E2E tests
RenaudRohlinger Feb 4, 2024
1a22b0b
WIP: Test (1)
sunag Feb 5, 2024
de9b203
handle compute with non-PBO -> PBO case
RenaudRohlinger Feb 5, 2024
b47480e
revert WIP
sunag Feb 5, 2024
c9a21ea
TSL: add `storageObject`
sunag Feb 5, 2024
00f6a5f
revision
sunag Feb 5, 2024
7687fcc
make sure attributeData gets pbo attached in GLSLNodeBuilder
RenaudRohlinger Feb 6, 2024
39691a5
no need transfer anymore
RenaudRohlinger Feb 6, 2024
8936a37
revert compute_texture
RenaudRohlinger Feb 6, 2024
474c0f7
remove copyBufferToSubBuffer
sunag Feb 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion examples/files.json
Original file line number Diff line number Diff line change
Expand Up @@ -373,7 +373,8 @@
"webgpu_postprocessing_anamorphic",
"webgpu_mirror",
"webgpu_multisampled_renderbuffers",
"webgpu_materials_texture_anisotropy"
"webgpu_materials_texture_anisotropy",
"webgpu_storage_pbo"
],
"webaudio": [
"webaudio_orientation",
Expand Down
2 changes: 1 addition & 1 deletion examples/jsm/nodes/core/AssignNode.js
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ class AssignNode extends TempNode {

const targetType = targetNode.getNodeType( builder );

const target = targetNode.build( builder );
const target = targetNode.build( builder, 'assign' );
const source = sourceNode.build( builder, targetType );

const snippet = `${ target } = ${ source }`;
Expand Down
66 changes: 65 additions & 1 deletion examples/jsm/nodes/utils/ArrayElementNode.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
import { texture } from '../Nodes.js';
import Node, { addNodeClass } from '../core/Node.js';
import { DataTexture, RGBAFormat, FloatType } from 'three';

class ArrayElementNode extends Node { // @TODO: If extending from TempNode it breaks webgpu_compute

Expand All @@ -19,13 +21,75 @@ class ArrayElementNode extends Node { // @TODO: If extending from TempNode it br

}

generate( builder ) {
setup( builder ) {

if ( this.node.isStorageBufferNode && ! builder.isAvailable( 'storageBuffer' ) ) {

if ( ! this.node.instanceIndex ) {

const attribute = this.node.value;

if ( attribute.pbo === undefined ) {

const square = Math.sqrt( attribute.array.length / 4 );
const width = Math.floor( square );
const height = Math.ceil( square );

const pboTexture = new DataTexture( attribute.array, width, height, RGBAFormat, FloatType );
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The circular-reference problem is because of UniformNode in ArrayElementNode, UniformNode will use ShaderNode and ShaderNode will include try ArrayElementNode again.

I think it would be better to move this uniform creation block to GLSLNodeBuilder something like builder.getPBOUniform() for example, this could fix the circular reference problem too.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have to consider a new Node, similar to what was done in BufferAttributeNode and buffer.toAttribute() as usage. It could be buffer.toTexture()

Copy link
Contributor

@LeviPesin LeviPesin Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, this feels like something that should better be placed in GLSLNodeBuilder.

Copy link
Collaborator

@sunag sunag Feb 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also consider add/move it to StorageBufferNode.element() to avoid checks a little, and return and new Node to keep operations more organized, it can even be a StorageArrayElementNode.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedbacks!

The downside of buffer.toTexture() would be that this implies that somehow the developer would expect that something happens in the WebGPU Backend too.
This feature is more like a patch to fix the fact WebGL cannot access other element of array buffers rather than a new feature such as toAttribute().

I think I can try to move all this part to GLSLNodeBuilder but I'm not sure on how to create a Node here. Maybe once everything works we can think about creating a new Node specific to Pixel Buffer Object, but once again it's only a WebGL thing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll be happy to help you with this, let me know when the PR is ready for review so I can analyze it again.

About the example what do you think about renaming it to webgpu_storage_sorting?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me! Thanks a lot for this 😊

pboTexture.needsUpdate = true;
pboTexture.isPBOTexture = true;
const pbo = texture( pboTexture );
pbo.setPrecision( 'high' );

attribute.pboNode = pbo;
attribute.pbo = pbo.value;


}

}


}

super.setup( builder );


}

generate( builder, output ) {

const nodeSnippet = this.node.build( builder );
const indexSnippet = this.indexNode.build( builder, 'uint' );

if ( this.node.isStorageBufferNode && ! builder.isAvailable( 'storageBuffer' ) ) {


const attribute = this.node.value;

if ( attribute.pboNode ) {

const nodeUniform = builder.getUniformFromNode( attribute.pboNode, 'texture', builder.shaderStage, builder.context.label );

const propertyNameTexture = builder.getPropertyName( nodeUniform );

const snippet = /* glsl */`
texelFetch(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can also move to GLSLNodeBuilder similar to what happens in builder.generateTextureLoad().

${propertyNameTexture},
ivec2(
${indexSnippet} / uint(4) % uint(textureSize(${propertyNameTexture}, 0).x),
${indexSnippet} / (uint(4) * uint(textureSize(${propertyNameTexture}, 0).x))
),
0
)[${indexSnippet} % uint(4)]`;

return output !== 'assign' ? snippet : nodeSnippet;

}




return nodeSnippet;

}
Expand Down
23 changes: 19 additions & 4 deletions examples/jsm/renderers/webgl/WebGLBackend.js
Original file line number Diff line number Diff line change
Expand Up @@ -430,7 +430,15 @@ class WebGLBackend extends Backend {
// switch active buffers
for ( let i = 0; i < transformBuffers.length; i ++ ) {

transformBuffers[ i ].switchBuffers();
if ( transformBuffers[ i ].pbo ) {

this.textureUtils.transferBufferToTexture( transformBuffers[ i ].bufferGPU, transformBuffers[ i ].pbo );

} else {

transformBuffers[ i ].switchBuffers();

}

}

Expand Down Expand Up @@ -1219,8 +1227,12 @@ class WebGLBackend extends Backend {

key += ':' + attributeData.id;

gl.bindBuffer( gl.ARRAY_BUFFER, attributeData.bufferGPU );
gl.enableVertexAttribArray( i );
if ( attribute.pbo === undefined ) {

gl.bindBuffer( gl.ARRAY_BUFFER, attributeData.bufferGPU );
gl.enableVertexAttribArray( i );

}

if ( attribute.isStorageBufferAttribute ) staticVao = false;

Expand Down Expand Up @@ -1296,7 +1308,10 @@ class WebGLBackend extends Backend {

const attributeData = transformBuffers[ i ];

gl.bindBufferBase( gl.TRANSFORM_FEEDBACK_BUFFER, i, attributeData.transformBuffer );
let buffer = attributeData.bufferGPU;
if ( attributeData.pbo !== undefined ) buffer = attributeData.bufferGPU;

gl.bindBufferBase( gl.TRANSFORM_FEEDBACK_BUFFER, i, buffer );

}

Expand Down
4 changes: 3 additions & 1 deletion examples/jsm/renderers/webgl/utils/WebGLAttributeUtils.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ class DualAttributeData {

this.buffers = [ attributeData.bufferGPU, dualBuffer ];
this.type = attributeData.type;
this.pbo = attributeData.pbo;
this.bytesPerElement = attributeData.BYTES_PER_ELEMENT;
this.version = attributeData.version;
this.isInteger = attributeData.isInteger;
Expand Down Expand Up @@ -129,11 +130,12 @@ class WebGLAttributeUtils {
type,
bytesPerElement: array.BYTES_PER_ELEMENT,
version: attribute.version,
pbo: attribute.pbo,
isInteger: type === gl.INT || type === gl.UNSIGNED_INT || attribute.gpuType === IntType,
id: _id ++
};

if ( attribute.isStorageBufferAttribute ) {
if ( attribute.isStorageBufferAttribute && attribute.pbo === undefined ) {

// create buffer for tranform feedback use
const bufferGPUDual = this._createBuffer( gl, bufferType, array, usage );
Expand Down
35 changes: 35 additions & 0 deletions examples/jsm/renderers/webgl/utils/WebGLTextureUtils.js
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,7 @@ class WebGLTextureUtils {
if ( texture.type === FloatType && extensions.has( 'OES_texture_float_linear' ) === false ) return; // verify extension for WebGL 1 and WebGL 2

if ( texture.anisotropy > 1 || currentAnisotropy !== texture.anisotropy ) {

const extension = extensions.get( 'EXT_texture_filter_anisotropic' );
gl.texParameterf( textureType, extension.TEXTURE_MAX_ANISOTROPY_EXT, Math.min( texture.anisotropy, backend.getMaxAnisotropy() ) );
backend.get( texture ).currentAnisotropy = texture.anisotropy;
Expand Down Expand Up @@ -289,6 +290,40 @@ class WebGLTextureUtils {

}

async transferBufferToTexture( buffer, texture ) {

const { gl, backend } = this;

const { textureGPU, glTextureType, glFormat, glType } = backend.get( texture );

const { width, height } = texture.source.data;


gl.bindBuffer( gl.PIXEL_UNPACK_BUFFER, buffer );

backend.state.bindTexture( glTextureType, textureGPU );

gl.texSubImage2D( glTextureType, 0, 0, 0, width, height, glFormat, glType, 0 );

gl.bindBuffer( gl.PIXEL_UNPACK_BUFFER, null );


// debug
const framebuffer = gl.createFramebuffer();
gl.bindFramebuffer( gl.FRAMEBUFFER, framebuffer );
gl.framebufferTexture2D( gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, glTextureType, textureGPU, 0 );

const readout = new Float32Array( width * height * 4 );

const altFormat = gl.getParameter( gl.IMPLEMENTATION_COLOR_READ_FORMAT );
const altType = gl.getParameter( gl.IMPLEMENTATION_COLOR_READ_TYPE );

gl.readPixels( 0, 0, width, height, altFormat, altType, readout );
gl.bindFramebuffer( gl.FRAMEBUFFER, null );
console.log( readout );

}

updateTexture( texture, options ) {

const { gl } = this;
Expand Down
Binary file added examples/screenshots/webgpu_storage_pbo.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading