Toggle between Splats and Points

[jarodabel]

A feature request I have for my work is to allow users to toggle the guassian splat viewer from splats to points and back. Mostly to show the reality of how many points there are. Has to do with the ability to draw annotations / measurements.

I have come up with my own way to do this but I think it would be pretty easy to add the ability since your code already does all the things.

      const response = await fetch(url);
      const arrayBuffer = await response.arrayBuffer();
      const parser = new PlyParserCodecBase();
      await parser.readPly(arrayBuffer);
      parser.decompress(0, 1, 5.0, 256);

      // eslint-disable-next-line @typescript-eslint/no-explicit-any
      const data: any = parser.splatData;
      const geometry = new THREE.BufferGeometry();

      // Assuming numSplats tells you the total number of points
      const numPoints = data.numSplats;

      const positions = new Float32Array(numPoints * 3);
      const colors = new Float32Array(numPoints * 3); // Assuming colors are RGB and normalized to [0,1]
      const SH_C0 = 0.28209479177387814;

      for (let i = 0; i < numPoints; i++) {
        // Position
        positions[i * 3 + 0] = data.x[i];
        positions[i * 3 + 1] = data.y[i];
        positions[i * 3 + 2] = data.z[i];

        // Colors (re-normalizing from the transformed values if necessary)
        colors[i * 3 + 0] = data.f_dc_0[i] * SH_C0 + 0.5;
        colors[i * 3 + 1] = data.f_dc_1[i] * SH_C0 + 0.5;
        colors[i * 3 + 2] = data.f_dc_2[i] * SH_C0 + 0.5;
      }

      geometry.setAttribute(
        'position',
        new THREE.BufferAttribute(positions, 3)
      );
      geometry.setAttribute('color', new THREE.BufferAttribute(colors, 3));
      geometry.setAttribute(
        'size',
        new THREE.BufferAttribute(new Float32Array(numPoints).fill(0.04), 1)
      );

      const vertexShader = `
attribute float size;
varying vec3 vColor;

void main() {
    vColor = color;
    vec4 mvPosition = modelViewMatrix * vec4(position, 1.0);
    gl_PointSize = size * (300.0 / -mvPosition.z);
    gl_Position = projectionMatrix * mvPosition;
}
`;

      const fragmentShader = `
uniform sampler2D pointTexture;
varying vec3 vColor;

void main() {
    gl_FragColor = vec4(vColor, 1.0);
    vec2 circCoord = 2.0 * gl_PointCoord - 1.0;
    if (dot(circCoord, circCoord) > 1.0) {
        discard;
    }
}
`;

      const material = new THREE.ShaderMaterial({
        vertexShader: vertexShader,
        fragmentShader: fragmentShader,
        transparent: true,
        vertexColors: true,
      });
      const points = new THREE.Points(geometry, material);
      scene.add(points);

[mkkellogg]

There's actually a way to view scenes as points right now, although it doesn't use THREE.Points for rendering (which is probably faster). You can call SplatMesh.setPointCloudModeEnabled(<enabled: boolean>), although now that I think about it, it would be much nicer if Viewer and DropInViewer had that function. I'll add that in the next release :) If you are just using the vanilla viewer, you can press the P key to toggle between point-cloud view and splat view.

[jarodabel]

Okay, thank you. I will check out the points. I should have investigated more.

[mkkellogg]

Sorry you went to all that trouble, maybe I need to highlight this capability a little more clearly in the documentation.