Implement irradiance volumes. #10268

pcwalton · 2023-10-26T02:43:44Z

Objective

Bevy could benefit from irradiance volumes, also known as voxel global illumination or simply as light probes (though this term is not preferred, as multiple techniques can be called light probes).
Irradiance volumes are a form of baked global illumination; they work by sampling the light at the centers of each voxel within a cuboid. At runtime, the voxels surrounding the fragment center are sampled and interpolated to produce indirect diffuse illumination.

Solution

This is divided into two sections. The first is copied and pasted from the irradiance volume module documentation and describes the technique. The second part consists of notes on the implementation.

Overview

An irradiance volume is a cuboid voxel region consisting of
regularly-spaced precomputed samples of diffuse indirect light. They're
ideal if you have a dynamic object such as a character that can move about
static non-moving geometry such as a level in a game, and you want that
dynamic object to be affected by the light bouncing off that static
geometry.

To use irradiance volumes, you need to precompute, or bake, the indirect
light in your scene. Bevy doesn't currently come with a way to do this.
Fortunately, Blender provides a baking tool as part of the Eevee
renderer, and its irradiance volumes are compatible with those used by Bevy.
The bevy-baked-gi project provides a tool, export-blender-gi, that can
extract the baked irradiance volumes from the Blender .blend file and
package them up into a .ktx2 texture for use by the engine. See the
documentation in the bevy-baked-gi project for more details as to this
workflow.

Like all light probes in Bevy, irradiance volumes are 1×1×1 cubes that can
be arbitrarily scaled, rotated, and positioned in a scene with the
[bevy_transform::components::Transform] component. The 3D voxel grid will
be stretched to fill the interior of the cube, and the illumination from the
irradiance volume will apply to all fragments within that bounding region.

Bevy's irradiance volumes are based on Valve's ambient cubes as used in
Half-Life 2 (Mitchell 2006, slide 27). These encode a single color of
light from the six 3D cardinal directions and blend the sides together
according to the surface normal.

The primary reason for choosing ambient cubes is to match Blender, so that
its Eevee renderer can be used for baking. However, they also have some
advantages over the common second-order spherical harmonics approach:
ambient cubes don't suffer from ringing artifacts, they are smaller (6
colors for ambient cubes as opposed to 9 for spherical harmonics), and
evaluation is faster. A smaller basis allows for a denser grid of voxels
with the same storage requirements.

If you wish to use a tool other than export-blender-gi to produce the
irradiance volumes, you'll need to pack the irradiance volumes in the
following format. The irradiance volume of resolution (Rx, Ry, Rz) is
expected to be a 3D texture of dimensions (Rx, 2Ry, 3Rz). The unnormalized
texture coordinate (s, t, p) of the voxel at coordinate (x, y, z) with
side S ∈ {-X, +X, -Y, +Y, -Z, +Z} is as follows:

s = x

t = y + ⎰  0 if S ∈ {-X, -Y, -Z}
        ⎱ Ry if S ∈ {+X, +Y, +Z}

        ⎧   0 if S ∈ {-X, +X}
p = z + ⎨  Rz if S ∈ {-Y, +Y}
        ⎩ 2Rz if S ∈ {-Z, +Z}

Visually, in a left-handed coordinate system with Y up, viewed from the
right, the 3D texture looks like a stacked series of voxel grids, one for
each cube side, in this order:

+X	+Y	+Z
-X	-Y	-Z

A terminology note: Other engines may refer to irradiance volumes as voxel
global illumination, VXGI, or simply as light probes. Sometimes light
probe refers to what Bevy calls a reflection probe. In Bevy, light probe
is a generic term that encompasses all cuboid bounding regions that capture
indirect illumination, whether based on voxels or not.

Note that, if binding arrays aren't supported (e.g. on WebGPU or WebGL 2),
then only the closest irradiance volume to the view will be taken into
account during rendering.

Implementation notes

This patch generalizes light probes so as to reuse as much code as possible between irradiance volumes and the existing reflection probes. This approach was chosen because both techniques share numerous similarities:

Both irradiance volumes and reflection probes are cuboid bounding regions.
Both are responsible for providing baked indirect light.
Both techniques involve presenting a variable number of textures to the shader from which indirect light is sampled. (In the current implementation, this uses binding arrays.)
Both irradiance volumes and reflection probes require gathering and sorting probes by distance on CPU.
Both techniques require the GPU to search through a list of bounding regions.
Both will eventually want to have falloff so that we can smoothly blend as objects enter and exit the probes' influence ranges. (This is not implemented yet to keep this patch relatively small and reviewable.)

To do this, we generalize most of the methods in the reflection probes patch #11366 to be generic over a trait, LightProbeComponent. This trait is implemented by both EnvironmentMapLight (for reflection probes) and IrradianceVolume (for irradiance volumes). Using a trait will allow us to add more types of light probes in the future. In particular, I highly suspect we will want real-time reflection planes for mirrors in the future, which can be easily slotted into this framework.

Changelog

This section is optional. If this was a trivial fix, or has no externally-visible impact, you can delete this section.

Added

A new IrradianceVolume asset type is available for baked voxelized light probes. You can bake the global illumination using Blender or another tool of your choice and use it in Bevy to apply indirect illumination to dynamic objects.

This commit implements *irradiance volumes* in Bevy, also known as *voxel global illumination* or simply as light probes (though this term is not preferred, as multiple techniques can be called light probes). Irradiance volumes are a form of baked global illumination; they work by sampling the light at the centers of each voxel within a cuboid. At runtime, the voxels surrounding the fragment center are sampled and interpolated to produce indirect diffuse illumination. The particular technique is known as *ambient cubes*, popularized by [Valve SIGGRAPH 2006]. Ambient cubes work by recording a 1×1 cubemap at each voxel position. This was chosen over the perhaps-better-known method of spherical harmonics because Blender's Eevee renderer can be easily used to generate them via the tool in [`bevy-baked-gi`]. Also, ambient cubes require fewer table lookups than second-order spherical harmonics do and don't suffer from ringing artifacts. Irradiance volumes are treated as assets, and [`bincode`] is used to deserialize them from disk. You can generate the `bincode` files by passing the `--upstream` flag to the `export-blender-gi` tool in [`bevy-baked-gi`]. **Important**: This commit doesn't provide the generic infrastructure used to look up the nearest light probe. That's part of the reflection probes pull request, bevyengine#10057. Therefore, this pull request requires hardwires the shader to support only one irradiance volume per scene. **This is a temporary limitation** and will be lifted once bevyengine#10057 lands. [Valve SIGGRAPH 2006]: https://advances.realtimerendering.com/s2006/Mitchell-ShadingInValvesSourceEngine.pdf [`bevy-baked-gi`]: https://github.com/pcwalton/bevy-baked-gi [`bincode`]: https://github.com/bincode-org/bincode

pcwalton · 2023-10-26T02:44:13Z

This needs a bit of cleaning up and documentation, but it works and is pretty close.

github-actions · 2023-10-26T03:04:52Z