Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support perspective projections with shear #12488

Closed
viridia opened this issue Mar 15, 2024 · 2 comments · Fixed by #15537
Closed

Support perspective projections with shear #12488

viridia opened this issue Mar 15, 2024 · 2 comments · Fixed by #15537
Labels
A-Rendering Drawing game state to the screen C-Feature A new feature, making something new possible

Comments

@viridia
Copy link
Contributor

viridia commented Mar 15, 2024

What problem does this solve or what need does it fill?

The ability to create a camera with a sheared projection matrix is important for a number of use cases:

  • Multi-monitor rendering. Imagine a 3x3 display of monitors, each being rendered by a separate processor, but the 3d scene spans the entire array. Now say you want to render the top-left monitor: the camera position is still centered on the middle monitor, the view frustum for the top left monitor is "sheared".
  • Efficient rendering of portals - generally a portal will only want to render a small portion of the window (the 2d bounding box of the portal) to the render target, rather than rendering the entire screen, which would be slow.

What solution would you like?

The setViewOffset() API in three.js is fairly simple. It accepts six parameters:

  • fullWidth — full width of multiview setup
  • fullHeight — full height of multiview setup
  • x — horizontal offset of subcamera
  • y — vertical offset of subcamera
  • width — width of subcamera
  • height — height of subcamera

All this does is adjust the perspective projection, and the corresponding view frustum, to render a rectangular subregion of the normal frustum. This is not the same as a clip rect, the subregion is "spread" over the entire viewport rect.

While it's possible that this feature already exists, I could not find it. I would expect that if it did exist, it would be a property of either Camera, Camera3DBundle, or ProjectionMatrix, but I could find nothing in those types relevant to this.

What alternative(s) have you considered?

I can't think of a workaround for this, other than rendering to a very large render target and then cutting out a small piece of it, which would destroy performance.

@viridia viridia added C-Feature A new feature, making something new possible S-Needs-Triage This issue needs to be labelled labels Mar 15, 2024
@SolarLiner SolarLiner added A-Rendering Drawing game state to the screen and removed S-Needs-Triage This issue needs to be labelled labels Mar 15, 2024
@viridia
Copy link
Contributor Author

viridia commented Mar 18, 2024

My motivation for wanting this, BTW, is that I am currently working on portals. Here's a screenshot showing my work in progress:

portal-quad

The red rectangle gizmo shows the bounding box of the portal quad, which is the green gizmo. I already have in place code to render a secondary camera to an offscreen buffer; however, that secondary camera is rendering the entire screen, when ideally it should only render a tiny section of the screen. This will require modifying both the perspective transform and the frustum of the secondary camera.

For portals, specifically, there's an additional requirement that I didn't mention, which is that the near plane of the frustum needs to be adjusted to align with the portal aperture, so that objects that are in front of the portal (that is, between the secondary camera and the target viewpoint) do not get included in the view. In my old three.js code, I used a auxiliary clipping plane for this, but Bevy doesn't have that feature.

@inodentry
Copy link
Contributor

inodentry commented Apr 20, 2024

I would like to add another use case that could benefit from this: screen shake in 3D.

If you want to shake the camera in 2D, you can just translate the camera along x/y, it's simple.

But in 3D, there are many different ways to accomplish screen shake, which look different, and it can be an artistic choice which way to implement the effect:

  • Translating the camera (usually along the plane of the screen in 3D space)
  • Rotating the camera (either by keeping a fixed "looking at" point, or via pitch/yaw)
  • Offsetting the projection, which can look very nice, because it maintains the perspective and makes the effect feel more intuitive

The third option, obviously, needs the game engine to support setting an X/Y offset to the perspective projection.

github-merge-queue bot pushed a commit that referenced this issue Oct 1, 2024
# Objective

- This PR fixes #12488

## Solution

- This PR adds a new property to `Camera` that emulates the
functionality of the
[setViewOffset()](https://threejs.org/docs/#api/en/cameras/PerspectiveCamera.setViewOffset)
API in three.js.
- When set, the perspective and orthographic projections will restrict
the visible area of the camera to a part of the view frustum defined by
`offset` and `size`.

## Testing

- In the new `camera_sub_view` example, a fixed, moving and control sub
view is created for both perspective and orthographic projection
- Run the example with `cargo run --example camera_sub_view`
- The code can be tested by adding a `SubCameraView` to a camera

---

## Showcase


![image](https://github.com/user-attachments/assets/75ac45fc-d75d-4664-8ef6-ff7865297c25)

- Left Half: Perspective Projection
- Right Half: Orthographic Projection
- Small boxes in order:
  - Sub view of the left half of the full image
- Sub view moving from the top left to the bottom right of the full
image
  - Sub view of the full image (acting as a control)
- Large box: No sub view

<details>
  <summary>Shortened camera setup of `camera_sub_view` example</summary>

```rust
    // Main perspective Camera
    commands.spawn(Camera3dBundle {
        transform,
        ..default()
    });

    // Perspective camera left half
    commands.spawn(Camera3dBundle {
        camera: Camera {
            sub_camera_view: Some(SubCameraView {
                // Set the sub view camera to the left half of the full image
                full_size: uvec2(500, 500),
                offset: ivec2(0, 0),
                size: uvec2(250, 500),
            }),
            order: 1,
            ..default()
        },
        transform,
        ..default()
    });

    // Perspective camera moving
    commands.spawn((
        Camera3dBundle {
            camera: Camera {
                sub_camera_view: Some(SubCameraView {
                    // Set the sub view camera to a fifth of the full view and
                    // move it in another system
                    full_size: uvec2(500, 500),
                    offset: ivec2(0, 0),
                    size: uvec2(100, 100),
                }),
                order: 2,
                ..default()
            },
            transform,
            ..default()
        },
        MovingCameraMarker,
    ));

    // Perspective camera control
    commands.spawn(Camera3dBundle {
        camera: Camera {
            sub_camera_view: Some(SubCameraView {
                // Set the sub view to the full image, to ensure that it matches
                // the projection without sub view
                full_size: uvec2(450, 450),
                offset: ivec2(0, 0),
                size: uvec2(450, 450),
            }),
            order: 3,
            ..default()
        },
        transform,
        ..default()
    });

    // Main orthographic camera
    commands.spawn(Camera3dBundle {
        projection: OrthographicProjection {
          ...
        }
        .into(),
        camera: Camera {
            order: 4,
            ..default()
        },
        transform,
        ..default()
    });

    // Orthographic camera left half
    commands.spawn(Camera3dBundle {
        projection: OrthographicProjection {
          ...
        }
        .into(),
        camera: Camera {
            sub_camera_view: Some(SubCameraView {
                // Set the sub view camera to the left half of the full image
                full_size: uvec2(500, 500),
                offset: ivec2(0, 0),
                size: uvec2(250, 500),
            }),
            order: 5,
            ..default()
        },
        transform,
        ..default()
    });

    // Orthographic camera moving
    commands.spawn((
        Camera3dBundle {
            projection: OrthographicProjection {
              ...
            }
            .into(),
            camera: Camera {
                sub_camera_view: Some(SubCameraView {
                    // Set the sub view camera to a fifth of the full view and
                    // move it in another system
                    full_size: uvec2(500, 500),
                    offset: ivec2(0, 0),
                    size: uvec2(100, 100),
                }),
                order: 6,
                ..default()
            },
            transform,
            ..default()
        },
        MovingCameraMarker,
    ));

    // Orthographic camera control
    commands.spawn(Camera3dBundle {
        projection: OrthographicProjection {
          ...
        }
        .into(),
        camera: Camera {
            sub_camera_view: Some(SubCameraView {
                // Set the sub view to the full image, to ensure that it matches
                // the projection without sub view
                full_size: uvec2(450, 450),
                offset: ivec2(0, 0),
                size: uvec2(450, 450),
            }),
            order: 7,
            ..default()
        },
        transform,
        ..default()
    });
```

</details>
robtfm pushed a commit to robtfm/bevy that referenced this issue Oct 4, 2024
# Objective

- This PR fixes bevyengine#12488

## Solution

- This PR adds a new property to `Camera` that emulates the
functionality of the
[setViewOffset()](https://threejs.org/docs/#api/en/cameras/PerspectiveCamera.setViewOffset)
API in three.js.
- When set, the perspective and orthographic projections will restrict
the visible area of the camera to a part of the view frustum defined by
`offset` and `size`.

## Testing

- In the new `camera_sub_view` example, a fixed, moving and control sub
view is created for both perspective and orthographic projection
- Run the example with `cargo run --example camera_sub_view`
- The code can be tested by adding a `SubCameraView` to a camera

---

## Showcase


![image](https://github.com/user-attachments/assets/75ac45fc-d75d-4664-8ef6-ff7865297c25)

- Left Half: Perspective Projection
- Right Half: Orthographic Projection
- Small boxes in order:
  - Sub view of the left half of the full image
- Sub view moving from the top left to the bottom right of the full
image
  - Sub view of the full image (acting as a control)
- Large box: No sub view

<details>
  <summary>Shortened camera setup of `camera_sub_view` example</summary>

```rust
    // Main perspective Camera
    commands.spawn(Camera3dBundle {
        transform,
        ..default()
    });

    // Perspective camera left half
    commands.spawn(Camera3dBundle {
        camera: Camera {
            sub_camera_view: Some(SubCameraView {
                // Set the sub view camera to the left half of the full image
                full_size: uvec2(500, 500),
                offset: ivec2(0, 0),
                size: uvec2(250, 500),
            }),
            order: 1,
            ..default()
        },
        transform,
        ..default()
    });

    // Perspective camera moving
    commands.spawn((
        Camera3dBundle {
            camera: Camera {
                sub_camera_view: Some(SubCameraView {
                    // Set the sub view camera to a fifth of the full view and
                    // move it in another system
                    full_size: uvec2(500, 500),
                    offset: ivec2(0, 0),
                    size: uvec2(100, 100),
                }),
                order: 2,
                ..default()
            },
            transform,
            ..default()
        },
        MovingCameraMarker,
    ));

    // Perspective camera control
    commands.spawn(Camera3dBundle {
        camera: Camera {
            sub_camera_view: Some(SubCameraView {
                // Set the sub view to the full image, to ensure that it matches
                // the projection without sub view
                full_size: uvec2(450, 450),
                offset: ivec2(0, 0),
                size: uvec2(450, 450),
            }),
            order: 3,
            ..default()
        },
        transform,
        ..default()
    });

    // Main orthographic camera
    commands.spawn(Camera3dBundle {
        projection: OrthographicProjection {
          ...
        }
        .into(),
        camera: Camera {
            order: 4,
            ..default()
        },
        transform,
        ..default()
    });

    // Orthographic camera left half
    commands.spawn(Camera3dBundle {
        projection: OrthographicProjection {
          ...
        }
        .into(),
        camera: Camera {
            sub_camera_view: Some(SubCameraView {
                // Set the sub view camera to the left half of the full image
                full_size: uvec2(500, 500),
                offset: ivec2(0, 0),
                size: uvec2(250, 500),
            }),
            order: 5,
            ..default()
        },
        transform,
        ..default()
    });

    // Orthographic camera moving
    commands.spawn((
        Camera3dBundle {
            projection: OrthographicProjection {
              ...
            }
            .into(),
            camera: Camera {
                sub_camera_view: Some(SubCameraView {
                    // Set the sub view camera to a fifth of the full view and
                    // move it in another system
                    full_size: uvec2(500, 500),
                    offset: ivec2(0, 0),
                    size: uvec2(100, 100),
                }),
                order: 6,
                ..default()
            },
            transform,
            ..default()
        },
        MovingCameraMarker,
    ));

    // Orthographic camera control
    commands.spawn(Camera3dBundle {
        projection: OrthographicProjection {
          ...
        }
        .into(),
        camera: Camera {
            sub_camera_view: Some(SubCameraView {
                // Set the sub view to the full image, to ensure that it matches
                // the projection without sub view
                full_size: uvec2(450, 450),
                offset: ivec2(0, 0),
                size: uvec2(450, 450),
            }),
            order: 7,
            ..default()
        },
        transform,
        ..default()
    });
```

</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Rendering Drawing game state to the screen C-Feature A new feature, making something new possible
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants