Add `sub_camera_view`, enabling sheared projection #15537

m-edlund · 2024-09-30T13:49:55Z

Objective

This PR fixes Support perspective projections with shear #12488

Solution

This PR adds a new property to Camera that emulates the functionality of the setViewOffset() API in three.js.
When set, the perspective and orthographic projections will restrict the visible area of the camera to a part of the view frustum defined by offset and size.

Testing

In the new camera_sub_view example, a fixed, moving and control sub view is created for both perspective and orthographic projection
Run the example with cargo run --example camera_sub_view
The code can be tested by adding a SubCameraView to a camera

Showcase

Left Half: Perspective Projection
Right Half: Orthographic Projection
Small boxes in order:
- Sub view of the left half of the full image
- Sub view moving from the top left to the bottom right of the full image
- Sub view of the full image (acting as a control)
Large box: No sub view

Shortened camera setup of `camera_sub_view` example

    // Main perspective Camera
    commands.spawn(Camera3dBundle {
        transform,
        ..default()
    });

    // Perspective camera left half
    commands.spawn(Camera3dBundle {
        camera: Camera {
            sub_camera_view: Some(SubCameraView {
                // Set the sub view camera to the left half of the full image
                full_size: uvec2(500, 500),
                offset: ivec2(0, 0),
                size: uvec2(250, 500),
            }),
            order: 1,
            ..default()
        },
        transform,
        ..default()
    });

    // Perspective camera moving
    commands.spawn((
        Camera3dBundle {
            camera: Camera {
                sub_camera_view: Some(SubCameraView {
                    // Set the sub view camera to a fifth of the full view and
                    // move it in another system
                    full_size: uvec2(500, 500),
                    offset: ivec2(0, 0),
                    size: uvec2(100, 100),
                }),
                order: 2,
                ..default()
            },
            transform,
            ..default()
        },
        MovingCameraMarker,
    ));

    // Perspective camera control
    commands.spawn(Camera3dBundle {
        camera: Camera {
            sub_camera_view: Some(SubCameraView {
                // Set the sub view to the full image, to ensure that it matches
                // the projection without sub view
                full_size: uvec2(450, 450),
                offset: ivec2(0, 0),
                size: uvec2(450, 450),
            }),
            order: 3,
            ..default()
        },
        transform,
        ..default()
    });

    // Main orthographic camera
    commands.spawn(Camera3dBundle {
        projection: OrthographicProjection {
          ...
        }
        .into(),
        camera: Camera {
            order: 4,
            ..default()
        },
        transform,
        ..default()
    });

    // Orthographic camera left half
    commands.spawn(Camera3dBundle {
        projection: OrthographicProjection {
          ...
        }
        .into(),
        camera: Camera {
            sub_camera_view: Some(SubCameraView {
                // Set the sub view camera to the left half of the full image
                full_size: uvec2(500, 500),
                offset: ivec2(0, 0),
                size: uvec2(250, 500),
            }),
            order: 5,
            ..default()
        },
        transform,
        ..default()
    });

    // Orthographic camera moving
    commands.spawn((
        Camera3dBundle {
            projection: OrthographicProjection {
              ...
            }
            .into(),
            camera: Camera {
                sub_camera_view: Some(SubCameraView {
                    // Set the sub view camera to a fifth of the full view and
                    // move it in another system
                    full_size: uvec2(500, 500),
                    offset: ivec2(0, 0),
                    size: uvec2(100, 100),
                }),
                order: 6,
                ..default()
            },
            transform,
            ..default()
        },
        MovingCameraMarker,
    ));

    // Orthographic camera control
    commands.spawn(Camera3dBundle {
        projection: OrthographicProjection {
          ...
        }
        .into(),
        camera: Camera {
            sub_camera_view: Some(SubCameraView {
                // Set the sub view to the full image, to ensure that it matches
                // the projection without sub view
                full_size: uvec2(450, 450),
                offset: ivec2(0, 0),
                size: uvec2(450, 450),
            }),
            order: 7,
            ..default()
        },
        transform,
        ..default()
    });

github-actions · 2024-09-30T13:50:09Z

Welcome, new contributor!

Please make sure you've read our contributing guide and we look forward to reviewing your pull request shortly ✨

crates/bevy_render/src/camera/camera.rs

alice-i-cecile · 2024-09-30T13:51:35Z

examples/3d/camera_sub_view.rs

@@ -0,0 +1,267 @@
+//! Renders multiple cameras with different sub view efffects.


This needs a less jargon-y explanation: what's a "sub view", and why might a user care about this?

I added mentions of some possible use cases, however the screen shake use case is not covered by my example. Is it fine to leave it in?

Yep, that's fine. We should add a dedicated "screen_shake" example at some point, but that's totally fine to leave for follow-up.

alice-i-cecile · 2024-09-30T13:52:19Z

I like the feature and the code is good, but I think we need to do a better job explaining why users might care :)

alice-i-cecile · 2024-09-30T14:23:33Z

crates/bevy_render/src/camera/camera.rs

@@ -66,6 +66,31 @@ impl Default for Viewport {
    }
 }

+/// Settings to define a camera sub view.
+///
+/// When added to a camera, only the sub-section of the image defined by `size`


Suggested change

/// When added to a camera, only the sub-section of the image defined by `size`

/// When [`Camera::sub_camera_view`] is `Some`, only the sub-section of the image defined by `size`

alice-i-cecile

Much clearer, thanks!

tychedelia

Seems like the code isn't taking into account scaling factor.

Also, we can bikeshed this more later, but I don't like "sub view" naming. I'm not sure what the prior art is here, but I would minimally prefer something spelled out like "ViewSubset" or even just the offset terminology used in the linked api by three.js.

tychedelia · 2024-09-30T17:13:07Z

examples/3d/camera_sub_view.rs

+                sub_camera_view: Some(SubCameraView {
+                    // Set the sub view camera to a fifth of the full view and
+                    // move it in another system
+                    full_size: uvec2(500, 500),


Will let @alice-i-cecile make a determination, but we tend not to use these function constructors anywhere in engine code (although I personally find them quite nice!)

Yeah, let's keep it consistent and use the non-function forms.

tychedelia · 2024-09-30T17:16:24Z

crates/bevy_render/src/camera/camera.rs

+#[derive(Debug, Clone, Copy, Reflect, PartialEq, Eq)]
+pub struct SubCameraView {
+    /// Size of the entire camera view
+    pub full_size: UVec2,


I'm a bit confused about the purpose of this field, shouldn't this be computed from the camera/viewport itself rather than manually specified?

It could theoretically be calculated from the viewport itself, however I think this would make the API more confusing for users. Take the example of a multi-monitor setup for example.

┌───┬───┐ │ A │ B │ ├───┼───┤ │ C │ D │ └───┴───┘

If each monitor is 1920x1080 and we decide to use a 1920x1080 viewport for each monitor, then the whole should be 3840x2160.

Now to determine the sub view values for monitor/viewport A, the sub view has to have half the dimensions of the "full image":

If we calculated the full view from the size of our viewport, the sub view would have to be 960x540 to correctly take up a quarter of the physical viewport that is 1920x1080

If we have the additional full_size field, the users don't have to think about the actual size of their physical viewport but can consider the size of the "full image" instead. And can then just use the value of the physical viewport of A for their sub view.

As I write this out I'm starting to realize that this should also work with full_size being 32x18 and size being 16x9.

I will change the example to make this more apparent, and adjust the doc comments to make it clearer that the ratio is what is relevant.

I will also allow for offset to be a regular Vec2 to allow for smaller steps when size and full_size is made up of small values.

Alright, that makes sense to me.

viridia · 2024-09-30T17:27:59Z

crates/bevy_render/src/camera/camera.rs

+/// whole image) is projected to the cameras viewport.
+#[derive(Debug, Clone, Copy, Reflect, PartialEq, Eq)]
+pub struct SubCameraView {
+    /// Size of the entire camera view


I'd add "in pixels".

The sub view is defined by the ratio between full_size, size and offset, so in this case "in pixels" would be misleading. That was however not sufficiently documented, so I have added further detail.

viridia

API looks pretty good. Looking forward to testing with it.

m-edlund · 2024-10-01T06:34:00Z

@tychedelia

Seems like the code isn't taking into account scaling factor.

I'm not quite sure what you did to get this effect. I tried with_scale_factor_override but I could not reproduce the issue. Regarding the fixed sizes of the viewports and the window, there is an error when the viewport extends over the edges of the actual window. So to avoid this, and to avoid adding code for automatically scaling the viewports (so as not to distract from the core of the example), I chose to make the values fixed.

If you want, I can expand the example to include automatic scaling of the viewports.

Also, we can bikeshed this more later, but I don't like "sub view" naming. I'm not sure what the prior art is here, but I would minimally prefer something spelled out like "ViewSubset" or even just the offset terminology used in the linked api by three.js.

I would not like to use "ViewSubset", since subset implies a countable number of possible views and that the current view is one or more of them. I'd be happy to use the offset naming analogous to three.js though :)

alice-i-cecile · 2024-10-01T14:11:21Z

Thanks for the careful documentation and patient responses to reviewers :) I'm happy with the state of this now: merging.

tychedelia · 2024-10-01T15:54:08Z

@alice-i-cecile This is still broken on macOS and will minimally need a quick follow up for that.

# Objective - This PR fixes bevyengine#12488 ## Solution - This PR adds a new property to `Camera` that emulates the functionality of the [setViewOffset()](https://threejs.org/docs/#api/en/cameras/PerspectiveCamera.setViewOffset) API in three.js. - When set, the perspective and orthographic projections will restrict the visible area of the camera to a part of the view frustum defined by `offset` and `size`. ## Testing - In the new `camera_sub_view` example, a fixed, moving and control sub view is created for both perspective and orthographic projection - Run the example with `cargo run --example camera_sub_view` - The code can be tested by adding a `SubCameraView` to a camera --- ## Showcase ![image](https://github.com/user-attachments/assets/75ac45fc-d75d-4664-8ef6-ff7865297c25) - Left Half: Perspective Projection - Right Half: Orthographic Projection - Small boxes in order: - Sub view of the left half of the full image - Sub view moving from the top left to the bottom right of the full image - Sub view of the full image (acting as a control) - Large box: No sub view <details> <summary>Shortened camera setup of `camera_sub_view` example</summary> ```rust // Main perspective Camera commands.spawn(Camera3dBundle { transform, ..default() }); // Perspective camera left half commands.spawn(Camera3dBundle { camera: Camera { sub_camera_view: Some(SubCameraView { // Set the sub view camera to the left half of the full image full_size: uvec2(500, 500), offset: ivec2(0, 0), size: uvec2(250, 500), }), order: 1, ..default() }, transform, ..default() }); // Perspective camera moving commands.spawn(( Camera3dBundle { camera: Camera { sub_camera_view: Some(SubCameraView { // Set the sub view camera to a fifth of the full view and // move it in another system full_size: uvec2(500, 500), offset: ivec2(0, 0), size: uvec2(100, 100), }), order: 2, ..default() }, transform, ..default() }, MovingCameraMarker, )); // Perspective camera control commands.spawn(Camera3dBundle { camera: Camera { sub_camera_view: Some(SubCameraView { // Set the sub view to the full image, to ensure that it matches // the projection without sub view full_size: uvec2(450, 450), offset: ivec2(0, 0), size: uvec2(450, 450), }), order: 3, ..default() }, transform, ..default() }); // Main orthographic camera commands.spawn(Camera3dBundle { projection: OrthographicProjection { ... } .into(), camera: Camera { order: 4, ..default() }, transform, ..default() }); // Orthographic camera left half commands.spawn(Camera3dBundle { projection: OrthographicProjection { ... } .into(), camera: Camera { sub_camera_view: Some(SubCameraView { // Set the sub view camera to the left half of the full image full_size: uvec2(500, 500), offset: ivec2(0, 0), size: uvec2(250, 500), }), order: 5, ..default() }, transform, ..default() }); // Orthographic camera moving commands.spawn(( Camera3dBundle { projection: OrthographicProjection { ... } .into(), camera: Camera { sub_camera_view: Some(SubCameraView { // Set the sub view camera to a fifth of the full view and // move it in another system full_size: uvec2(500, 500), offset: ivec2(0, 0), size: uvec2(100, 100), }), order: 6, ..default() }, transform, ..default() }, MovingCameraMarker, )); // Orthographic camera control commands.spawn(Camera3dBundle { projection: OrthographicProjection { ... } .into(), camera: Camera { sub_camera_view: Some(SubCameraView { // Set the sub view to the full image, to ensure that it matches // the projection without sub view full_size: uvec2(450, 450), offset: ivec2(0, 0), size: uvec2(450, 450), }), order: 7, ..default() }, transform, ..default() }); ``` </details>

alice-i-cecile added C-Feature A new feature, making something new possible A-Rendering Drawing game state to the screen labels Sep 30, 2024

alice-i-cecile reviewed Sep 30, 2024

View reviewed changes

crates/bevy_render/src/camera/camera.rs Show resolved Hide resolved

alice-i-cecile reviewed Sep 30, 2024

View reviewed changes

alice-i-cecile added D-Modest A "normal" level of difficulty; suitable for simple features or challenging fixes S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged labels Sep 30, 2024

alice-i-cecile reviewed Sep 30, 2024

View reviewed changes

alice-i-cecile approved these changes Sep 30, 2024

View reviewed changes

alice-i-cecile requested review from viridia and tychedelia September 30, 2024 14:24

alice-i-cecile added S-Needs-Review Needs reviewer attention (from anyone!) to move forward and removed S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged labels Sep 30, 2024

tychedelia suggested changes Sep 30, 2024

View reviewed changes

viridia reviewed Sep 30, 2024

View reviewed changes

viridia approved these changes Sep 30, 2024

View reviewed changes

m-edlund added 12 commits October 1, 2024 10:03

feat: add example

de77ca7

feat: dynamic updates

5e9f12d

feat: add perspective calculation

31079f9

fix: orthographic projection

8d7ec18

feat: orthographic & negative offset

2b9bedb

fix: minor clean up

8065c89

fix: main fixes

1142637

fix: typo

efb3dcf

fix: build-templated-pages

3324c8a

fix: more detail on sub view example

2ade89a

fix: more detail on SubCameraView

2224cb8

fix: suggested description

ce16503

m-edlund added 5 commits October 1, 2024 10:03

fix: avoid short hand constructors

00e35af

fix: explain ratio better in example

8ed8b2a

fix: use vec2 for offset

df6aad9

fix: improve docs

09c6b56

fix: no longer use point light bundle

f422172

m-edlund force-pushed the feat/main/cameraSubView branch from 40d0499 to f422172 Compare October 1, 2024 08:08

alice-i-cecile added S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it and removed S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Oct 1, 2024

alice-i-cecile mentioned this pull request Oct 1, 2024

Add a screen shake example #15564

Closed

alice-i-cecile added this pull request to the merge queue Oct 1, 2024

Merged via the queue into bevyengine:main with commit c323db0 Oct 1, 2024
31 checks passed

m-edlund mentioned this pull request Oct 1, 2024

Screen shake example #15567

Merged

BenjaminBrienen mentioned this pull request Oct 1, 2024

macOS: camera with shear not working #15576

Closed

m-edlund mentioned this pull request Oct 2, 2024

Sub View only projects correctly when aspect ratio of size and full_size match #15600

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `sub_camera_view`, enabling sheared projection #15537

Add `sub_camera_view`, enabling sheared projection #15537

m-edlund commented Sep 30, 2024

github-actions bot commented Sep 30, 2024

alice-i-cecile Sep 30, 2024

m-edlund Sep 30, 2024

alice-i-cecile Sep 30, 2024

alice-i-cecile commented Sep 30, 2024

alice-i-cecile Sep 30, 2024

alice-i-cecile left a comment

tychedelia left a comment

tychedelia Sep 30, 2024

alice-i-cecile Sep 30, 2024

tychedelia Sep 30, 2024

m-edlund Oct 1, 2024

tychedelia Oct 1, 2024

viridia Sep 30, 2024

m-edlund Oct 1, 2024

viridia left a comment

m-edlund commented Oct 1, 2024 •

edited

Loading

alice-i-cecile commented Oct 1, 2024

tychedelia commented Oct 1, 2024

		@@ -0,0 +1,267 @@
		//! Renders multiple cameras with different sub view efffects.

	/// When added to a camera, only the sub-section of the image defined by `size`
	/// When [`Camera::sub_camera_view`] is `Some`, only the sub-section of the image defined by `size`

Add sub_camera_view, enabling sheared projection #15537

Add sub_camera_view, enabling sheared projection #15537

Conversation

m-edlund commented Sep 30, 2024

Objective

Solution

Testing

Showcase

github-actions bot commented Sep 30, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alice-i-cecile commented Sep 30, 2024

Choose a reason for hiding this comment

alice-i-cecile left a comment

Choose a reason for hiding this comment

tychedelia left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

viridia left a comment

Choose a reason for hiding this comment

m-edlund commented Oct 1, 2024 • edited Loading

alice-i-cecile commented Oct 1, 2024

tychedelia commented Oct 1, 2024

Add `sub_camera_view`, enabling sheared projection #15537

Add `sub_camera_view`, enabling sheared projection #15537

m-edlund commented Oct 1, 2024 •

edited

Loading