Skip to content

Commit

Permalink
New LeafTransform3D, replacing OutOfTreeTransform3D (#7015)
Browse files Browse the repository at this point in the history
### What

* Part of #6831
* Replacement for #6988

Introduces a new `LeafTransform3D` archetype that is always applicable.
It entails a copy of all of `Transform3D`'s components - axis length and
transform relation have been omitted so far.

Surprisingly, I didn't have much need for the extensive extensions we
have on `Transform3D` so far: Leaf transform is much less commonly used
and deals with arrays, making it sufficiently different from
`Transform3D`. Also a lot of the extensions associated with
`Transform3D` are there for legacy reasons - with the new more
componetized interface we get much more reasonable ergonomics out of the
box!

This PR entails a major overhaul of the `TransformContext`. For *sure*
not the last time we do this (looking at you 2D transform handling &
not-so-great 2D<->3D interactions!), but the intention is to be a bit
more forward looking and to enforce use of leaf transforms everywhere.

Single component leaf transforms are supported everywhere now. Multi
component leaf transforms logs a warning for all visualizers except
Mesh3D and Asset3D where it bottoms out in instantiating the mesh
multiple times:


https://github.com/user-attachments/assets/62d26661-cd8c-4b4a-b912-063ef60e063a

Snippet demonstrating combination of `Transform3D` with
`LeafTransforms3D`:


https://github.com/user-attachments/assets/ebb2ce5b-6d9a-407d-9f21-57b92f4ac25c

Follow-up PRs will improve the interaction of various archetypes with
`LeafTransforms3D` as well as remove now unused legacy types.


### Checklist
* [x] run main ci to ensure that roundtrip & snippet tests are passing
* [x] check transform checklist again!
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using examples from latest `main` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7015?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7015?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!
* [x] If have noted any breaking changes to the log API in
`CHANGELOG.md` and the migration guide

- [PR Build Summary](https://build.rerun.io/pr/7015)
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

To run all checks from `main`, comment on the PR with `@rerun-bot
full-check`.
  • Loading branch information
Wumpf authored Jul 31, 2024
1 parent 8d8dc76 commit cdeac62
Show file tree
Hide file tree
Showing 141 changed files with 3,863 additions and 1,054 deletions.
4 changes: 4 additions & 0 deletions Cargo.lock
Original file line number Diff line number Diff line change
Expand Up @@ -5012,6 +5012,7 @@ dependencies = [
"re_viewport_blueprint",
"serde",
"smallvec",
"vec1",
"web-time",
]

Expand Down Expand Up @@ -6975,6 +6976,9 @@ name = "vec1"
version = "1.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2bda7c41ca331fe9a1c278a9e7ee055f4be7f5eb1c2b72f079b4ff8b5fce9d5c"
dependencies = [
"smallvec",
]

[[package]]
name = "version_check"
Expand Down
6 changes: 4 additions & 2 deletions crates/store/re_chunk/src/helpers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -235,11 +235,13 @@ impl UnitChunkShared {
/// The maximum value amongst all components is what's returned.
#[inline]
pub fn num_instances(&self) -> u64 {
debug_assert!(self.num_rows() == 1);
self.components
.values()
.map(|list_array| {
list_array.validity().map_or_else(
|| list_array.len(),
let array = list_array.value(0);
array.validity().map_or_else(
|| array.len(),
|validity| validity.len() - validity.unset_bits(),
)
})
Expand Down
1 change: 1 addition & 0 deletions crates/store/re_types/definitions/rerun/archetypes.fbs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 4 additions & 9 deletions crates/store/re_types/definitions/rerun/archetypes/asset3d.fbs
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,12 @@ namespace rerun.archetypes;
///
/// See also [archetypes.Mesh3D].
///
/// If there are multiple [archetypes.LeafTransforms3D] instances logged to the same entity as a mesh,
/// an instance of the mesh will be drawn for each transform.
///
/// \example archetypes/asset3d_simple title="Simple 3D asset" image="https://static.rerun.io/asset3d_simple/af238578188d3fd0de3e330212120e2842a8ddb2/1200w.png"
/// \example archetypes/asset3d_out_of_tree !api title="3D asset with out-of-tree transform"
table Asset3D (
"attr.rust.derive": "PartialEq",
"attr.rust.derive": "PartialEq, Eq",
"attr.docs.category": "Spatial 3D",
"attr.docs.view_types": "Spatial3DView, Spatial2DView: if logged above active projection"
) {
Expand All @@ -31,11 +33,4 @@ table Asset3D (
/// If omitted, the viewer will try to guess from the data blob.
/// If it cannot guess, it won't be able to render the asset.
media_type: rerun.components.MediaType ("attr.rerun.component_recommended", nullable, order: 2000);

// --- Optional ---

/// An out-of-tree transform.
///
/// Applies a transformation to the asset itself without impacting its children.
transform: rerun.components.OutOfTreeTransform3D ("attr.rerun.component_optional", nullable, order: 3000);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
namespace rerun.archetypes;


/// One or more transforms between the parent and the current entity which are *not* propagated in the transform hierarchy.
///
/// For transforms that are propagated in the transform hierarchy, see [archetypes.Transform3D].
///
/// If both [archetypes.LeafTransforms3D] and [archetypes.Transform3D] are present,
/// first the tree propagating [archetypes.Transform3D] is applied, then [archetypes.LeafTransforms3D].
///
/// Currently, most visualizers support only a single leaf transform per entity.
/// Check archetype documentations for details - if not otherwise specified, only the first leaf transform is applied.
///
/// From the point of view of the entity's coordinate system,
/// all components are applied in the inverse order they are listed here.
/// E.g. if both a translation and a max3x3 transform are present,
/// the 3x3 matrix is applied first, followed by the translation.
///
/// \example archetypes/leaf_transforms3d_combined title="Regular & leaf transform in tandom" image="https://static.rerun.io/leaf_transform3d/41674f0082d6de489f8a1cd1583f60f6b5820ddf/1200w.png"
table LeafTransforms3D (
"attr.rust.derive": "Default, PartialEq",
"attr.rust.generate_field_info",
"attr.docs.category": "Spatial 3D",
"attr.docs.view_types": "Spatial3DView, Spatial2DView: if logged above active projection"
) {
/// Translation vectors.
translations: [rerun.components.LeafTranslation3D] ("attr.rerun.component_optional", nullable, order: 1100);

/// Rotations via axis + angle.
rotation_axis_angles: [rerun.components.LeafRotationAxisAngle] ("attr.rerun.component_optional", nullable, order: 1200);

/// Rotations via quaternion.
quaternions: [rerun.components.LeafRotationQuat] ("attr.rerun.component_optional", nullable, order: 1300);

/// Scaling factors.
scales: [rerun.components.LeafScale3D] ("attr.rerun.component_optional", nullable, order: 1400);

/// 3x3 transformation matrices.
mat3x3: [rerun.components.LeafTransformMat3x3] ("attr.rerun.component_optional", nullable, order: 1500);

// TODO(andreas): Support TransformRelation?
// TODO(andreas): Support axis_length?
}
4 changes: 4 additions & 0 deletions crates/store/re_types/definitions/rerun/archetypes/mesh3d.fbs
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,12 @@ namespace rerun.archetypes;
///
/// See also [archetypes.Asset3D].
///
/// If there are multiple [archetypes.LeafTransforms3D] instances logged to the same entity as a mesh,
/// an instance of the mesh will be drawn for each transform.
///
/// \example archetypes/mesh3d_indexed title="Simple indexed 3D mesh" image="https://static.rerun.io/mesh3d_simple/e1e5fd97265daf0d0bc7b782d862f19086fd6975/1200w.png"
/// \example archetypes/mesh3d_partial_updates !api title="3D mesh with partial updates" image="https://static.rerun.io/mesh3d_partial_updates/a11e4accb0257dcd9531867b7e1d6fd5e3bee5c3/1200w.png"
/// \example archetypes/mesh3d_leaf_transforms3d title="3D mesh with leaf transforms" image="https://static.rerun.io/mesh3d_leaf_transforms3d/c2d0ee033129da53168f5705625a9b033f3a3d61/1200w.png"
table Mesh3D (
"attr.rust.derive": "PartialEq",
"attr.docs.category": "Spatial 3D",
Expand Down
1 change: 0 additions & 1 deletion crates/store/re_types/definitions/rerun/components.fbs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,12 @@ table RotationAxisAngle (
) {
rotation: rerun.datatypes.RotationAxisAngle (order: 100);
}

/// 3D rotation represented by a rotation around a given axis that doesn't propagate in the transform hierarchy.
table LeafRotationAxisAngle (
"attr.docs.unreleased",
"attr.rust.derive": "Default, Copy, PartialEq",
"attr.rust.repr": "transparent"
) {
rotation: rerun.datatypes.RotationAxisAngle (order: 100);
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,23 @@ namespace rerun.components;
/// A 3D rotation expressed as a quaternion.
///
/// Note: although the x,y,z,w components of the quaternion will be passed through to the
/// datastore as provided, when used in the Viewer Quaternions will always be normalized.
/// datastore as provided, when used in the Viewer, quaternions will always be normalized.
struct RotationQuat (
"attr.docs.unreleased",
"attr.rust.derive": "Default, Copy, PartialEq, bytemuck::Pod, bytemuck::Zeroable",
"attr.rust.repr": "transparent"
) {
quaternion: rerun.datatypes.Quaternion (order: 100);
}

/// A 3D rotation expressed as a quaternion that doesn't propagate in the transform hierarchy.
///
/// Note: although the x,y,z,w components of the quaternion will be passed through to the
/// datastore as provided, when used in the Viewer, quaternions will always be normalized.
struct LeafRotationQuat (
"attr.docs.unreleased",
"attr.rust.derive": "Default, Copy, PartialEq, bytemuck::Pod, bytemuck::Zeroable",
"attr.rust.repr": "transparent"
) {
quaternion: rerun.datatypes.Quaternion (order: 100);
}
13 changes: 13 additions & 0 deletions crates/store/re_types/definitions/rerun/components/scale3d.fbs
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,16 @@ struct Scale3D (
) {
scale: rerun.datatypes.Vec3D (order: 100);
}

/// A 3D scale factor that doesn't propagate in the transform hierarchy.
///
/// A scale of 1.0 means no scaling.
/// A scale of 2.0 means doubling the size.
/// Each component scales along the corresponding axis.
struct LeafScale3D (
"attr.docs.unreleased",
"attr.rust.derive": "Copy, PartialEq, bytemuck::Pod, bytemuck::Zeroable",
"attr.rust.repr": "transparent"
) {
scale: rerun.datatypes.Vec3D (order: 100);
}
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,46 @@ struct TransformMat3x3 (
matrix: rerun.datatypes.Mat3x3 (order: 100);
}


/// A 3x3 transformation matrix Matrix that doesn't propagate in the transform hierarchy.
///
/// 3x3 matrixes are able to represent any affine transformation in 3D space,
/// i.e. rotation, scaling, shearing, reflection etc.
///
/// Matrices in Rerun are stored as flat list of coefficients in column-major order:
/// ```text
/// column 0 column 1 column 2
/// -------------------------------------------------
/// row 0 | flat_columns[0] flat_columns[3] flat_columns[6]
/// row 1 | flat_columns[1] flat_columns[4] flat_columns[7]
/// row 2 | flat_columns[2] flat_columns[5] flat_columns[8]
/// ```
///
/// \py However, construction is done from a list of rows, which follows NumPy's convention:
/// \py ```python
/// \py np.testing.assert_array_equal(
/// \py rr.components.LeafTransformMat3x3([1, 2, 3, 4, 5, 6, 7, 8, 9]).flat_columns, np.array([1, 4, 7, 2, 5, 8, 3, 6, 9], dtype=np.float32)
/// \py )
/// \py np.testing.assert_array_equal(
/// \py rr.components.LeafTransformMat3x3([[1, 2, 3], [4, 5, 6], [7, 8, 9]]).flat_columns,
/// \py np.array([1, 4, 7, 2, 5, 8, 3, 6, 9], dtype=np.float32),
/// \py )
/// \py ```
/// \py If you want to construct a matrix from a list of columns instead, use the named `columns` parameter:
/// \py ```python
/// \py np.testing.assert_array_equal(
/// \py rr.components.LeafTransformMat3x3(columns=[1, 2, 3, 4, 5, 6, 7, 8, 9]).flat_columns,
/// \py np.array([1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=np.float32),
/// \py )
/// \py np.testing.assert_array_equal(
/// \py rr.components.LeafTransformMat3x3(columns=[[1, 2, 3], [4, 5, 6], [7, 8, 9]]).flat_columns,
/// \py np.array([1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=np.float32),
/// \py )
/// \py ```
struct LeafTransformMat3x3 (
"attr.docs.unreleased",
"attr.rust.derive": "Default, Copy, PartialEq, bytemuck::Pod, bytemuck::Zeroable",
"attr.rust.repr": "transparent"
) {
matrix: rerun.datatypes.Mat3x3 (order: 100);
}
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@ enum TransformRelation: byte (
) {
/// The transform describes how to transform into the parent entity's space.
///
/// E.g. a translation of (0, 1, 0) with this `TransformRelation` logged at `parent/child` means
/// E.g. a translation of (0, 1, 0) with this [components.TransformRelation] logged at `parent/child` means
/// that from the point of view of `parent`, `parent/child` is translated 1 unit along `parent`'s Y axis.
/// From perspective of `parent/child`, the `parent` entity is translated -1 unit along `parent/child`'s Y axis.
ParentFromChild(default),

/// The transform describes how to transform into the child entity's space.
///
/// E.g. a translation of (0, 1, 0) with this `TransformRelation` logged at `parent/child` means
/// E.g. a translation of (0, 1, 0) with this [components.TransformRelation] logged at `parent/child` means
/// that from the point of view of `parent`, `parent/child` is translated -1 unit along `parent`'s Y axis.
/// From perspective of `parent/child`, the `parent` entity is translated 1 unit along `parent/child`'s Y axis.
ChildFromParent,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,12 @@ struct Translation3D (
) {
vector: rerun.datatypes.Vec3D (order: 100);
}

/// A translation vector in 3D space that doesn't propagate in the transform hierarchy.
struct LeafTranslation3D (
"attr.docs.unreleased",
"attr.rust.derive": "Default, Copy, PartialEq, bytemuck::Pod, bytemuck::Zeroable",
"attr.rust.repr": "transparent"
) {
vector: rerun.datatypes.Vec3D (order: 100);
}
1 change: 1 addition & 0 deletions crates/store/re_types/src/archetypes/.gitattributes

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit cdeac62

Please sign in to comment.