Animation Primitives

[james7132]

RENDERED

This RFC details the absolute bare minimum viable API for setting up a flexible and extendable property based animation system in Bevy... or at least part of it. This gets probably about 1/3 of the way there.

[alice-i-cecile]

Ping @lassade; I'm very curious on your opinions on this if you have the time and interest.

[alice-i-cecile]

Generally I'm very impressed. These are good abstractions that are powerful, flexible and build on each other.

[lassade]

a few notes

1. Not sure how curves and sampling could be decoupled into to separated traits, to me they are the same;
2. Sampling also involves how data should be interpolated: step, linear and catmul-row; I propose to have 3 different types of curves like in my original implementation;
3. For Quaternions there is a difference between lerp vs slerp, other than that generally the type it self defines how interpolate between values. You will almost never will want to use slerp by the way;
4. Color interpolation will be a nightmare If the Colors is defined as a enum that supports sRGB, LinSRGB, HSL and others, because each color space gives a different interpolation result; If you let the user choose the interpolation color space it might have a heavy performance cost, probably many times greater than using slerp;
5. Sampling should include some form of cursor (cache locality, think a curve with many keyframes) and should and use binary search (faster keyframe search). But you should only use the latter to make a simpler initial implementation;

[lassade]

In the storage domain you should include how streamed data will be handled and at mention available animation formats namely ACL (https://github.com/nfrechette/acl) in the prior art.

[james7132]

Almost completely revised the RFC end to end:

• Removed the mention of animation graph and mixing entirely, going to have a separate RFC for this
• Dropped the idea of Sample<T>, and instead opting for defining the base characteristics required for an animatable property instead, which will be the base for the composition API
• Dove deeper into the implementation details and memory layout.
• Addressed review comments from @alice-i-cecile and @lassade

Still quite a few TODOs, namely discussing a strategy for implementing AnimationClip and it's serialization.

---

To respond to @lassade's comments (next time please leave them inline so I can address them 1-by-1 instead of a giant block like this).

1. Not sure how curves and sampling could be decoupled into to separated traits, to me they are the same;

Good point, retracted that idea after discussing this with you on Discord and reading your Animator implementation. I've changed it now to instead have a trait on the sampled value named Animatable, which takes your Lerp and logically extends it one step further for allowing anyone to make their own blendable value. This will be used to blend sampled values from curves the next Animation Graph attempt I'm testing instead of relying on generic nodes as previously proposed. This should remove generics from everywhere except for the Curve<T> trait itself.

2. Sampling also involves how data should be interpolated: step, linear and catmul-row; I propose to have 3 different types of curves like in my original implementation;

I'm tentatively addressing this by trying to use a newtype and implement Animatable on it instead. Having T be a generic parameter to Curve instead of an associated type actually allows us to have multiple Curve<T> implementations for a given type, and we can unwrap the newtype in a blanket impl if we need to. Having multiple curve types was originally always in the plan, just didn't have time to write down the full details, but if the underlying structure of the curve does not change fundamentally change, but the interpolation behavior changes, we can just newtype the value type instead. There's an argument to be made here about SoA vs AoS in wrapping values like this, which is why the CurveVariable in your implementation is the way it is, but I'm not sure if SoA is the answer here, particularly since we're not speed-iterating through the curve, but occasionally sampling a value from it, sometimes randomly.

3. For Quaternions there is a difference between lerp vs slerp, other than that generally the type it self defines how interpolate between values. You will almost never will want to use slerp by the way;

This can be addressed by the newtype pattern that I mentioned above. Though I'm curious why we wouldn't want to use slerp other than performance.

4. Color interpolation will be a nightmare If the Colors is defined as a enum that supports sRGB, LinSRGB, HSL and others, because each color space gives a different interpolation result; If you let the user choose the interpolation color space it might have a heavy performance cost, probably many times greater than using slerp;

Right, and that's sort of an implementation detail that we need to address, and is sort of separate from the design detailed here. I'm well aware this is an issue, but it's something we need to resolve in general, not just for animation.

5. Sampling should include some form of cursor (cache locality, think a curve with many keyframes) and should and use binary search (faster keyframe search). But you should only use the latter to make a simpler initial implementation;

I added a section on this, though I'm not 100% sold on it. After reading your implementation, I get the distinct impression that this very heavily complicates the implementation as the cursor must be mutably saved somewhere, and you might have the same curve/clip being sampled in different locations by multiple entities (or even within the same entity). That's not to say I think we should definitely not add it in, but it does seem like an optimization we can add in after we deliver a MVP on this.

In the storage domain you should include how streamed data will be handled and at mention available animation formats namely ACL (https://github.com/nfrechette/acl) in the prior art.

Great point! Added that a mention of ACL in. I need to rewrite that section to mention the other efforts in the Rust community and other public literature about the approaches to this.

[Weibye]

To me this looks like flexible and powerful primitives to build on top of. I particularly like how this would build animation in bevy from the ground up to be able to animate anything and not just skeletal animation.

• Obvious outstanding TODOs
• Don't feel I have enough experience to give feedback on the suggested technical implementation details
• It is clearly and concisely written, so even with limited technical knowledge it is easy to read and understand

Overall this is great work, thank you!

[ValorZard]

Looking at the curve system as explained in the RFC, it reminds me of how Godot does animations, specifically how Godot lets you animate literally everything including UI all using the same UI graph. I'm wondering, will the Curve syntax work for everything, or only for certain things like skeletal animation?

[Aceeri]

I like the concept, I'm going to think about it for a bit but here are a couple of nitpick-y grammar/etc.

[Aceeri]

If it can only be sampled with non-negative time then wouldn't it only be clamped to duration here?

[Aceeri]

Suggested change

	surroudning keyframe before sampling a value from the curve itself. This can
	surrounding keyframe before sampling a value from the curve itself. This can

[Aceeri]

Suggested change

	it's useful to compress the most usecases seen by a
	it's useful to compress the usecases most likely to be seen by a user.

[Aceeri]

Suggested change

	#[derive(Debug, Clone, PartialEq, Eq, Hash, Ord, ParitialOrd)]
	#[derive(Debug, Clone, PartialEq, Eq, Hash, Ord, PartialOrd)]

[Aceeri]

Suggested change

	#[derive(Debug, Clone, PartialEq, Eq, Hash, Ord, ParitialOrd)]
	#[derive(Debug, Clone, PartialEq, Eq, Hash, Ord, PartialOrd)]

[Aceeri]

Suggested change

	A `AnimationClip` is a serializable asset type that encompasses a mapping of
	An `AnimationClip` is a serializable asset type that encompasses a mapping of

[Aceeri]

Suggested change

	facturing of the ecosystem with multiple incompatible baseline animation system
	fracturing of the ecosystem with multiple incompatible baseline animation system

[Aceeri]

Suggested change

	mulitple cache misses when sampling from animation clips?
	multiple cache misses when sampling from animation clips?

[mweatherley]

I might be a couple years late to the party here, but I have some thoughts on Curve<T> as associated with other curve-related matters that have recently been on my mind.

In investigating matters related to, e.g. mesh generation by curve extrusion, I have also come up against curve sampling and similar questions related to what appear here as CurveFixed etc. For instance, in order to perform acceptable mesh extrusion for curves in general, one really needs at least the following in addition to the points on the curve:

1. Tangents — ideally with regularity assumptions, so that, for example, T = Dir3 is supported for many curves we construct in addition to T = Vec3 (or equivalent)
2. Arclength (T = f32 or similar)
3. Rotation-minimizing frames, given by T = Mat3, T = Quat (interpreted as orientation quaternions) or similar, with something like spherical interpolation papering over the gaps

Notably, for curves constructed by splines, (1) is available algebraically (hence for arbitrary parameter values with no interpolation necessary), while (2) and (3) must generally be constructed by sampling (although for some simple classes of curves they could also be algebraic). Here are what I see as the main differences:

• For these kinds of application in particular, sampling arbitrary time parameters is often less important, since the resolution can often be bound to the task at hand directly — if I want to generate a mesh with some given curve-space resolution, I can just sample the values I need at that frequency (albeit perhaps with some substeps to avoid numerical error) in the first place
• Insofar as interpolation is needed, it would generally consist only of Animatable::interpolate
• Concrete Curve implementations will often blend discrete data (possibly interpolated) with data that can be exactly computed from underlying algebraic structures

I think the good news is that these differences are mostly easy to accommodate. For the first, it is not really much of a rift, as long as the Curve<T> implementor has its resolution specified at the time of construction anyway.

For the third, I see this mostly as creating difficulties when serializing and deserializing. For example, something like type erasing the underlying algebraic structure, if necessary, could require passage from exact information to discrete samples; luckily, if we come to that bridge, the other kinds of data that we need are still "compatible" in the sense that they could be recomputed from discrete data anyway (that is to say that RMF, for instance, really only needs a Curve<(Vec3, Dir3)> in its construction).

The second is where I wanted to raise a minor point. I am against the exact definition of Curve<T> in this proposal only because it requires T: Animatable rather than something more general (especially with a name as all-encompassing as Curve). Concretely, what I would prefer is to do the following:

• Introduce a weaker trait, Interpolable (or similar) in bevy_math, with Animatable: Interpolable in bevy_animation
• Introduce the trait Curve<T: Interpolable> in bevy_math and implement it for sensible values of T with our spline output curves
• Have animation depend on Curve<T> where T: Animatable explicitly

I think that this would provide a solid common baseline (at least at the level of API) for the distinct areas.

(By the way, thank you for writing this, and sorry if I am verbose.)