Importing Animation to Identical Rig Breaks It

[xMakerx]

Godot version

4.0.beta12.mono.official [3c9bf4b]

System information

Windows 11, RTX 2060 SUPER, Forward+

Issue description

This is the result when I import an animation I saved on an identical model in a different scene.

Here is what the animation looks like in the scene where the animation is saved.

I have a model with animations broken up into separate glTF files. Each file has an identical setup except for a differing animation. My expectation is that when I import an animation saved from within another scene, I expect the animation to work the same as it does in the scene that saved it. This is NOT the case, and, as you'll see, when I import the animation into the animation library of the base model I get a "slanted" result.

Steps to reproduce

How to Reproduce w/Reproduction Project

1. Open "tt_r_chr_dgs_skirt_neutral.gltf" in a new scene. MAKE SURE THIS SCENE ISN'T INHERITED -- no yellow text in the explorer
2. Select the AnimationPlayer, open the "Animation" pane, and hit the Play button. You should see a basic idle animation playing.
3. Save the animation by hitting Animation > Manage Animations.. > and hit the floppy disk icon to the right of the "default" animation. Save this animation and close the scene. You're done in this scene.
4. Open "tt_r_chr_dgs_skirt.gltf" in a new scene. MAKE SURE THIS SCENE ISN'T INHERITED -- no yellow text in the explorer
5. Add an AnimationPlayer node underneath the root node "tt_r_chr_dgs_skirt". The AnimationPlayer should automatically fill in the Root Node after you create it.
6. Select the new AnimationPlayer node and head to the "Animation" pane. You should have an empty Animation editor.
7. Open the animation you saved earlier by hitting Animation > Manage Animations.. > Add Library. You can call this library whatever you want. Now, hit the folder icon and browse to the animation you saved earlier. Hit OK and play the animation. You should see that the imported animation is slanted.

Minimal reproduction project

ReproductionProject.zip

NOTE: YOU DO NOT NEED THE MATERIALS OR THE SCRIPT TO REPRODUCE THIS ISSUE. IGNORE ANY DEPENDENCY ISSUES

[fire]

1. this is not a humanoid following the godot humanoid profile
2. This looks like a Blender Rigify skeleton so it has non connected skeletons which doesn't fit the godot engine profile
3. Since the retargeting feature only works on humanoid profiles, bone and animations cannot be automatically renamed or made matching
4. Godot Engine 4.0 does not have a IK retargeting functionality.
5. Also

[fire]

Let me know how to improve your experience of Godot Engine

[xMakerx]

I wouldn't consider this "retargeting" as it's the same model and all I'm doing is importing all animations into one AnimationPlayer to be played back. This is something other engines have no problem doing without the need of spending the time creating a profile of some sort.

I see "retargeting" as when I have two distinct rigs and I'm trying to adapt an animation to work on a rig it wasn't made for. This isn't the case here. It should all be drag and drop and seamless since the only thing different is the scene.

[lyuma]

The bones in the two armatures are rotated differently. A 30 degree rotation in the def_right_hip in the left scene, means something different than a 30 degree rotation in the right scene. Similarly, a 34 degree rotation in the def_right_knee does something different in the left scene and the right scene. See here what happened after I tried to make the hip and knee euler angles match by hand:

In fact it gives sort of the opposite of the lean that you saw in the original question (since I was copying the rotations in the reverse direction) This experiment shows that the armatures are not the same.

Godot has no way of knowing whether when the animation says "rotate def_right_hip to 30 degrees, you want it like the left scene or like the right scene, since those are both 30 degrees in their respective armatures.

There are two options:

1. either you go back to the content creation tool and edit it so that the bones are rotated the same way. For example, in blender, as long as the rest (edit mode) pose matches, this should work.
2. or use a retargeting system to copy animations between the two rigs, as Fire was hinting. There are additional bones not accounted for in the default humanoid profile, and the default profile expects spine to be a child of pelvis, so you will likely want to make that change to make it compatible, or create a custom profile.

This is something other engines have no problem doing without the need of spending the time creating a profile of some sort.

I would be interested to see your workflow for getting these models into the "other engine" (Which engine is it?)
I believe unless you use retargeting, you will run into similar problems in this specific example because of the mismatched bone rotations (the fact that 30 degrees means different things in each model)

[xMakerx]

Both Panda3D and Unity don't suffer from this issue when importing a base model and separate animation files. They must have some sort of normalization going on under the hood to make it work.

Models were created in Maya and exported with maya2glTF.

[xMakerx]

@fire @lyuma After sharing this with a friend who has a ton of experience in Panda3D, he thinks the issue is due to how Godot creates a unique armature for each glTF file. Godot needs a way to import just the animation data from a glTF file without generating a new armature. The base file should define the armature and subsequent animation files should provide the data that moves that armature.

[lyuma]

@xMakerx if it works in Unity, would it be possible to share the Unity scenes, along with the imported model files and animation, including .meta? (You can do this with Export Package of the .unity, .anim / controller and .fbx/.glb or just right click the whole folder NOTE: When making the package, make sure to uncheck include dependencies to avoid inadvertently packaging up a bunch of unrelated script files.)

Seeing how it is set up in Unity such that it works will be helpful for me to understand where the Godot import workflow is breaking.

Godot needs a way to import just the animation data from a glTF file without generating a new armature.

That is how Godot already works, In fact, there's a dropdown in the import that says "Import As" and you can choose AnimationLibrary.

But you're running into trouble for some reason: the animation data from both files are like speaking different languages. A 30 degree rotation on one is different than a 30 degree rotation on the other. So I'm trying to figure out why. The Unity project could hold the key.

[TokageItLab]

For some animation systems, this may not be a problem if the animation file only takes out the difference in its rotation.

In Godot, there is currently a realtime retarget custom module fot handling that case, and I guess that sharing these animations will work well if it can be applied without a SkeletonProfile (or you adopt your model with SkeletonProfile ).

But currently there are no plans to do so yet. If you do not use that module nor reatarget-system, you will need to match the edit bone rotation in blender if the two models have different edit bone rotations in blender, as mentions above.

Edit bone rotation is a value automatically calculated from the Head-Tail positional relationship and Roll value. You can easily check it with the this add-on.

If you are using Maya, Maya has Local Rotation Axes like the concept of rest in Godot, but since Maya does not output rest as a glTF, some retargeting may be needed externally if you want to share the animation output from it. @reduz is more knowledgeable about Maya.

[xMakerx]

@lyuma I was able to get this working in Unity using their Legacy animation system. So, I didn't have to worry about creating an avatar profile or anything like that.

I've attached the Unity package for you to look at as well.
UnityAnimationExample.zip

I wanted to add that taking the .FBX files and using Godot's official fbx2gltf converter to get the files imported into Godot does not fix this problem. I tried making the animation file import as an Animation Library as suggested to no avail. This is a problem with how Godot is treating these files. @TokageItLab @fire

[lyuma]

Thanks for the Unity (.fbx) testcase. Here is my progress report so far...

I can see that the FBX versions of the models play fine in Unity, as do versions converted using FBX2glTF.

I was also able to reproduce the same problem you saw in Godot by playing the animation from your attached glTF file, onto a converted FBX in Unity, manually replacing the conflicting bone names with colons (I may have messed this up, since I did this part by hand):

So something does seem to be wrong with the .gltf models in your attached project, possibly. Since glTF is Godot's format of choice, this is still a workflow problem that affects Godot and needs to be figured out.

But... now is the more interesting thing I discovered.

I used FBX2glTF to convert models into .glb format. The animation here is called "Take 001":
test_gltf.zip
I am able to import them into Unity 2020.3.x or later, using the VRM unitypackage from https://github.com/vrm-c/UniVRM/releases
Here is Unity, playing the animation from the converted neutral.glb on the original .glb

Here is Godot, playing the same animation from the converted neutral.glb on the original .glb (I used "Import As" -> "AnimationLibrary" for this test, but I do not believe it makes a difference.)

So there is probably a bug in Godot here.

As another testcase, I made two more glb files which work fine in Godot, Unity and windows 3D Viewer:
animations_merged_good_in_godot.zip
One of these working .glb was made by using UniGLTF to export the gltf from Unity with the animation on it.
The other was made by hand in VS Code by copying bufferViews, samplers, animations, meshes and skins from one to the other, but keeping the original node tree. This rules out changes to the node tree.

Here's the test project with all 4 glb files in it (and one more copy named _neutral_anim.glb set to import as AnimationLibrary.
gltf_test_project.zip

Anyway, so far it still seems like a Godot bug, but I will need to continue investigating. Now that we have working and non-working examples in the same project, it might be easier to compare and see what's different between them.

[lyuma]

Sorry I didn't reply sooner. We did determine the issue. The fact that swapping out the node hierarchy from one file with the other proved to be a key clue in uncovering the cause.

So the issue is that there is an animation track affecting the node "root". This animation track has one keyframe, which sets the root node to a rotation of (0°, 0°, 6.3°)

The thing is, due to the default export settings from the DCC tool used (in this case, maya. but I'm pretty sure blender would do the same thing), gltf models are posed in the first frame of its animation when exporting.
The issue with Godot is, it tries to remove animation tracks that don't animate anything. This cleans up a lot of tracks and for most files, it's nice to edit animations like this.

Here are the screenshots showing the issue.
Here is Godot beta 14. Look how clean these tracks are! But there's one missing.... root Rotation isn't there.

Here is how the FBX Unity project shows up. See how many unnecessary position, rotation and scale tracks there are. But here you can see we do have the 6.3 degree rotation about Z:

And finally, here is the "fixed" Godot build with my new PR #72007

You can see how all these unnecessary tracks now show up in Godot, but we do have our 6.3 degree rotation and it technically makes this animation play correctly.

So next step is to review the PR and understand the implications.

Thanks once again so much for the report, the extensive testcase, and for providing the working Unity project. I think all of this detail has given me and @fire a complete understanding of the cause and a proposed fix.

[xMakerx]

I appreciate the update! I wanted to ask what the Optimize Animation option does if the default behavior is to strip tracks. Maybe this functionality should be available but under the Optimization options? Instead of a single checkbox maybe there's different optimization levels?

I think by default, the Engine should recreate the input verbatim. Optimization should be a choice later on.

Animations have been the biggest headache for me in Godot, so this is actually incredible news that you and @fire figured out the issue. Thank thank thank you! I will be eagerly awaiting the official release with this included.