Using GLTF2 loaded files with <primitive object=...> in ThreeJS Fiber

[alankent]

I am trying to use the current version of the OSSOS library with ThreeJS to take advantage of the retargeting and IK support. I am doing so inside a React app using threeJS Fiber. So I have a <primitive object=...> that I used to use the threejs GLTF loader to load a file and then grab the .scene member to use as the root character.

But the data structure returned by the OSSOS library GLTF2 file loader appears to be a bit different. For example, there is .json instead, and .json.scene exists but has the value of 0, which is not a model.

Can I use OSSOS to get an object compatible with the <primitive object=...> markup used with threejs and fiber?

Note: I am not a deep expert. I was trying to use the threejs retargeting sample code, but it did not work for me. I want to take an animation clip from one model and retarget it to a different bone structure. My fallback is to try and reimplement the whole bone rotation logic you described on the video (which conceptually made sense... but I would rather not have to code it, lol!)

I was trying to apply Mixamo animation clips to a VRM model created by VRoid Studio in case that makes a difference.

Thx for any advice! I could not find much in the way of docs to work it out sorry.

[sketchpunk]

The GLTF loader i made is for piecemeal parsing of content, not a scene loader. You can technically use the loader thats part of 3js, but you'll need to go through the trouble of finding the model & its skeleton. Then you'll need to convert that 3js skeleton to an ossos armature objection to use anything in the library. You can continue using the 3js skeleton but you'll need a function that will transfer an ossos pose to a 3js skeleton. The next version of Ossos has an example of mixing Ossos's Armature and 3JS's Skeleton. It should still mostly work for the previous version, maybe with a few tweaks.

https://github.com/sketchpunklabs/ossos/blob/ossos_next/prototypes/armature/002_gltf_3js_pose.html

Download the line I can try to make an example of loading using 3js's GLTF loader.

When it comes to react or fiber, I have no experience with them. Ossos is meant to be engine agnostic, so its always going to take the developer some work to get it to run in their perspective engines. So far i've only made examples for vanilla threejs and babylonjs.

The other big hitch is Ossos isn't ready for prime time yet and it might take awhile to get there.

Side note, Mixamo animations can be difficult to retarget. The data comes in with a ZUp orientation. I believe I add a root rotation offset, kind of like an additive to the root bone so that it gets reoriented to YUp to make it easier to do regular or ik retargeting.

[alankent]

Looking at it another way, I have all the rendering etc working in pure 3js. All I want is to retarget an animation clip from one model to another (both glb files). I have the bone name hierarchies for both (I know the to and from bone names). Would it be less work if I instead try to just use that part of the OSSOS code and just call that function to do all the bone transforms? I can then copy that into 3js animation clips (tracks are basically name + array of time offsets + positions/scales/quaternions). (I can load the GLB file with OSSOS, retarget the animation clip, copy to 3js, discard the the GLB file loaded.)

And yes, Y is up vs Z is up is a problem in my project too. I am trying to generate GLB and USD animation clips. In the USD world, it works with UE, Unity, Maya, ... so lots of assets have different concepts of "up".

[alankent]

(I was also interested in the IK and spring bone code in the future, but right now I was just trying to retarget an animation clip.)