Draco and MeshOpt extensions

[bertt]

Hi, whats the status of using Draco compression? I see 5-8 times size reduction when using Draco, so ideally would like to enable it when saving to glb file.

[vpenades]

I have to admit the prospect of adding draco compression is not very good.

I've been thinking a lot about different approaches to support Draco, and this is what I came with:

First and foremost:

• I would like to keep the library 100% C# code whenever possible. Not only for platform compatibility but also from the maintenance perspective.
• It is really needed? I can see a need to be able to read draco compressed files, because that would give the library the capability of consuming all the models in the wild. While saving models with draco compression, although desirable, it should not be a showstopper.
• Draco compression is not that widely supported yet: There's very few engines or frameworks that do support draco compression, and it's actually easier to find libraries and engines that can't read draco compressed files. so if you want to keep your files compatible, you better keep them uncompressed. Draco compression is only recomended when you know in advance the loader will support it.

Given these considerations, these are the possible solutions I am considering:

• Write a C# draco decoder that will be used to read draco compressed files, but without being able to write draco compressed files.
• Add some hooks at key points of the library so when reading/writing draco compressed blocks, it would call another library, or a third party library, that would do the encoding/decoding.
• Hook into the glTF pipeline, so, when dealing with draco, the files will go through the glTF pipeline to encode/decode draco as a pre/post processing step.

For now I am not considering binding the core library to the draco native DLLs.

Things would change dramatically if someone develops a c# implementation of Draco, but I don't see that possibility in the foreseeable future

Finally, although I would love to support Draco, it's way over the available time I have, and I preffer to invest it in other, more useful glTF extensions; I'm waiting for the Universal Basis texture extensions to be final to begin working on them.

So for now I'm afraid the best you can do is to write your glTFs to a temporary directory, and post-process them with the glTF pipeline or some other library able to write draco compressed files.

[bertt]

ok thanks for the explanation! At the moment after glTF/3D Tile creation I do a post process step (using gltf-pipeline) but its not optimal because of the temporary files involved .

[vpenades]

At some point, when I have time (very busy right now) I'll look at Andress Attender draco for unity bindings...

[pigLoveRabbit520]

@vpenades
2023 now, Draco compression is very necessary.

[vpenades]

@pigLoveRabbit520 if you care to read this thread, I already explained the case.

Draco extension requires using Google's Draco compression library which is a very large and very complex C++ project, and the Draco Extension uses that project under the hood.

I don't have enough free time for neither port that draco project to c#, which would literaly take months of work, nor to write bindings and test them on all platforms. If someone takes care of porting the google library to c# I would be glad to support the extension, otherwise, it's beyond my pay check.

Other users are using gltf-pipeline to pre process or post process gltf's with draco. If you really need draco, I suggest you to use the pipeline to strip draco from the models you want to read before import them.

[pigLoveRabbit520]

ok

[danfma]

@vpenades Hey, man!

What if we add an encoding plugin support for your library? I don't know if we have that today. This way, people interested can create a plugin for your library for Draco or any other encoding we have now and in the future.

This way, your code, the Core, will still be 100% C# but you will let people extend your library to satisfy their needs.

For example, I want to use your library today but all models of our company now are using Draco.

For one specific case, if we have a way to read a model without decoding/encoding the texture that would be awesome, because I only need to get the boundaries of some models.

Other users are using gltf-pipeline to pre process or post process gltf's with draco. If you really need draco, I suggest you to use the pipeline to strip draco from the models you want to read before import them.

Yes, that is something that came into my mind as a workaround.

[danfma]

Something came into my mind and I will give it a try because that might solve our problems... Apparently, Draco is being exposed as WASM and, as far as I know, we can run WASM in .NET nowadays. This might be a way to use it without relying on any native library too.

[vpenades]

@danfma right now we're in the process to modify the library to make a bit easier to write extensions as third party libraries, but for something like Draco it would also require some entanglement with the loader code, so the buffers can be decompressed on load.

Using WASM is a possible solution, I guess you're referring to the WasmTime project.

In the meantime, it could be reasonably easy to allow the library to load gltf files with draco by completely skipping validation. Of course that would produce an invalid and unusable model, but then, an external code could run an additional post load step over the model, and decompress and fix the draco buffers to make the model usable again.

In the long run, I guess it could be possible to include some interfaces that could be called by extensions on load, to allow this kind of post processing steps to be run after load, but before validation, so compression extensions could have the right hook to work.

For images, there's also a hook to redirect what you want to do with them on load

[danfma]

My time is a bit limited but count on me if you need some help!

[arcasoy]

Our team would also be interested in this functionality, as we use Draco compressed models primarily as well.

[vpenades]

I decided to move this long standing issue to a discussion to give it some visibility.

My current official position remains the same:

• The development cost/time of implementing draco remains too high for me to handle it on my own.
• glTF also has EXT_MeshOpt_Compression which serves the same functionality as draco, but it's better documented. So I would be more inclined to use this one instead of draco.

So if someone has better ideas, this is the right place to discuss, under the understanding that discussing it does not bind me to do the development.

As a provisional workaround, it is possible to use third party tools to post process the models:

• glTF Pack - Command Line download
• glTF Pipeline

[xiaoyuvax]

I tested and found this solution is better than glTFPack or glTF Pipeline, by using a nuget library to transcode compressed gltf to uncompressed (an intermediate tmp file is needed so far):
https://github.com/EverseDevelopment/DracoWrapper
it is a wrapper project to use google's Draco Compression algorithm with C#
note:despite the nuget package is for netfx, but it does work for net core (net8 passed, just ignore the warning).

here's the code that worked for me:

var decoder = new GltfDecoder();
var res = decoder.DecodeFromFileToScene(fileName);
Scene scene = res.Value();

// Note here: don't set compressOption, if u want it to be saved as decompressed.
//DracoCompressionOptions options = new(){}                        
//SceneUtils.SetDracoCompressionOptions(options, scene);

var encoder = new GltfEncoder();
var tmpFile = Path.Combine(Path.GetDirectoryName(fileName), "~" + Path.GetFileName(fileName));
encoder.EncodeSceneToFile(scene, tmpFile);

var model = SharpGLTF.Schema2.ModelRoot.Load(tmpFile);

Voila! Bingo!

I hope the author of this wrapper enable decodeToBuffer() method or certain decodeToStream() for SharpGLTF to receive the decompressed filestream.
in order to realize that, can u add an overload to ModelRoot.Load() as to receive stream input?

[vpenades]

that's definitely some progress, and worth checking it out, but still a long ways to go.

DracoWrapper seems to be compiled against net471 exclusively, whereas SharpGLTF is cross platform... also any implementation should not block the implementation of MeshOpt. So at the very least, any implementation of draco should go into a separate library, and communicate with the core with ad-hoc interfaces.

[xiaoyuvax]

To compile against netcore is not a problem, i've actually built one for net8, the .dll can be referenced properly but can not be loaded somehow(with exception thrown), no idea about the reason. i am not C++ expert, so if someone (better be the author) would like to look into it, the corss-platform problem may be sooner solved. Anyway,i've left comments in the issue board, hope they will see them asap.
or u may contact the author directly to discuss about further improvements.

Otherwise, i am interested to see if possible to port this wrapper to C# somehow with unsafe code.

[vpenades]

Upon closer inspection, DracoWrapper is definitely NOT a good option.

There's several approaches to bind C++ to C#... and one of them is the rarely used /CLR mixed assembly compilation option. This options allows a C++ assembly to be usable as a managed library directly, without any p/Invoke wrapping.

This is the approach used by DracoWrapper. More information can be found here.

So DracoWrapper is not really a wrapper, but the actual native Draco library compiled using the CLR compliant compile option.

CLR libraries have some heavy limitations, that would prevent the library to be usable in way too many scenarios to be worth the effort.

[xiaoyuvax]

I've guessed similarly after i dissassembled the DLL file, when trying to see if it's possible to port it to C#, for if it were a pure managed assembly, it'd be properly disassembled, anyway it's not what i'd expected.
well, it now turns out to be a slightly better in-process workaround for my windows project instead of out-of-process solutions such as glTFPack or glTF Pipeline.

btw, I guess it would not be so hard to fork this repo to turn it into a p/invoke wrapper, am i right?

[xiaoyuvax]

A pure C# port of Google Draco came out only 3 weeks ago, check if u can incorporate this into SharpGLTF? @vpenades
https://github.com/openize-drako/Openize.Drako-for-.NET

[vpenades]

It looks promising. Looking at the nuget package, it does not perfectly align with the supported frameworks, (netstandard2.0 not supported), but I think it can be worked out.