Depth API

[SixArne]

Hi,

Are there any plans on implementing a depth API for meta devices?

I would love to contribute and help but I have never worked on open source before.
(I do have a very solid knowledge of C++)

I expected to find the Meta Native SDK somewhere in the repository but I couldn't find it.

Regards,

[dsnopek]

We really want to implement the Depth API, and will get around to it eventually, but no one has really started to look into this yet. If you'd like to give it a try, that'd be great!

There's a possibility that this will require making some changes to Godot itself, but there's no way to tell without digging into it.

I expected to find the Meta Native SDK somewhere in the repository but I couldn't find it.

We do include the Meta headers - you can find the header for the necessary OpenXR extension here:

https://github.com/GodotVR/godot_openxr_vendors/blob/master/thirdparty/ovr_openxr_mobile_sdk/OpenXR/Include/openxr/meta_environment_depth.h

[dsnopek]

I've been researching this a tiny bit, and here's some links to resources:

• Meta's docs: https://developers.meta.com/horizon/documentation/native/android/mobile-depth
• Sample using it: https://github.com/meta-quest/Meta-OpenXR-SDK/tree/main/Samples/XrSamples/XrPassthroughOcclusion

Looking through the sample, it looks like we're expected to change our fragment shader to consult the depth texture that we get from the OpenXR extension. Here's the relevant bit: https://github.com/meta-quest/Meta-OpenXR-SDK/blob/main/Samples/XrSamples/XrPassthroughOcclusion/Src/XrPassthroughOcclusionGl.cpp#L632

In order to make this work, we'll either need to:

1. Modify Godot so that the builtin shader templates have the ability to do this when given a depth texture, and then allow GDExtension to provide that texture to Godot, or
2. Require that users modify all of their shaders to do this, and then godot_openxr_vendors just makes this depth texture available for users to do that

Nr 2 could potentially be made easier if Godot allowed users to override Godot's shader templates - see PR godotengine/godot#94427

In any case, this won't be the easiest OpenXR extension to integrate with Godot

[maunvz]

I'm wondering if adding a Compositor Effect to the XRCamera3D to post process the eye buffers with the depth information might work... Still reading up on how they work to see if that's an option, but I imagine it'd be less efficient than doing this in a single pass with the builtin shaders

[dsnopek]

There are now two WIP PRs attempting to implement this:

• [WIP] Integrate XR_META_environment_depth #230
• Another WIP for the FB_environment_depth extension #232

It would be nice if we could get the approach of using global uniforms from PR #232 working. It's not the most ideal developer experience (you need to use a custom shader for all materials), but it doesn't require changing Godot itself, and should be efficient performance-wise.