🚀 Build CaptureSession with lazy-initialized Surfaces (performance)

[mrousavy]

What feature or enhancement are you suggesting?

The Camera2 framework allows creating OutputConfigurations that do not have a Surface yet.

In this case, the OutputConfiguration is considered lazy, and cannot be streamed to yet (omit it from capture request), but it can already be used to initialize the CameraCaptureSession, in which case the initialization of the actual Surface can happen asynchronously/in parallel.

This is especially useful for the Preview View, which can take a few milliseconds to initialize as this is scheduled on the UI Thread by Android OS.

If we implement this optimized flow in VisionCamera, we can speed up session creation even more by allowing the session to be started without a finalized Preview View yet, and then only finalize the configuration once the Preview View has actually been started.
Additionally, we currently always have two session configurations happening on load - one with all outputs but the preview, and one with all outputs including the preview. When we implement this optimized flow in VisionCamera we can skip the first call, effectively making this even faster than it already will be and lowering memory usage since we avoid unnecessary allocations.

There are two things that need to be done about this:

• PreviewView needs to be a special output, since we don't want it's initialization to trigger a configureOutputs call (as that will tear down every output and rebuild a session, effectively making this optimization useless)
• On surface creation we need to call finalizeOutputConfiguration on the Camera Session.
• We still have the third state, DESTROYED, for the Preview View as once the Surface is destroyed by Android OS it cannot be used as a lazy surface again - in this case we need to rebuild everything as expected.

What Platforms whould this feature/enhancement affect?

Android

Alternatives/Workarounds

Works

Additional information

• I agree to follow this project's Code of Conduct
• I searched for similar feature requests in this repository and found none.

[mrousavy]

Implemented in VisionCamera V4!