🐛 Preset prop has no effect on Android

[Kypsis]

What were you trying to do?

Setting the preset has no effect on recording quality on Android. Whatever preset is chosen, recorded quality is always 19201080 FullHD. This also somewhat relatedly raises the question is the frameBuffer, surface(preview) and recording the same size. Sidenote, I remember being able to change each of those to be independent in Flutter with a library hack (preview on Android was capped to 1280720 max for whatever reason).

What happened instead?

Nothing happens :(

Relevant log output

No response

Device

Pixel 4a

VisionCamera Version

latest master

Additional information

• I am using Expo
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[mrousavy]

preset is iOS only. To change the recording size, use a custom format

[Kypsis]

The format works, kinda, even if it's somewhat clunky to use. There is an issue with format though that no matter the resolution picked the recorded file size always ends up at ca 30MB. Also even though the preview seems to respect higher resolutions (gets that zoomed in effect if using 4:3 formats) the recording itself seems to be capped at FullHD (1920x1080). Using 30 fps and yuv in all of the tests.

[mrousavy]

even if it's somewhat clunky to use

why?

recording itself seems to be capped at FullHD

I think that's an issue by the native CameraX library from Google, maybe the recording doesn't support changing resolution yet? You could play around with the following code:

react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt

Lines 359 to 362 in 1c32726

	val aspectRatio = aspectRatio(format.photoSize.width, format.photoSize.height)
	previewBuilder.setTargetAspectRatio(aspectRatio)
	imageCaptureBuilder.setDefaultResolution(format.photoSize)
	videoCaptureBuilder.setDefaultResolution(format.photoSize)

[mrousavy]

working on a fix here: #325

[Kypsis]

That actually works to unlock proper aspect ratios for recordings. Still this:

format={
              device?.formats?.filter(
                format =>
                  format.videoWidth === 4032 &&
                  format.videoHeight === 3024 &&
                  format.colorSpaces.includes("yuv"),
              )[0]
            }

records at 1600x1200 max so might indeed be CameraX issue. Still much better than it getting locked to 1920x1080. As for presets on android vs just only format, ease of use and sensible defaults are always compelling use cases🙂

[mrousavy]

As for presets on android vs just only format, ease of use and sensible defaults are always compelling use cases🙂

preset is a property of the native iOS camera library only. Android's CameraX library does not have this feature, if you want support for this feature you can implement some sort of preset resolver/mapper for Android yourself, VisionCamera is an open source project I work on in my free time

[mrousavy]

I'll try to find out why CameraX is locked to 1920x1080 🤔

[Kypsis]

As for presets on android vs just only format, ease of use and sensible defaults are always compelling use cases🙂

preset is a property of the native iOS camera library only. Android's CameraX library does not have this feature, if you want support for this feature you can implement some sort of preset resolver/mapper for Android yourself, VisionCamera is an open source project I work on in my free time

I was kinda afraid that is the case. Still it's not a showstopper or anything just a nice convenience feature.

As for the other, it is not locked to 1920x1080 anymore. Just the upper range got capped to 1920x1080 previously (now the upper range at least follows aspect ratio of the given format, so 4032x3024 is 1600x 1200). You could still go lower (even before the proposed change) like 1280x720 and that would work.

[mrousavy]

btw instead of filter(x => true)[0] you can use find(x => true), which is less code, cleaner to read and faster. I'd still use useMemo to memoize this and avoid running this filtering on every re-render.

Oh btw you're running VisionCamera from master right? Could you please go into CameraPage.tsx and disable photo and video and always enable the frameProcessor and let me know if the app works and the Frame Processor logs to the Metro Bundler console?

[Kypsis]

I'm actually running vision camera on a feature branch of a production app and both the VisionCamera master. But will try asap.

[mrousavy]

Because somehow I'm getting this error on Android since I've added console logging support for frame processors (this commit: 0f7ee51):

wondering if you're seeing the same or if console.logging works for you

[Kypsis]

Seems to work okay.

[mrousavy]

wow, amazing. Think I can promote the beta to a stable release now?

I also think it's amazing that your camera says 60 FPS since I thought CameraX only supports 30 FPS? 🤨

[Kypsis]

There's like 84 formats to choose from and alot of them have fps ranges from 15 to 60 on Pixel 4a. 😅
As for stable I think all the features more or less work as advertised (and quite a lot are out of your hands when it comes to CameraX bugs) so from my POV it looks good. 👍

[Kypsis]

What format is selected and what actually gets recorded is kinda weird though. 320x240 format => recording 352x288, 640x480 => 1024x768, 1280x720 => 1600x1200, 1920x1080 => 1600x1200, no format(default) = >1920x1080. All formats yuv.
Also quite the data rate difference:

[mrousavy]

oh yeah I see - that's because CameraX doesn't expose the video width/height in a format - everything's loosely coupled. I'll try to open a feature request for CameraX, but this is out of my hands

[Kypsis]

I reverted the #325 changes just to verify, but without #325 changes the 1280x720 format =>

Which is interesting because after #325 you get 1600x1200. CameraX certainly has bugs a plenty.

[mrousavy]

Hey! I've rewritten the entire Android codebase of VisionCamera from CameraX to Camera2 in the efforts of ✨ VisionCamera V3.

I just now completed the Camera2 rewrite and I believe the core structure is running, but there might be some edge cases to iron out. Can you try and test the PR #1674 for me to see if you can still reproduce this issue here?

Here's an instruction on how you can test that: #1674 (comment)

If the issue cannot be reproduced with that version/PR anymore, then hoorayy, I fixed it! 🎉
Otherwise please let me know and I'll keep this issue open to keep track of it.

Thank you!