🐛 Android Black Screen

[Easy47]

What were you trying to do?

When I launch the example app, it just gives me a black screen. I can still press the buttons for flash and for swap but the flash does not work either and the swap does not remove the black screen.(I don't have this problem with version 2.13, only with 2.14).

It seems that I have the permissions and I have no error message, just a black screen on my phone with the buttons. I tried this #879 too but it the same.

I'm starting to wonder if this is a bug with 2.14 ? I have gone through the issues and everyone seems to have errors with the black screen.

Any idea ?
Thanks

Reproduceable Code

import * as React from 'react';
import { useRef, useState, useMemo, useCallback, useEffect } from 'react';

import { useIsFocused } from '@react-navigation/native';
import { StyleSheet, View, PermissionsAndroid } from 'react-native';
import { PinchGestureHandler, PinchGestureHandlerGestureEvent, TapGestureHandler, TouchableOpacity } from 'react-native-gesture-handler';
import Reanimated, { Extrapolate, interpolate, useAnimatedGestureHandler, useAnimatedProps, useSharedValue } from 'react-native-reanimated';
import IonIcon from 'react-native-vector-icons/Ionicons';
import {
  CameraDeviceFormat,
  PhotoFile,
  sortFormats,
  useCameraDevices,
  Camera as Cam,
  frameRateIncluded,
  VideoFile,
  useFrameProcessor
} from 'react-native-vision-camera';
import type { NativeStackScreenProps } from '@react-navigation/native-stack';
import type { Routes } from './Routes';
import { CONTENT_SPACING, SAFE_AREA_PADDING } from './Constants';
import { CaptureButton } from './views/CaptureButton';

import { scanOCR } from 'vision-camera-ocr';
const ReanimatedCamera = Reanimated.createAnimatedComponent(Cam);
Reanimated.addWhitelistedNativeProps({
  zoom: true,
});

const BUTTON_SIZE = 40;
const SCALE_FULL_ZOOM = 3;

type Props = NativeStackScreenProps<Routes, 'CameraPage'>;
export function CameraPage({ navigation }: Props): React.ReactElement {
  const camera = useRef<Cam>(null);
  const [isCameraInitialized, setIsCameraInitialized] = useState(false);
  const [cameraPosition, setCameraPosition] = useState<'front' | 'back'>('back');
  const [enableNightMode, setEnableNightMode] = useState(false);
  const [flash, setFlash] = useState<'off' | 'on'>('off');
  const isPressingButton = useSharedValue(false);

  const devices = useCameraDevices();
  const zoom = useSharedValue(0);
  const isFocussed = useIsFocused();

  const isActive = isFocussed;
  const device = devices[cameraPosition];

  const minZoom = device?.minZoom ?? 1;
  const maxZoom = Math.min(device?.maxZoom ?? 1, 20);
  const neutralZoom = device?.neutralZoom ?? 1;
  const supportsFlash = device?.hasFlash ?? false;

  const onMediaCaptured = useCallback(
    (media: PhotoFile | VideoFile, type: 'photo' | 'video') => {
      console.log(`Media captured! ${JSON.stringify(media)}`);
      navigation.navigate('MediaPage', {
        path: media.path,
        type: type,
      });
    },
    [navigation],
  );

  const onInitialized = useCallback(() => {
    console.log('Camera initialized!');
    setIsCameraInitialized(true);
    console.log("supeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeer");
  }, []);

  const onFlipCameraPressed = useCallback(() => {
    setCameraPosition((p) => (p === 'back' ? 'front' : 'back'));
  }, []);

  const onDoubleTap = useCallback(() => {
    onFlipCameraPressed();
  }, [onFlipCameraPressed]);

  const onFlashPressed = useCallback(() => {
    setFlash((f) => (f === 'off' ? 'on' : 'off'));
    console.log("sheesh");
    console.log(format);
  }, []);

  const onPinchGesture = useAnimatedGestureHandler<PinchGestureHandlerGestureEvent, { startZoom?: number }>({
    onStart: (_, context) => {
      context.startZoom = zoom.value;
    },
    onActive: (event, context) => {
      // we're trying to map the scale gesture to a linear zoom here
      const startZoom = context.startZoom ?? 0;
      const scale = interpolate(event.scale, [1 - 1 / SCALE_FULL_ZOOM, 1, SCALE_FULL_ZOOM], [-1, 0, 1], Extrapolate.CLAMP);
      zoom.value = interpolate(scale, [-1, 0, 1], [minZoom, startZoom, maxZoom], Extrapolate.CLAMP);
    },
  });

  //#region Callbacks
  const setIsPressingButton = useCallback(
    (_isPressingButton: boolean) => {
      isPressingButton.value = _isPressingButton;
    },
    [isPressingButton],
  );

  const formats = useMemo<CameraDeviceFormat[]>(() => {
    if (device?.formats == null) {
      return [];
    }
    return device.formats.sort(sortFormats);
  }, [device?.formats]);

  const supportsCameraFlipping = useMemo(() => devices.back != null && devices.front != null, [devices.back, devices.front]);

  const canToggleNightMode = enableNightMode ? true : device?.supportsLowLightBoost ?? false;

  const format = useMemo(() => {
    const result = formats.filter((x) => x.photoHeight < 1200 && x.photoWidth < 900);
    return result.find((f) => f.frameRateRanges.some((r) => frameRateIncluded(r, 30)));
  }, [formats]);

  const cameraAnimatedProps = useAnimatedProps(() => {
    const z = Math.max(Math.min(zoom.value, maxZoom), minZoom);
    return {
      zoom: z,
    };
  }, [maxZoom, minZoom, zoom]);

  useEffect(() => {
    zoom.value = neutralZoom;
  }, [neutralZoom, zoom]);

  const frameProcessor = useFrameProcessor((frame) => {
    'worklet';
    console.log("okkkkk");
    const scannedOcr = scanOCR(frame);
    console.log(scannedOcr.result.text);
  }, []);

  return (
    <View style={styles.container}>
        {device && (
          <PinchGestureHandler onGestureEvent={onPinchGesture} enabled={isActive}>
            <Reanimated.View style={StyleSheet.absoluteFill}>
              <TapGestureHandler onEnded={onDoubleTap} numberOfTaps={2}>
                <ReanimatedCamera
                  photo
                  ref={camera}
                  style={StyleSheet.absoluteFill}
                  device={device}
                  format={format}
                  isActive={isActive}
                  onInitialized={onInitialized}
                  enableZoomGesture={false}
                  animatedProps={cameraAnimatedProps}
                  orientation="portrait"
                />
              </TapGestureHandler>
            </Reanimated.View>
          </PinchGestureHandler>
        )}

            <CaptureButton
              style={styles.captureButton}
              camera={camera}
              onMediaCaptured={onMediaCaptured}
              cameraZoom={zoom}
              minZoom={minZoom}
              maxZoom={maxZoom}
              flash={supportsFlash ? flash : 'off'}
              enabled={isCameraInitialized && isActive}
              setIsPressingButton={setIsPressingButton}
            />

      <View style={styles.rightButtonRow}>
          {canToggleNightMode && (
            <TouchableOpacity style={styles.button} onPress={() => setEnableNightMode(!enableNightMode)}>
              <IonIcon name={enableNightMode ? 'moon' : 'moon-outline'} color="white" size={26} />
            </TouchableOpacity>
          )}
          {supportsFlash && (
            <TouchableOpacity style={styles.button} onPress={onFlashPressed}>
              <IonIcon name={flash === 'on' ? 'flash' : 'flash-off'} color="white" size={26} />
            </TouchableOpacity>
          )}
          {supportsCameraFlipping && (
            <TouchableOpacity style={styles.button} onPress={onFlipCameraPressed}>
              <IonIcon name="camera-reverse" color="white" size={26} />
            </TouchableOpacity>
          )}
    </View>
    </View>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    backgroundColor: 'black',
  },
  captureButton: {
    position: 'absolute',
    alignSelf: 'center',
    bottom: SAFE_AREA_PADDING.paddingBottom,
    backgroundColor: 'rgba(140, 20, 140, 0.3)',

  },
  button: {
    marginBottom: CONTENT_SPACING,
    width: BUTTON_SIZE,
    height: BUTTON_SIZE,
    borderRadius: BUTTON_SIZE / 2,
    backgroundColor: 'rgba(140, 140, 140, 0.3)',
    justifyContent: 'center',
    alignItems: 'center',
  },
  rightButtonRow: {
    position: 'absolute',
    right: SAFE_AREA_PADDING.paddingRight,
    top: SAFE_AREA_PADDING.paddingTop,
  },
  text: {
    color: 'white',
    fontSize: 11,
    fontWeight: 'bold',
    textAlign: 'center',
  },
});

export default CameraPage;

What happened instead?

Black screen

Relevant log output

No output :/ Seems like everything works but I still have a blackscreen !

Device

Android

VisionCamera Version

2.14.0

Additional information

• I am using Expo
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[Easy47]

I figured that if i delete this line "format={format}" i don't have the black screen anymore ... don't know why

[mrousavy]

Hey, thanks for your issue report!

Problem

Back in 2021 I developed VisionCamera around iOS' native device / format API. This Swift API is really flexible and allows you to get a list of all devices, and each device has a list of all supported formats. That's exactly how VisionCamera exposes it to JavaScript as well.

On Android however, I use CameraX. CameraX is the new recommended native camera library for Android. It does a lot of abstractions already, and while it's possible to get a list of all camera devices (front, back, wide angle, ..), it doesn't tell you which format is supported on which device. There's no such thing as "formats" in CameraX, you kinda just tell CameraX what resolution you want to record in, and it tries to match that as good as it can. All of the filtering is done internally then.

This sucks, because that's the same abstraction that I built using VisionCamera, and now it's not really possible to expose a list of formats through JavaScript, since I kinda have to make them up and just guess that it works, this is done here:

react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraViewModule.kt

Lines 168 to 342 in 4781ad9

	// TODO: This uses the Camera2 API to list all characteristics of a camera device and therefore doesn't work with Camera1. Find a way to use CameraX for this
	// https://issuetracker.google.com/issues/179925896
	@ReactMethod
	fun getAvailableCameraDevices(promise: Promise) {
	val startTime = System.currentTimeMillis()
	coroutineScope.launch {
	withPromise(promise) {
	val cameraProvider = ProcessCameraProvider.getInstance(reactApplicationContext).await()
	val extensionsManager = ExtensionsManager.getInstanceAsync(reactApplicationContext, cameraProvider).await()
	ProcessCameraProvider.getInstance(reactApplicationContext).await()
	val manager = reactApplicationContext.getSystemService(Context.CAMERA_SERVICE) as? CameraManager
	?: throw CameraManagerUnavailableError()
	val cameraDevices: WritableArray = Arguments.createArray()
	manager.cameraIdList.forEach loop@{ id ->
	val cameraSelector = CameraSelector.Builder().byID(id).build()
	val characteristics = manager.getCameraCharacteristics(id)
	val hardwareLevel = characteristics.get(CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL)!!
	val capabilities = characteristics.get(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES)!!
	val isMultiCam = Build.VERSION.SDK_INT >= Build.VERSION_CODES.P &&
	capabilities.contains(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA)
	val deviceTypes = characteristics.getDeviceTypes()
	val cameraConfig = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)!!
	val lensFacing = characteristics.get(CameraCharacteristics.LENS_FACING)!!
	val hasFlash = characteristics.get(CameraCharacteristics.FLASH_INFO_AVAILABLE)!!
	val maxScalerZoom = characteristics.get(CameraCharacteristics.SCALER_AVAILABLE_MAX_DIGITAL_ZOOM)!!
	val supportsDepthCapture = Build.VERSION.SDK_INT >= Build.VERSION_CODES.M &&
	capabilities.contains(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_DEPTH_OUTPUT)
	val supportsRawCapture = capabilities.contains(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_RAW)
	val isoRange = characteristics.get(CameraCharacteristics.SENSOR_INFO_SENSITIVITY_RANGE)
	val digitalStabilizationModes = characteristics.get(CameraCharacteristics.CONTROL_AVAILABLE_VIDEO_STABILIZATION_MODES)
	val opticalStabilizationModes = characteristics.get(CameraCharacteristics.LENS_INFO_AVAILABLE_OPTICAL_STABILIZATION)
	val zoomRange = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R)
	characteristics.get(CameraCharacteristics.CONTROL_ZOOM_RATIO_RANGE)
	else null
	val name = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.P)
	characteristics.get(CameraCharacteristics.INFO_VERSION)
	else null
	val fpsRanges = characteristics.get(CameraCharacteristics.CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES)!!
	val supportsHdr = extensionsManager.isExtensionAvailable(cameraSelector, ExtensionMode.HDR)
	val supportsLowLightBoost = extensionsManager.isExtensionAvailable(cameraSelector, ExtensionMode.NIGHT)
	// see https://developer.android.com/reference/android/hardware/camera2/CameraDevice#regular-capture
	val supportsParallelVideoProcessing = hardwareLevel != CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY && hardwareLevel != CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED
	val fieldOfView = characteristics.getFieldOfView()
	val map = Arguments.createMap()
	map.putString("id", id)
	map.putArray("devices", deviceTypes)
	map.putString("position", parseLensFacing(lensFacing))
	map.putString("name", name ?: "${parseLensFacing(lensFacing)} ($id)")
	map.putBoolean("hasFlash", hasFlash)
	map.putBoolean("hasTorch", hasFlash)
	map.putBoolean("isMultiCam", isMultiCam)
	map.putBoolean("supportsParallelVideoProcessing", supportsParallelVideoProcessing)
	map.putBoolean("supportsRawCapture", supportsRawCapture)
	map.putBoolean("supportsDepthCapture", supportsDepthCapture)
	map.putBoolean("supportsLowLightBoost", supportsLowLightBoost)
	map.putBoolean("supportsFocus", true) // I believe every device here supports focussing
	if (zoomRange != null) {
	map.putDouble("minZoom", zoomRange.lower.toDouble())
	map.putDouble("maxZoom", zoomRange.upper.toDouble())
	} else {
	map.putDouble("minZoom", 1.0)
	map.putDouble("maxZoom", maxScalerZoom.toDouble())
	}
	map.putDouble("neutralZoom", 1.0)
	val supportedVideoResolutions: List<Size>
	val cameraInfos = cameraSelector.filter(cameraProvider.availableCameraInfos)
	if (cameraInfos.size > 0) {
	supportedVideoResolutions = QualitySelector
	.getSupportedQualities(cameraInfos[0])
	.map { QualitySelector.getResolution(cameraInfos[0], it)!! }
	} else {
	supportedVideoResolutions = emptyList()
	}
	// TODO: Optimize?
	val maxImageOutputSize = cameraConfig.outputFormats
	.flatMap { cameraConfig.getOutputSizes(it).toList() }
	.maxByOrNull { it.width * it.height }!!
	val formats = Arguments.createArray()
	cameraConfig.outputFormats.forEach { formatId ->
	val formatName = parseImageFormat(formatId)
	cameraConfig.getOutputSizes(formatId).forEach { size ->
	val isHighestPhotoQualitySupported = areUltimatelyEqual(size, maxImageOutputSize)
	// Get the number of seconds that each frame will take to process
	val secondsPerFrame = try {
	cameraConfig.getOutputMinFrameDuration(formatId, size) / 1_000_000_000.0
	} catch (error: Throwable) {
	Log.e(TAG, "Minimum Frame Duration for MediaRecorder Output cannot be calculated, format \"$formatName\" is not supported.")
	null
	}
	val frameRateRanges = Arguments.createArray()
	if (secondsPerFrame != null && secondsPerFrame > 0) {
	val fps = (1.0 / secondsPerFrame).toInt()
	val frameRateRange = Arguments.createMap()
	frameRateRange.putInt("minFrameRate", 1)
	frameRateRange.putInt("maxFrameRate", fps)
	frameRateRanges.pushMap(frameRateRange)
	}
	fpsRanges.forEach { range ->
	val frameRateRange = Arguments.createMap()
	frameRateRange.putInt("minFrameRate", range.lower)
	frameRateRange.putInt("maxFrameRate", range.upper)
	frameRateRanges.pushMap(frameRateRange)
	}
	val colorSpaces = Arguments.createArray()
	colorSpaces.pushString(formatName)
	val videoStabilizationModes = Arguments.createArray()
	videoStabilizationModes.pushString("off")
	if (digitalStabilizationModes != null) {
	if (digitalStabilizationModes.contains(CameraCharacteristics.CONTROL_VIDEO_STABILIZATION_MODE_ON)) {
	videoStabilizationModes.pushString("auto")
	videoStabilizationModes.pushString("standard")
	}
	}
	if (opticalStabilizationModes != null) {
	if (opticalStabilizationModes.contains(CameraCharacteristics.LENS_OPTICAL_STABILIZATION_MODE_ON)) {
	videoStabilizationModes.pushString("cinematic")
	}
	}
	// TODO: Get the pixel format programatically rather than assuming a default of 420v
	val pixelFormat = "420v"
	val format = Arguments.createMap()
	format.putDouble("photoHeight", size.height.toDouble())
	format.putDouble("photoWidth", size.width.toDouble())
	// since supportedVideoResolutions is sorted from highest resolution to lowest,
	// videoResolution will be the highest supported video resolution lower than or equal to photo resolution
	// TODO: Somehow integrate with CamcorderProfileProxy?
	val videoResolution = supportedVideoResolutions.find { it.width <= size.width && it.height <= size.height }
	format.putDouble("videoHeight", videoResolution?.height?.toDouble())
	format.putDouble("videoWidth", videoResolution?.width?.toDouble())
	format.putBoolean("isHighestPhotoQualitySupported", isHighestPhotoQualitySupported)
	format.putInt("maxISO", isoRange?.upper)
	format.putInt("minISO", isoRange?.lower)
	format.putDouble("fieldOfView", fieldOfView) // TODO: Revisit getAvailableCameraDevices (is fieldOfView accurate?)
	format.putDouble("maxZoom", (zoomRange?.upper ?: maxScalerZoom).toDouble())
	format.putArray("colorSpaces", colorSpaces)
	format.putBoolean("supportsVideoHDR", false) // TODO: supportsVideoHDR
	format.putBoolean("supportsPhotoHDR", supportsHdr)
	format.putArray("frameRateRanges", frameRateRanges)
	format.putString("autoFocusSystem", "none") // TODO: Revisit getAvailableCameraDevices (autoFocusSystem) (CameraCharacteristics.CONTROL_AF_AVAILABLE_MODES or CameraCharacteristics.LENS_INFO_FOCUS_DISTANCE_CALIBRATION)
	format.putArray("videoStabilizationModes", videoStabilizationModes)
	format.putString("pixelFormat", pixelFormat)
	formats.pushMap(format)
	}
	}
	map.putArray("formats", formats)
	cameraDevices.pushMap(map)
	}
	val difference = System.currentTimeMillis() - startTime
	Log.w(TAG, "CameraViewModule::getAvailableCameraDevices took: $difference ms")
	return@withPromise cameraDevices
	}
	}
	}

Solution

I've been thinking about this for a long time now. I want to fix this issue once and for all with VisionCamera v3.

V3 has these features planned:

1. Will be a Fabric/TurboModule only. Legacy architecture will not be supported for the sake of simplicity (will fix a ton of build errors)
2. Write-back frame processors (draw on frame using RN Skia, draw face masks, filters, overlays, color shadings, shaders, Metal, etc..)
3. Completely redesigned API for device/format selection (resolution, fps, low-light, ...)

I will brainstorm with @thomas-coldwell in our Vienna Margelo office to think about how we can achieve a perfect API redesign that works on both iOS and Android without compromising too much of the flexibility that we have today.
This means, the new API should still allow things like:

• Control exactly how much FPS you want to record in
• Know exactly if a desired format is supported and be able to fall back to a different one
• Control the exact resolution and know what is supported (e.g. higher than 1080, but no higher than 4k, ...)
• Control settings like low-light mode, compression, recording format H.264 or H.265, etc.

For this we need a lot of time though.

If you (or your company) is interested in financially supporting those efforts, please let me know. Without sponsorship we can't work on this.

[mrousavy]

Btw., V3 discussion is here: #1376

[Brittognanaraj]

Try this android:hardwareAccelerated="�false" in AndroidManifest.xml makes the camera no longer display. Check your AndroidManifest and set it back to true may help the camera work again.

[mrousavy]

Hey! I've rewritten the entire Android codebase of VisionCamera from CameraX to Camera2 in the efforts of ✨ VisionCamera V3.

I just now completed the Camera2 rewrite and I believe the core structure is running, but there might be some edge cases to iron out. Can you try and test the PR #1674 for me to see if you can still reproduce this issue here?

Here's an instruction on how you can test that: #1674 (comment)

If the issue cannot be reproduced with that version/PR anymore, then hoorayy, I fixed it! 🎉
Otherwise please let me know and I'll keep this issue open to keep track of it.

Thank you!