Skip to content

Commit

Permalink
chore: Cleanup TS codebase and fix Skia Preview View style/layout iss…
Browse files Browse the repository at this point in the history
…ue (#2767)

* chore: Cleanup TS codebase

* chore: More refactors

* fix: Put Skia Canvas into CameraView

* Update Camera.tsx

* fix: Fix JNI types
  • Loading branch information
mrousavy authored Apr 21, 2024
1 parent d575008 commit 64ef743
Show file tree
Hide file tree
Showing 38 changed files with 70 additions and 71 deletions.
8 changes: 4 additions & 4 deletions docs/docs/guides/FRAME_PROCESSORS_CREATE_OVERVIEW.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -203,10 +203,10 @@ Your Frame Processor Plugins have to be fast. Use the FPS Graph (`enableFpsGraph
#### 🚀 Create your first Frame Processor Plugin for [iOS](frame-processors-plugins-ios) or [Android](frame-processors-plugins-android)!

[1]: https://github.com/mrousavy/react-native-vision-camera/blob/main/package/src/Frame.ts
[2]: https://github.com/mrousavy/react-native-vision-camera/blob/main/package/ios/Frame%20Processor/Frame.h
[3]: https://github.com/mrousavy/react-native-vision-camera/blob/main/package/android/src/main/java/com/mrousavy/camera/frameprocessor/Frame.java
[2]: https://github.com/mrousavy/react-native-vision-camera/blob/main/package/ios/FrameProcessors/Frame.h
[3]: https://github.com/mrousavy/react-native-vision-camera/blob/main/package/android/src/main/java/com/mrousavy/camera/frameprocessors/Frame.java
[4]: https://github.com/facebook/react-native/blob/9a43eac7a32a6ba3164a048960101022a92fcd5a/React/Base/RCTBridgeModule.h#L20-L24
[5]: https://developer.apple.com/documentation/coremedia/cmsamplebuffer
[6]: https://developer.android.com/reference/androidx/camera/core/ImageProxy
[7]: https://github.com/mrousavy/react-native-vision-camera/blob/main/package/ios/Frame%20Processor/SharedArray.h
[8]: https://github.com/mrousavy/react-native-vision-camera/blob/main/package/android/src/main/java/com/mrousavy/camera/frameprocessor/SharedArray.java
[7]: https://github.com/mrousavy/react-native-vision-camera/blob/main/package/ios/FrameProcessors/SharedArray.h
[8]: https://github.com/mrousavy/react-native-vision-camera/blob/main/package/android/src/main/java/com/mrousavy/camera/frameprocessors/SharedArray.java
2 changes: 1 addition & 1 deletion docs/docs/guides/FRAME_PROCESSORS_CREATE_PLUGIN_CPP.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ For example:

```cpp
// 1. Include headers
#include "frameprocessor/FrameHostObject.h"
#include "frameprocessors/FrameHostObject.h"

// 2. Create C++ func
auto myPlugin = [=](jsi::Runtime& runtime,
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/guides/FRAME_PROCESSORS_TIPS.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ In my benchmarks, Frame Processors can run more than 1000 times a second.

Frame Processor Plugins are native functions (written in Objective-C, Swift, C++, Java or Kotlin) that are injected into the VisionCamera JS-Runtime. They can be synchronously called from your JS Frame Processors (using JSI) without ever going over the bridge. Because VisionCamera provides an easy-to-use plugin API, you can easily create a Frame Processor Plugin yourself. Some examples include [Barcode Scanning](https://developers.google.com/ml-kit/vision/barcode-scanning), [Face Detection](https://developers.google.com/ml-kit/vision/face-detection), [Image Labeling](https://developers.google.com/ml-kit/vision/image-labeling), [Text Recognition](https://developers.google.com/ml-kit/vision/text-recognition) and more.

> Learn how to [create Frame Processor Plugins](frame-processors-plugins-overview), or check out the [example Frame Processor Plugin for iOS](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/ios/Frame%20Processor%20Plugins/Example%20Swift%20Plugin/ExampleSwiftFrameProcessor.swift) or [Android](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/android/app/src/main/java/com/mrousavy/camera/example/ExampleKotlinFrameProcessorPlugin.kt).
> Learn how to [create Frame Processor Plugins](frame-processors-plugins-overview), or check out the [example Frame Processor Plugin for iOS](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/ios/FrameProcessors%20Plugins/Example%20Swift%20Plugin/ExampleSwiftFrameProcessor.swift) or [Android](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/android/app/src/main/java/com/mrousavy/camera/example/ExampleKotlinFrameProcessorPlugin.kt).
### The `Frame` object

Expand Down
4 changes: 2 additions & 2 deletions docs/docs/guides/FRAME_PROCESSOR_CREATE_PLUGIN_IOS.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ VISION_EXPORT_FRAME_PROCESSOR(FaceDetectorFrameProcessorPlugin, detectFaces)
The Frame Processor Plugin will be exposed to JS through the `VisionCameraProxy` object. In this case, it would be `VisionCameraProxy.initFrameProcessorPlugin("detectFaces")`.
:::
4. **Implement your Frame Processing.** See the [Example Plugin (Objective-C)](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/ios/Frame%20Processor%20Plugins/Example%20Plugin/ExampleFrameProcessorPlugin.m) for reference.
4. **Implement your Frame Processing.** See the [Example Plugin (Objective-C)](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/ios/FrameProcessors%20Plugins/Example%20Plugin/ExampleFrameProcessorPlugin.m) for reference.
</TabItem>
<TabItem value="swift">
Expand Down Expand Up @@ -118,7 +118,7 @@ VISION_EXPORT_SWIFT_FRAME_PROCESSOR(FaceDetectorFrameProcessorPlugin, detectFace
// highlight-end
```
5. **Implement your frame processing.** See [Example Plugin (Swift)](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/ios/Frame%20Processor%20Plugins/Example%20Swift%20Plugin/ExampleSwiftFrameProcessor.swift) for reference.
5. **Implement your frame processing.** See [Example Plugin (Swift)](https://github.com/mrousavy/react-native-vision-camera/blob/main/package/example/ios/FrameProcessors%20Plugins/Example%20Swift%20Plugin/ExampleSwiftFrameProcessor.swift) for reference.
</TabItem>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ class VisionCameraProxy : public jsi::HostObject {

class VisionCameraInstaller : public jni::JavaClass<VisionCameraInstaller> {
public:
static auto constexpr kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessor/VisionCameraInstaller;";
static auto constexpr kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessors/VisionCameraInstaller;";
static void registerNatives();
static void install(jni::alias_ref<jni::JClass> clazz, jni::alias_ref<JVisionCameraProxy::javaobject> proxy);
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ using namespace facebook;
using namespace jni;

struct JFrame : public JavaClass<JFrame> {
static constexpr auto kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessor/Frame;";
static constexpr auto kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessors/Frame;";

public:
int getWidth() const;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ using namespace facebook;

struct JFrameProcessor : public jni::HybridClass<JFrameProcessor> {
public:
static auto constexpr kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessor/FrameProcessor;";
static auto constexpr kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessors/FrameProcessor;";
static void registerNatives();
static jni::local_ref<JFrameProcessor::javaobject> create(const std::shared_ptr<RNWorklet::JsiWorklet>& worklet,
const std::shared_ptr<RNWorklet::JsiWorkletContext>& context);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ using namespace facebook;
using namespace jni;

struct JFrameProcessorPlugin : public JavaClass<JFrameProcessorPlugin> {
static constexpr auto kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessor/FrameProcessorPlugin;";
static constexpr auto kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessors/FrameProcessorPlugin;";

public:
/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ using namespace facebook;
using namespace jni;

struct JJSUnionValue : public JavaClass<JJSUnionValue> {
static constexpr auto kJavaDescriptor = "Lcom/mrousavy/camera/types/JSUnionValue;";
static constexpr auto kJavaDescriptor = "Lcom/mrousavy/camera/core/types/JSUnionValue;";

local_ref<JString> getUnionValue() {
const auto getUnionValueMethod = getClass()->getMethod<JString()>("getUnionValue");
return getUnionValueMethod(self());
}
};

} // namespace vision
} // namespace vision
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ using namespace facebook;
using namespace jni;

struct JOrientation : public JavaClass<JOrientation, JJSUnionValue> {
static constexpr auto kJavaDescriptor = "Lcom/mrousavy/camera/types/Orientation;";
static constexpr auto kJavaDescriptor = "Lcom/mrousavy/camera/core/types/Orientation;";
};

} // namespace vision
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ using namespace facebook;
using namespace jni;

struct JPixelFormat : public JavaClass<JPixelFormat, JJSUnionValue> {
static constexpr auto kJavaDescriptor = "Lcom/mrousavy/camera/types/PixelFormat;";
static constexpr auto kJavaDescriptor = "Lcom/mrousavy/camera/core/types/PixelFormat;";
};

} // namespace vision
} // namespace vision
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ using namespace facebook;

class JSharedArray : public jni::HybridClass<JSharedArray> {
public:
static auto constexpr kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessor/SharedArray;";
static auto constexpr kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessors/SharedArray;";
static void registerNatives();

public:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ class JVisionCameraProxy : public jni::HybridClass<JVisionCameraProxy> {
#endif

static auto constexpr TAG = "VisionCameraProxy";
static auto constexpr kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessor/VisionCameraProxy;";
static auto constexpr kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessors/VisionCameraProxy;";

explicit JVisionCameraProxy(const jni::alias_ref<JVisionCameraProxy::jhybridobject>& javaThis, jsi::Runtime* jsRuntime,
const std::shared_ptr<facebook::react::CallInvoker>& jsCallInvoker,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ using namespace facebook;
*/
class JVisionCameraScheduler : public jni::HybridClass<JVisionCameraScheduler> {
public:
static auto constexpr kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessor/VisionCameraScheduler;";
static auto constexpr kJavaDescriptor = "Lcom/mrousavy/camera/frameprocessors/VisionCameraScheduler;";
static jni::local_ref<jhybriddata> initHybrid(jni::alias_ref<jhybridobject> jThis);
static void registerNatives();

Expand Down
46 changes: 22 additions & 24 deletions package/src/Camera.tsx
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
import React from 'react'
import { requireNativeComponent, findNodeHandle, View, StyleSheet } from 'react-native'
import type { CameraDevice } from './CameraDevice'
import { requireNativeComponent, findNodeHandle, StyleSheet } from 'react-native'
import type { CameraDevice } from './types/CameraDevice'
import type { ErrorWithCause, CameraCaptureError } from './CameraError'
import { CameraRuntimeError, tryParseNativeCameraError, isErrorWithCause } from './CameraError'
import type { CameraProps, OnShutterEvent } from './CameraProps'
import type { CameraProps, OnShutterEvent } from './types/CameraProps'
import { CameraModule } from './NativeCameraModule'
import type { PhotoFile, TakePhotoOptions } from './PhotoFile'
import type { Point } from './Point'
import type { RecordVideoOptions, VideoFile } from './VideoFile'
import type { PhotoFile, TakePhotoOptions } from './types/PhotoFile'
import type { Point } from './types/Point'
import type { RecordVideoOptions, VideoFile } from './types/VideoFile'
import { VisionCameraProxy } from './FrameProcessorPlugins'
import { CameraDevices } from './CameraDevices'
import type { EmitterSubscription, NativeSyntheticEvent, NativeMethods } from 'react-native'
import type { Code, CodeScanner, CodeScannerFrame } from './CodeScanner'
import type { TakeSnapshotOptions } from './Snapshot'
import type { Code, CodeScanner, CodeScannerFrame } from './types/CodeScanner'
import type { TakeSnapshotOptions } from './types/Snapshot'
import { SkiaCameraCanvas } from './skia/SkiaCameraCanvas'
import type { Frame } from './Frame'
import type { Frame } from './types/Frame'

//#region Types
export type CameraPermissionStatus = 'granted' | 'not-determined' | 'denied' | 'restricted'
Expand Down Expand Up @@ -586,7 +586,7 @@ export class Camera extends React.PureComponent<CameraProps, CameraState> {
const torch = this.state.isRecordingWithFlash ? 'on' : props.torch
const isRenderingWithSkia = frameProcessor?.type === 'drawable-skia'

const result = (
return (
<NativeCameraView
{...props}
cameraId={device.id}
Expand All @@ -603,24 +603,16 @@ export class Camera extends React.PureComponent<CameraProps, CameraState> {
enableFrameProcessor={frameProcessor != null}
enableBufferCompression={props.enableBufferCompression ?? shouldEnableBufferCompression}
enableFpsGraph={frameProcessor != null && props.enableFpsGraph}
preview={isRenderingWithSkia ? false : props.preview ?? true}
/>
)

if (frameProcessor?.type === 'drawable-skia') {
return (
<View style={props.style} pointerEvents="box-none">
preview={isRenderingWithSkia ? false : props.preview ?? true}>
{frameProcessor?.type === 'drawable-skia' && (
<SkiaCameraCanvas
style={StyleSheet.absoluteFill}
style={styles.customPreviewView}
offscreenTextures={frameProcessor.offscreenTextures}
resizeMode={props.resizeMode}
/>
{result}
</View>
)
} else {
return result
}
)}
</NativeCameraView>
)
}
}
//#endregion
Expand All @@ -631,3 +623,9 @@ const NativeCameraView = requireNativeComponent<NativeCameraViewProps>(
// @ts-expect-error because the type declarations are kinda wrong, no?
Camera,
)

const styles = StyleSheet.create({
customPreviewView: {
flex: 1,
},
})
2 changes: 1 addition & 1 deletion package/src/CameraDevices.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { NativeModules, NativeEventEmitter } from 'react-native'
import type { CameraDevice } from './CameraDevice'
import type { CameraDevice } from './types/CameraDevice'

const CameraDevicesManager = NativeModules.CameraDevices as {
getConstants: () => {
Expand Down
2 changes: 1 addition & 1 deletion package/src/FrameProcessorPlugins.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { Frame, FrameInternal } from './Frame'
import type { Frame, FrameInternal } from './types/Frame'
import { CameraRuntimeError } from './CameraError'
import { CameraModule } from './NativeCameraModule'
import { assertJSIAvailable } from './JSIHelper'
Expand Down
2 changes: 1 addition & 1 deletion package/src/devices/getCameraDevice.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { CameraDevice, CameraPosition, PhysicalCameraDeviceType } from '../CameraDevice'
import type { CameraDevice, CameraPosition, PhysicalCameraDeviceType } from '../types/CameraDevice'

export interface DeviceFilter {
/**
Expand Down
2 changes: 1 addition & 1 deletion package/src/devices/getCameraFormat.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { AutoFocusSystem, CameraDevice, CameraDeviceFormat, VideoStabilizationMode } from '../CameraDevice'
import type { AutoFocusSystem, CameraDevice, CameraDeviceFormat, VideoStabilizationMode } from '../types/CameraDevice'
import { CameraRuntimeError } from '../CameraError'

interface Size {
Expand Down
2 changes: 1 addition & 1 deletion package/src/hooks/useCameraDevice.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { useMemo } from 'react'
import type { CameraDevice, CameraPosition } from '../CameraDevice'
import type { CameraDevice, CameraPosition } from '../types/CameraDevice'
import type { DeviceFilter } from '../devices/getCameraDevice'
import { getCameraDevice } from '../devices/getCameraDevice'
import { useCameraDevices } from './useCameraDevices'
Expand Down
2 changes: 1 addition & 1 deletion package/src/hooks/useCameraDevices.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { useEffect, useState } from 'react'
import type { CameraDevice } from '../CameraDevice'
import type { CameraDevice } from '../types/CameraDevice'
import { CameraDevices } from '../CameraDevices'

/**
Expand Down
2 changes: 1 addition & 1 deletion package/src/hooks/useCameraFormat.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { useMemo } from 'react'
import type { CameraDevice, CameraDeviceFormat } from '../CameraDevice'
import type { CameraDevice, CameraDeviceFormat } from '../types/CameraDevice'
import type { FormatFilter } from '../devices/getCameraFormat'
import { getCameraFormat } from '../devices/getCameraFormat'

Expand Down
2 changes: 1 addition & 1 deletion package/src/hooks/useCodeScanner.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { useCallback, useMemo, useRef } from 'react'
import type { Code, CodeScanner, CodeScannerFrame } from '../CodeScanner'
import type { Code, CodeScanner, CodeScannerFrame } from '../types/CodeScanner'

export function useCodeScanner(codeScanner: CodeScanner): CodeScanner {
const { onCodeScanned, ...codeScannerOptions } = codeScanner
Expand Down
4 changes: 2 additions & 2 deletions package/src/hooks/useFrameProcessor.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import type { DependencyList } from 'react'
import { useMemo } from 'react'
import { wrapFrameProcessorWithRefCounting } from '../FrameProcessorPlugins'
import type { ReadonlyFrameProcessor } from '../CameraProps'
import type { Frame } from '../Frame'
import type { ReadonlyFrameProcessor } from '../types/CameraProps'
import type { Frame } from '../types/Frame'

/**
* Create a new Frame Processor function which you can pass to the `<Camera>`.
Expand Down
23 changes: 12 additions & 11 deletions package/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
export * from './Camera'
export * from './CameraDevice'
export * from './CameraError'
export * from './CameraProps'
export * from './Frame'
export * from './FrameProcessorPlugins'
export * from './Orientation'
export * from './PhotoFile'
export * from './Snapshot'
export * from './PixelFormat'
export * from './Point'
export * from './VideoFile'
export * from './CodeScanner'

export * from './types/CameraDevice'
export * from './types/CameraProps'
export * from './types/Frame'
export * from './types/Orientation'
export * from './types/PhotoFile'
export * from './types/Snapshot'
export * from './types/PixelFormat'
export * from './types/Point'
export * from './types/VideoFile'
export * from './types/CodeScanner'

export * from './devices/getCameraFormat'
export * from './devices/getCameraDevice'
Expand All @@ -20,7 +21,7 @@ export * from './hooks/useCameraDevice'
export * from './hooks/useCameraDevices'
export * from './hooks/useCameraFormat'
export * from './hooks/useCameraPermission'
export * from './hooks/useFrameProcessor'
export * from './hooks/useCodeScanner'
export * from './hooks/useFrameProcessor'

export * from './skia/useSkiaFrameProcessor'
2 changes: 1 addition & 1 deletion package/src/skia/SkiaCameraCanvas.tsx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import React, { useCallback, useState } from 'react'
import type { LayoutChangeEvent, ViewProps } from 'react-native'
import type { CameraProps } from '../CameraProps'
import type { CameraProps } from '../types/CameraProps'
import type { ISharedValue } from 'react-native-worklets-core'
import { ReanimatedProxy } from '../dependencies/ReanimatedProxy'
import type { SkImage } from '@shopify/react-native-skia'
Expand Down
6 changes: 3 additions & 3 deletions package/src/skia/useSkiaFrameProcessor.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import type { Frame, FrameInternal } from '../Frame'
import type { Frame, FrameInternal } from '../types/Frame'
import type { DependencyList } from 'react'
import { useEffect, useMemo } from 'react'
import type { Orientation } from '../Orientation'
import type { Orientation } from '../types/Orientation'
import { VisionCameraProxy, wrapFrameProcessorWithRefCounting } from '../FrameProcessorPlugins'
import type { DrawableFrameProcessor } from '../CameraProps'
import type { DrawableFrameProcessor } from '../types/CameraProps'
import type { ISharedValue, IWorkletNativeApi } from 'react-native-worklets-core'
import { WorkletsProxy } from '../dependencies/WorkletsProxy'
import type { SkCanvas, SkPaint, SkImage, SkSurface } from '@shopify/react-native-skia'
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import type { ViewProps } from 'react-native'
import type { CameraDevice, CameraDeviceFormat, VideoStabilizationMode } from './CameraDevice'
import type { CameraRuntimeError } from './CameraError'
import type { CameraRuntimeError } from '../CameraError'
import type { CodeScanner } from './CodeScanner'
import type { Frame } from './Frame'
import type { Orientation } from './Orientation'
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { CameraCaptureError } from './CameraError'
import type { CameraCaptureError } from '../CameraError'
import type { TemporaryFile } from './TemporaryFile'

export interface RecordVideoOptions {
Expand Down

0 comments on commit 64ef743

Please sign in to comment.