Live object detection example, confidence is very low

[JobiJoba]

Hi,

I'm trying the example given in this repo live_object_detection_ssd_mobilenet but the confidence of what the camera see is very low (less than 0.5).

Is there a way to improve that? What are the variable that make the confidence that low?

I'm using a Oppo Reno 7Z
(I've downloaded the model using the script)
No errors in the debug console

Thanks

[gregorscholz]

Can you please provide some screenshots? Maybe look at the label.txt. The model does not recognise that many things.

[JobiJoba]

Hi, I'm using the default label.txt downloaded from the script (so there is like 96 entries).

I've the same setup as in the example gif so like a bottle of water / coffee cup / laptop :) the only thing he recognize with a high score of confidence is my table when I put the phone camera face down on it.

b7fc3740-224f-4dfa-b1d5-9ec129530837.mp4

If I change the confidence under 0.5 (I didn't change anything else in the code), you'll start to see some square detection but not fantastic as the confidence is very low :)

The object detection ssd mobile net example works well, if I take a picture of my laptop he recognize it.

So I'm not sure if the model is the issue in fact. The phone? Is there a list of compatible phone?

[gregorscholz]

Maybe its the phone, do you have another phone you can test it with?

[nelzaatari]

im facing the same issue, on colab everything works fine, did anyone find any solution

[ghost]

Hi everyone,

I believe I have found a bug. Changing ResolutionPreset from "medium" to "low" may improve accuracy. (I have only tested this on my Android device.)

[JobiJoba]

Well this is shocking but using an older phone the example detect correctly ... a bug with Oppo Reno 7 ? -_-

[daniwaxman]

I have the same issue.

[JobiJoba]

I start using flutter_vision which under the hood use tflite and obviously , have the same issue.

I've created another issue there with more detail in the hope someone have an idea why it's doing that or guide me through idea :)

vladiH/flutter_vision#36

[vianziro]

I have a different problem, I created a model using yolov5 and converted it to tflite. when I try my model, the detection process doesn't work, but in colab everything works normally.

[JobiJoba]

Anyone would have an hint on what could be done to fix that issue?

I've tried another implementation (https://github.com/Pawandeep-prog/realtime_object_detection android only) on my Oppo Reno 7Z and that works perfectly, they use under the hood tflite also but it's a bit different from what is done in this project.

Unfortunately, it wasn't working on multiples phones so we had to find another solution with a webview but it would be nice to come back to a native solution (for performance reason).

[JobiJoba]

For those having the same issue, i did the following to fix it:

• Add this image utils file into the project: https://github.com/zezo357/pytorch_lite/blob/694cb50523e6803441b4ded074909543e130c75e/lib/image_utils.dart

• in detector_service.dart

instead of :

void _convertCameraImage(CameraImage cameraImage) {
   var preConversionTime = DateTime.now().millisecondsSinceEpoch;

   convertCameraImageToImage(cameraImage).then((image) {
     if (image != null) {
       if (Platform.isAndroid) {
         image = image_lib.copyRotate(image, angle: 90);
       }

       final results = analyseImage(image, preConversionTime);
       _sendPort.send(_Command(_Codes.result, args: [results]));
     }
   });
 }

I do

 void _convertCameraImage(CameraImage cameraImage) async {
    var preConversionTime = DateTime.now().millisecondsSinceEpoch;

    // var image = await ImageUtilsIsolate.convertCameraImage(cameraImage);
    var image = ImageUtils.convertCameraImage(cameraImage);
    if (image != null) {
      if (Platform.isAndroid) {
        image = image_lib.copyRotate(image, angle: 90);
      }

      // print("Image $image");
      final results = analyseImage(image, preConversionTime);
      _sendPort.send(_Command(_Codes.result, args: [results]));
    }
  }

I didn't test on many phone (2) but my phone which wasn't "detecting" anything can now detect without any issue at the speed of light :P

thanks a lot to the Pytorch Library which works wonderfully but unfortunately is not performant enough overall.

[yanghoonkim]

I tried to check the processed image using convertCameraImageToImage in image_utils.dart in live_object_detection_ssd_mobilenet example and found that it results in very strange image (both for ios and android).

Using the image_util.dart mentioned by @JobiJoba, it is solved.