How can I get the precision, recall, map@0.5 of the network prediction using this slicing method？

[Grenaf]

Hello fcakyon，in order to detect small targets, I used a lot of 640x640 slice images to train yolov5s（slice image labels are yolov5 format）, and then used sahi to take the high-resolution images synthesized for detection(high-resolution images are prediction set, there are no labels).

• I have a question how can I get precision, recall, map@0.5 on the prediction set or validation set(high-resolution images), my prediction set are some high-resolution pictures, and the training set are slices of some high-resolution pictures(slice image labels are yolov5 format).
• I see a way about this question https://github.com/obss/sahi/issues/182#issuecomment-894101529, I am not sure how to use it，because my dataset labels are YOLO format。Can I use sahi to detect multiple high-resolution images at once and get its evaluation, precision, recall, map@0.5, map@0.5: 0.95？

[fcakyon]

@Grenaf thanks for your valuable feedback!

As you mentioned, you can perform sliced inference, save results in coco result format and evaluate map50 map0.5: 0.95 etc following the steps mentioned here: https://github.com/obss/sahi/issues/182#issuecomment-894101529

[fcakyon]

You have to convert your dataset into coco format before performing sahi predict and sahi coco evaluate.

[Grenaf]

@Grenaf thanks for your valuable feedback!

As you mentioned, you can perform sliced inference, save results in coco result format and evaluate map50 map0.5: 0.95 etc following the steps mentioned here: #182 (comment)

So I need to use the COCO format label, these two parameters need to set the image path of the test set（images） and the label（COCO fomat） of the test set

[Grenaf]

You have to convert your dataset into coco format before performing sahi predict and sahi coco evaluate.

I have a question. Two prediction methods are used in sahi. When I use the sahi coco evaluate --dataset_json_path dataset.json --result_json_path result.json command, which method will the model choose to evaluate the model。get_prediction() or get_slice_prediction()

[fcakyon]

You have to convert your dataset into coco format before performing sahi predict and sahi coco evaluate.

I have a question. Two prediction methods are used in sahi. When I use the sahi coco evaluate --dataset_json_path dataset.json --result_json_path result.json command, which method will the model choose to evaluate the model。get_prediction() or get_slice_prediction()

evaluate command does not decide that. You have to set it in the predict command as
sahi predict --no_sliced_prediction while creating the result.json file.

By default, sahi predict performs both sliced and full inference. Check the predict docs for the details: https://github.com/obss/sahi/blob/main/docs/CLI.md#predict-command-usage

[fcakyon]

@Grenaf thanks for your valuable feedback!
As you mentioned, you can perform sliced inference, save results in coco result format and evaluate map50 map0.5: 0.95 etc following the steps mentioned here: #182 (comment)

So I need to use the COCO format label, these two parameters need to set the image path of the test set（images） and the label（COCO fomat） of the test set

You need 2 things:

• a coco formatted dataset json file
• and a coco formatted result json file

COCO formatted result json file can be created using sahi predict --source image/dir/ --dataset_json_file coco_dataset.json command.

[Grenaf]

So I need these files, annotation file and results.json?

[fcakyon]

So I need these files, annotation file and results.json?

You are right!

[fcakyon]

@Grenaf closing this issue if you dont have any more questions :)

[Grenaf]

Results do not correspond to current coco set

• sahi predict --model_confidence_threshold 0.6 --source F:\deeplearning_project\sahi_0.8.13\FWX\original\original_test_imageslabels --model_path F:\deeplearning_project\sahi_0.8.13\FWX\original\original_best.pt --model_type yolov5 --no_sliced_prediction --dataset_json_path F:\deeplearning_project\sahi_0.8.13\FWX\original\original_test_imageslabels.json --export_visual
  Then i get result.json
• sahi coco evaluate --dataset_json_path F:\deeplearning_project\sahi_0.8.13\FWX\original\original_test_imageslabels.json --result_json_path F:\deeplearning_project\sahi_0.8.13\FWX\original\result.json
  but an error occured
  

I used these two instructions, but it prompted an error. I'm not sure what the problem is.

[fcakyon]

@Grenaf this issue is fixed with: #299 but havent released a new version for sahi yet. You can either install sahi from main branch with: pip install git+https://github.com/obss/sahi.git@main or wait for a new release (will happen today 💯 ).

[Grenaf]

this issue is fixed with: #299 but havent released a new version for sahi yet. You can either install sahi from main branch with: pip install git+https://github.com/obss/sahi.git@main or wait for a new release (will will happen today).

Okay, don’t worry, look forward to your new news.

[fcakyon]

@Grenaf latest version (0.8.14) is live!

[Grenaf]

@Grenaf latest version (0.8.14) is live!

• Today I used sahi 0.8.14, but a new problem has arisen, and I am a little confused about the results of the evaluation.
  • I use sahi predict --model_confidence_threshold 0.6 --source FWX/original/original_test_imageslabels --model_path FWX/original/original_best.pt --model_type yolov5 --no_sliced_prediction --dataset_json_path FWX/original/original_test_imageslabels.json --export_visual
  • sahi coco evaluate --dataset_json_path FWX/original/original_test_imageslabels.json --result_json_path FWX/original/result.json --out_dir FWX/output/coco --type bbox --iou_thrs 0.5

the result just like this，Is this result normal? Through the visuals_with_gt obtained by the first command, I saw that the model successfully predicted the target. I think the values of map and ap may be abnormal。

[fcakyon]

@Grenaf it means that your category ids in your trained model and coco json dataset does not match. You have to fix the categories field of your coco dataset json to have proper evaluation.

In most cases, mapping category ids from 0 to N solves the issue. If you can provide your model i can tell you the proper category id to name mapping.

[Grenaf]

• I use the yolov5s model, there are a total of six categories of targets to be detected. The original yolov5s label category id is 0-5, in order to use coco evaluate I need to change the yolo label to coco format https://github.com/Taeyoung96/Yolo-to-COCO-format-converter, and then the category id becomes 1-6. So what documents should I provide? Thanks

[Grenaf]

@Grenaf it means that your category ids in your trained model and coco json dataset does not match. You have to fix the categories field of your coco dataset json to have proper evaluation.

In most cases, mapping category ids from 0 to N solves the issue. If you can provide your model i can tell you the proper category id to name mapping.

• I found the problem. When I convert the yolo label to the coco format, the target category id in original_test_imageslabels.json is 1-6
  

• When I execute sahi predict --model_confidence_threshold 0.6 --source FWX/original/original_test_imageslabels --model_path FWX/original/original_best.pt --model_type yolov5 --no_sliced_prediction --dataset_json
  _path FWX/original/original_test_imageslabels.json --export_visual

• result.json ，the category in result.json does not match the category in original_test_imageslabels.json. Taking missing categories id as an example, its id in original_test_imageslabels.json is 1, but the id in result is 0. So the problem is categories id Caused by a mismatch. So is there a way to solve this problem，do I need to modify the category information in the original_test_imageslabels.json file?Looking forward to your reply！
  

[fcakyon]

You model is trained with category ids 0-5 but your dataset json contains category ids 1-6. You have to map your category ids in the dataset json to 0-5 to properly perform evaluation. You can use sahi coco utilities to manage that: https://github.com/obss/sahi/blob/main/docs/COCO.md

[IrDIE]

I implemented validation with SAHI for YOLO8
https://github.com/IrDIE/YOLO8_SAHI/tree/master

💙 If you like it - press ⭐