Getting segmentation fault when running InferenceSlicer with OBB model with thread_workers > 1

[zxsk1974]

Search before asking

• I have searched the Supervision issues and found no similar bug report.

Bug

InferenceSlicer throws Segmentation fault with thread_workers = 4:

Segmentation fault (core dumped)

Environment

Supervision 0.24.0
Python 3.11

Minimal Reproducible Example

from ultralytics import YOLO
import cv2
import sys
import torch
import numpy as np
from pathlib import Path
import math
import supervision as sv

def callback(image_slice: np.ndarray) -> sv.Detections:
    result = model(image_slice, conf=0.6)
    print("Results:", result)
    return sv.Detections.from_ultralytics(result[0])

model = YOLO('yolo11x-obb.pt')

print("GPU:", torch.cuda.is_available())
if torch.cuda.is_available():
    device = torch.device("cuda")
    print("Using GPU:", torch.cuda.get_device_name())

# load image
file_name = Path(sys.argv[1])
image = cv2.imread(sys.argv[1]) #Image.open(file_name, mode='r')
print("Loaded image:", file_name)

image_wh = (image.shape[1], image.shape[0])
slice_wh = (1024, 1024)
overlap_ratio_wh = (0.2, 0.2)
overlap_ratio_w, overlap_ratio_h = overlap_ratio_wh
slice_w, slice_h = slice_wh
overlap_wh = (math.ceil(slice_w * overlap_ratio_w), math.ceil(slice_h * overlap_ratio_h))

slicer = sv.InferenceSlicer(
  callback=callback,
  overlap_filter=sv.OverlapFilter.NON_MAX_MERGE,
  iou_threshold=0.15,
  slice_wh=slice_wh,
  overlap_ratio_wh=None,
  overlap_wh=overlap_wh,
  thread_workers=2
)

detections = slicer(image)

labels = [
    f"{class_name} {confidence:.1f}"
    for class_name, confidence
    in zip(detections['class_name'], detections.confidence)
]

label_annotator = sv.LabelAnnotator(text_scale=0.2, text_thickness=1, text_padding=0, text_position=sv.Position.TOP_LEFT)
bbox_annotator = sv.BoxAnnotator(color=sv.ColorPalette.DEFAULT.colors[6], thickness=2)
obb_annotator = sv.OrientedBoxAnnotator(color=sv.ColorPalette.DEFAULT.colors[6], thickness=2)

print(f"Image shape: {image_wh[0]}w x {image_wh[1]}h")
print(f"Tile size: {slice_wh[0]}w x {slice_wh[1]}h")
print(f"Overlap: {overlap_wh[0]}w x {overlap_wh[1]}h. Ratio {overlap_ratio_wh}")
print(f"Overlap Filter: {sv.OverlapFilter.NON_MAX_MERGE}")
print(f"Found {len(detections)} objects")

annotated_image = obb_annotator.annotate(scene=image.copy(), detections=detections)
annotated_image = label_annotator.annotate(scene=annotated_image, detections=detections, labels=labels)

cv2.imwrite(file_name.stem + "-output.jpg", annotated_image)
![airplane-graveyard-zoomed](https://github.com/user-attachments/assets/0bf79b23-4775-43f3-8573-bfe0e96cbf91)

Additional

To reproduce:

python detect-image-slicer-bug.py airplane-graveyard-zoomed.jpg

.jpg

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[LinasKo]

Hi @zxsk1974 👋

Thank you for reporting it. Unfortunately, many models aren't made to be run from multiple threads. We plan to shift to running in batches (linked PR), but it is unfortunately relatively low on the priority list.

[zxsk1974]

@LinasKo batching will work too, can I try it on pre-release branch?

[LinasKo]

Not as a pre-release branch. I don't expect to merge this any time soon.

However, I brought the branch up-to-date with the latest supervision version.
Feel free to install via
pip install git+https://github.com/LinasKo/supervision.git@feature/batched-inference-slicer.

Alternatively, you may fork it from my repo. Install it in the same way, but from your own namespace.

[LinasKo]

Let us know if it works - that'd give more reason to revisit the PR 🙂

[zxsk1974]

Interesting, but sometimes the Slicer with 2 threads works till the end of detection and than throws the error, but not segmentation fault. This is still released version of SV.
The error:

Traceback (most recent call last):
File "/home/sergey/Workspace/detection-tiling-custom-obb/detect-image-slicer.py", line 89, in
detections = slicer(image)
^^^^^^^^^^^^^
File "/home/sergey/anaconda3/envs/yolo/lib/python3.11/site-packages/supervision/detection/tools/inference_slicer.py", line 162, in call
detections_list.append(future.result())
^^^^^^^^^^^^^^^
File "/home/sergey/anaconda3/envs/yolo/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/home/sergey/anaconda3/envs/yolo/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/home/sergey/anaconda3/envs/yolo/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sergey/anaconda3/envs/yolo/lib/python3.11/site-packages/supervision/detection/tools/inference_slicer.py", line 191, in _run_callback
detections = self.callback(image_slice)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sergey/Workspace/detection-tiling-custom-obb/detect-image-slicer.py", line 50, in callback
result = model(image_slice, conf=0.6)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sergey/anaconda3/envs/yolo/lib/python3.11/site-packages/ultralytics/engine/model.py", line 176, in call
return self.predict(source, stream, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sergey/anaconda3/envs/yolo/lib/python3.11/site-packages/ultralytics/engine/model.py", line 547, in predict
self.predictor.setup_model(model=self.model, verbose=is_cli)
File "/home/sergey/anaconda3/envs/yolo/lib/python3.11/site-packages/ultralytics/engine/predictor.py", line 303, in setup_model
self.model = AutoBackend(
^^^^^^^^^^^^
File "/home/sergey/anaconda3/envs/yolo/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/sergey/anaconda3/envs/yolo/lib/python3.11/site-packages/ultralytics/nn/autobackend.py", line 145, in init
model = model.fuse(verbose=verbose)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sergey/anaconda3/envs/yolo/lib/python3.11/site-packages/ultralytics/nn/tasks.py", line 208, in fuse
delattr(m, "bn") # remove batchnorm
^^^^^^^^^^^^^^^^
File "/home/sergey/anaconda3/envs/yolo/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1799, in delattr
super().delattr(name)
AttributeError: 'Conv' object has no attribute 'bn'

[LinasKo]

Could be an ultralytics version issue. Unless you need yolo11, I'd try ultralytics==8.2.103.

[zxsk1974]

@LinasKo apologies for the delay - I got pulled into another area of the project. I just tried your branch build and it worked very well. I observed 25-30% speed up from batch size 1 to batch size 10, than almost no speedup up to batch size 100+, since GPU utilization became bottle neck. I noticed that your branch was forked from 0.24.0 version, so if you merge 0.25.1 changes into your branch, I can test it again. Please let me know when it will merged into release branch. Thank you again.

[SkalskiP]

Hi @zxsk1974 👋🏻 no worries! Which version of the code you just tested?

[zxsk1974]

From this branch - batched-inference-slicer:
pip install git+https://github.com/LinasKo/supervision.git@feature/batched-inference-slicer.