Image Inference from memory

[DrewRidley]

❔Question

Hello, I was wondering if there is a way to inference on images in memory. For example, I wanted to serve the model as a web server but writing the file to the disk to inference and then deleting the file invokes significant overhead that could easily be avoided. If this is at all possible to do I would much appreciate some insight.
Thanks.

Additional context

[github-actions]

👋 Hello @DrewDaPilot, thank you for your interest in 🚀 YOLOv5! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://www.ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python 3.8 or later with all requirements.txt dependencies installed, including torch>=1.7. To install run:

$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), testing (test.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@DrewDaPilot yes, you can pass various different media formats directly to YOLOv5 PyTorch Hub models for inference, including of course images already loaded into memory. See PyTorch Hub Tutorial for details

yolov5/models/common.py

Lines 238 to 247 in 9ccfa85

	def forward(self, imgs, size=640, augment=False, profile=False):
	# Inference from various sources. For height=720, width=1280, RGB images example inputs are:
	# filename: imgs = 'data/samples/zidane.jpg'
	# URI: = 'https://github.com/ultralytics/yolov5/releases/download/v1.0/zidane.jpg'
	# OpenCV: = cv2.imread('image.jpg')[:,:,::-1] # HWC BGR to RGB x(720,1280,3)
	# PIL: = Image.open('image.jpg') # HWC x(720,1280,3)
	# numpy: = np.zeros((720,1280,3)) # HWC
	# torch: = torch.zeros(16,3,720,1280) # BCHW
	# multiple: = [Image.open('image1.jpg'), Image.open('image2.jpg'), ...] # list of images

YOLOv5 Tutorials

• Train Custom Data  🚀 RECOMMENDED
• Tips for Best Training Results  ☘️ RECOMMENDED
• Weights & Biases Logging  🌟 NEW
• Supervisely Ecosystem  🌟 NEW
• Multi-GPU Training
• PyTorch Hub  ⭐ NEW
• ONNX and TorchScript Export
• Test-Time Augmentation (TTA)
• Model Ensembling
• Model Pruning/Sparsity
• Hyperparameter Evolution
• Transfer Learning with Frozen Layers  ⭐ NEW
• TensorRT Deployment

[DrewRidley]

Thanks for the reply. I tried what you suggested and I get a strange error.

results = model([pilImg], size=640) File "C:\Users\Drew\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "C:\Users\Drew/.cache\torch\hub\ultralytics_yolov5_master\models\common.py", line 260, in forward files.append(Path(im.filename).with_suffix('.jpg').name if isinstance(im, Image.Image) else f'image{i}.jpg') File "C:\Users\Drew\anaconda3\lib\pathlib.py", line 868, in with_suffix raise ValueError("%r has an empty name" % (self,)) ValueError: WindowsPath('.') has an empty name 127.0.0.1 - - [05/Apr/2021 10:52:47] "POST /upload HTTP/1.1" 500 -

'import torch
from flask import Flask, request
from PIL import Image

model = torch.hub.load('ultralytics/yolov5', 'yolov5s')
app = Flask(name)

@app.route('/upload', methods=['POST'])
def upload():
if 'file' in request.files:
image = request.files['file']
pilImg = Image.open(image.stream)
results = model([pilImg], size=640)
results.print()
return '{ success: true }'
return "{ success: false}"'

[glenn-jocher]

@DrewDaPilot thanks for the bug report. This is a duplicate issue of #2702 which was also raised today, and is caused by PIL Image objects lacking filenames. We should have a fix for this later today.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[coneill65]

@DrewDaPilot thanks for the bug report. This is a duplicate issue of #2702 which was also raised today and is caused by PIL Image objects lacking filenames. We should have a fix for this later today.
@glenn-jocher by any chance has this been fixed? cause I have been having this same error.
here is my code.

import torch
import d3dshot

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True)


d = d3dshot.create()
img = d.screenshot()
# Image
# img = 'https://github.com/ultralytics/yolov5/raw/master/data/images/zidane.jpg'  # batched list of images

# Inference
results = model(img)

# Results
results.print()
results.show()  # or .show()

# Data
print(results.pandas().xyxy[0])  # print img1 predictions (pixels)
#                   x1           y1           x2           y2   confidence        class
# tensor([[7.50637e+02, 4.37279e+01, 1.15887e+03, 7.08682e+02, 8.18137e-01, 0.00000e+00],
#         [9.33597e+01, 2.07387e+02, 1.04737e+03, 7.10224e+02, 5.78011e-01, 0.00000e+00],
#         [4.24503e+02, 4.29092e+02, 5.16300e+02, 7.16425e+02, 5.68713e-01, 2.70000e+01]])

here's the error.

Traceback (most recent call last):
  File "C:/Users/colin/ai_projects/main.py", line 14, in <module>
    results = model(img)
  File "C:\Users\colin\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\colin\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\torch\autograd\grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\colin/.cache\torch\hub\ultralytics_yolov5_master\models\common.py", line 258, in forward
    im, f = np.asarray(exif_transpose(im)), getattr(im, 'filename') or f
AttributeError: 'Image' object has no attribute 'filename'

[coneill65]

the error is because it's not encoded as a NumPy image here is the working code for anyone that may be interested.

import torch
import d3dshot

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True)


d = d3dshot.create(capture_output="numpy")
img = d.screenshot()
# Image
# img = 'https://github.com/ultralytics/yolov5/raw/master/data/images/zidane.jpg'  # batched list of images

# Inference
results = model(img)

# Results
results.print()
results.show()  # or .show()

# Data
print(results.pandas().xyxy[0])  # print img1 predictions (pixels)
#                   x1           y1           x2           y2   confidence        class
# tensor([[7.50637e+02, 4.37279e+01, 1.15887e+03, 7.08682e+02, 8.18137e-01, 0.00000e+00],
#         [9.33597e+01, 2.07387e+02, 1.04737e+03, 7.10224e+02, 5.78011e-01, 0.00000e+00],
#         [4.24503e+02, 4.29092e+02, 5.16300e+02, 7.16425e+02, 5.68713e-01, 2.70000e+01]])

[glenn-jocher]

@coneill65 if a 3rd party package (especially one not in our requirements) is causing you issues in your custom code the place to inquire is directly with the package authors.