Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for creating VideoFrame directly from bytes (from_bytes) #1513

Merged
merged 1 commit into from
Sep 3, 2024

Conversation

DexerBR
Copy link
Contributor

@DexerBR DexerBR commented Sep 3, 2024

This pull request introduces a new feature to the VideoFrame class, allowing the creation of a VideoFrame object directly from byte data. With the new from_bytes method, it's possible to build a frame without the need for an intermediate converter like PIL or NumPy.

Key changes:

  • Added from_bytes method: Allows creating a VideoFrame from bytes, specifying width, height, image format, and options for horizontal and vertical flipping.
  • Refactored copy_array_to_plane function: Updated to call a new helper function, copy_bytes_to_plane, which handles byte copying with support for flipping.

These changes remove the need for intermediate converters, such as PIL or NumPy, streamlining the process of creating video frames directly from bytes. The implementation supports the "rgba" format and raises an exception for unsupported formats.

Performance Comparison:

Here are the results of the performance tests for different implementations (using a 1280x720px buffer in rgba format, unsigned byte):

Implementation 1: Using PIL to create an image and then converting it to a VideoFrame:

from PIL import Image
import time

start_time = time.time()

img = Image.frombytes("RGBA", (width, height), buffer).transpose(Image.FLIP_TOP_BOTTOM)
frame = av.VideoFrame.from_image(img)

end_time = time.time()
print(f"Implementation 1 - Took ~ {end_time - start_time}")

Implementation 1 - Took ~ 0.0301 seconds.

Implementation 2: Directly creating a VideoFrame from bytes, with support for flipping:

start_time = time.time()
import time

frame = av.VideoFrame.from_bytes(buffer, width, height, flip_horizontal=False, flip_vertical=True)

end_time = time.time()
print(f"Implementation 2 - Took ~ {end_time - start_time}")

Implementation 2 - Took ~ 0.0023 seconds.

That is a performance improvement of around 13x!

@WyattBlue WyattBlue merged commit 5dc17e8 into PyAV-Org:main Sep 3, 2024
10 checks passed
@DexerBR DexerBR deleted the from_bytes branch September 3, 2024 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants