problem when running decode sample #106

incomingflyingbrick · 2024-09-18T08:29:23Z

I am trying to decode a video using a sample, but I got the error saying source and output dimention mismatch at this line
# Copy pixels to numpy ndarray success, details = pyDwn.Run(surf_dst, frame)
decode sample
Is this a software bug of the library ?

The text was updated successfully, but these errors were encountered:

RomanArzumanyan · 2024-09-18T08:34:34Z

Hi @incomingflyingbrick

What input do you use ? Is it same as in sample or did you change it ?
This may happen if video change it's resolution in the middle. Then PySurfaceDownloader will scold incoming numpy ndarray because it no longer matches the updated Surface resolution.

incomingflyingbrick · 2024-09-18T14:27:22Z

@RomanArzumanyan Hi, I tried different videos with mp4 format, including the video in the test folder named test.mp4, but none of them worked, still showing the following error.
TaskExecInfo.SRC_DST_SIZE_MISMATCH Traceback (most recent call last): File "/storage/ongoing/new/test_decord/test_vali.py", line 78, in <module> raise StopExecution StopExecution

RomanArzumanyan · 2024-09-18T14:37:41Z

@incomingflyingbrick

That's very strange.
Can you modify the sample to output both surf_dst and frame shape ?
It can be done like this:

print(frame.shape)
print(surf_dst.__cuda_array_interface__)

# Copy pixels to numpy ndarray
success, details = pyDwn.Run(surf_dst, frame)
if not success:
    print(details)
    raise StopExecution

When working properly, the output shall be like this (for test.mp4 video of resolution 848x464):

(1180416,)
{'shape': (464, 848, 3}, ...

848x464 RGB Surface requires 848x464x3 == 1180416 bytes.

incomingflyingbrick · 2024-09-18T14:51:13Z

@RomanArzumanyan
Hi, I changed the code and got the following output

(2764800,)
{'shape': (1280, 720, 3), 'typestr': '<u1', 'data': (139942994903040, False), 'version': 3, 'strides': (2560, 3, 1), 'stream': 36464064}
TaskExecInfo.SRC_DST_SIZE_MISMATCH
Traceback (most recent call last):
  File "/storage/ongoing/new/test_decord/test_vali.py", line 79, in <module>
    raise StopExecution
StopExecution

It seems the shape is correct, but still unable to obtain a successful result.

RomanArzumanyan · 2024-09-19T07:56:12Z

Hi @incomingflyingbrick

It looks like something is happening in the C++ land although I'm not sure what.
Can you build VALI from branch issue_106 and run the same decoding sample ? It adds more verbose debug output like this:

/home/roman/git/VALI/src/python_vali/src/PySurfaceDownloader.cpp::Run
array size:   1180416
buffer size:  1180416
surface size: 1180416
/home/roman/git/VALI/src/TC/src/TaskCudaDownloadSurface.cpp::Run
buffer size:  1180416
surface size: 1180416

Building from source is not difficult, you just check out the feature branch and run pip isntall . in console.

incomingflyingbrick · 2024-09-20T06:48:52Z

@RomanArzumanyan I got the following output using the issue_106 branch build from the source.

array size:   0
buffer size:  0
surface size: 230400
/storage/ongoing/new/test_decord/VALI/src/TC/src/TaskCudaDownloadSurface.cpp::Run
buffer size:  0
surface size: 230400
TaskExecInfo.SRC_DST_SIZE_MISMATCH
Traceback (most recent call last):
  File "/storage/ongoing/new/test_decord/test_vali.py", line 79, in <module>
    raise StopExecution
StopExecution

RomanArzumanyan · 2024-09-20T06:57:35Z

Thanks for the update @incomingflyingbrick

It means that ndarray was created on the python side with size of 230400 but those changes aren't seen on C++ side for some reason.

Need to google on this. Looks like some sort of lazy init of ndarray. Didn't face that before.
I'll come back as I find something.

P. S.
Meanwhile may I ask you to check the lazy init suggestion by initializing ndarray with zeroes ?

# Numpy array which contains decoded RGB Surface
frame = np.ndarray(
    dtype=np.uint8,
    shape=surf_dst.HostSize)
frame[:]=0

incomingflyingbrick · 2024-09-20T09:10:02Z

@RomanArzumanyan
Hi I got the same error again after I changed to lazy init.

/storage/ongoing/new/test_decord/VALI/src/python_vali/src/PySurfaceDownloader.cpp::Run
array size:   0
buffer size:  0
surface size: 230400
/storage/ongoing/new/test_decord/VALI/src/TC/src/TaskCudaDownloadSurface.cpp::Run
buffer size:  0
surface size: 230400
TaskExecInfo.SRC_DST_SIZE_MISMATCH
Traceback (most recent call last):
  File "/storage/ongoing/new/test_decord/test_vali.py", line 80, in <module>
    raise StopExecution
StopExecution

wuqingzhou828 · 2024-11-19T09:43:11Z

I am trying to decode a rtsp using a sample, but I got the error saying source and output dimention mismatch at this line

VALI/samples/sample_decode_show.ipynb

Lines 118 to 122 in 3889a79

    
           "    # Copy pixels to numpy ndarray\n", 
        
           "    success, details = pyDwn.Run(surf_dst, frame)\n", 
        
           "    if not success:\n", 
        
           "        print(details)\n", 
        
           "        raise StopExecution\n",

Is this a software bug of the library ?
this is my code:

import numpy as np
from PIL import Image
import python_vali as vali

url = "rtsp://xxx:xxx@xx/cam/realmonitor?channel=1&subtype=0"

class StopExecution(Exception):
    def _render_traceback_(self):
        return []

if __name__ == '__main__':
    # GPU-accelerated decoder
    pyDec = vali.PyDecoder(
        url,
        {'rtsp_transport': 'tcp'},
        gpu_id=0)
    print("# GPU-accelerated decoder")

    # GPU-accelerated converter
    pyCvt = vali.PySurfaceConverter(
        pyDec.Format,
        vali.PixelFormat.RGB,
        gpu_id=0)

    # GPU-accelerated Surface downloader
    pyDwn = vali.PySurfaceDownloader(gpu_id=0)

    # Raw decoded Surface
    print(pyDec.Format, pyDec.Width, pyDec.Height)  # PixelFormat.NV12 3840 2160

    # Raw decoded Surface
    surf_src = vali.Surface.Make(
        format=pyDec.Format,
        width=pyDec.Width,
        height=pyDec.Height,
        gpu_id=0)

    # Raw Surface, converted to RGB
    surf_dst = vali.Surface.Make(
        format=vali.PixelFormat.RGB,
        width=pyDec.Width,
        height=pyDec.Height,
        gpu_id=0)

    # Numpy array which contains decoded RGB Surface
    # frame = np.ndarray(
    #     dtype=np.uint8,
    #     shape=surf_dst.HostSize)
    frame = np.ndarray(
        dtype=np.uint8,
        shape=surf_dst.HostSize)
    print('surf_dst.HostSize', surf_dst.HostSize)



    # Packet data
    pkt_data = vali.PacketData()

    print("frame", frame.shape) # frame (24883200,)
    print("surf_src", surf_src.__cuda_array_interface__) # surf_src {'shape': (3240, 3840, 0), 'typestr': '<u1', 'data': (139628011061248, False), 'version': 3, 'strides': (4096, 1, 0), 'stream': 94450176927296}
    print("surf_dst", surf_dst.__cuda_array_interface__) # surf_dst {'shape': (2160, 3840, 3), 'typestr': '<u1', 'data': (139627977506816, False), 'version': 3, 'strides': (11776, 3, 1), 'stream': 94450176927296}

    # Packet data
    pkt_data = vali.PacketData()

    for i in range(1):
        # Decode single Surface
        success, details = pyDec.DecodeSingleSurface(surf_src, pkt_data)
        if not success:
            print(details)
            raise StopExecution
        print('pkt_data', pkt_data.dts, pkt_data.pts)

        # Convert to RGB
        success, details = pyCvt.Run(surf_src, surf_dst)
        print('# Convert to RGB', details)
        if not success:
            print(details)
            raise StopExecution

        # Copy pixels to numpy ndarray
        success, details = pyDwn.Run(surf_dst, frame)
        if not success:
            print(details)
            raise StopExecution

        # Reshape to proper dimensions
        res_frame = np.reshape(
            frame,
            (pyDec.Height, pyDec.Width, 3))

        # Save the image
        image = Image.fromarray(res_frame)
        image.save('./test.jpg')

        # Output packet data
        print(pkt_data)

RomanArzumanyan · 2024-11-19T11:34:42Z

Hi @wuqingzhou828

Could you please check this small patch:

# Numpy array which contains decoded RGB Surface
frame = np.ndarray(
    dtype=np.uint8,
    shape=(surf_dst.HostSize))

Looks like I'm passing a single value to constructor instead of tuple.
BTW this is how it's done in unit tests:

VALI/tests/test_PySurface.py

Lines 116 to 119 in 3889a79

    
           # Check if memory is bit 2 bit equal 
        
           frame_dst = np.ndarray( 
        
               shape=(surf_dst.HostSize), 
        
               dtype=np.uint8)

RomanArzumanyan · 2024-11-20T08:01:43Z

@incomingflyingbrick

Tagging for better visibility.
Could you please do the same check on your machine ?

wuqingzhou828 · 2024-11-21T02:39:02Z

success, details = pyDwn.Run(surf_dst, frame)

success, details = pyDwn.Run(surf_dst, frame), # Convert to RGB TaskExecInfo.SUCCESS
the code rasie StopExecution, by this code success, details = pyDwn.Run(surf_dst, frame)

RomanArzumanyan · 2024-11-21T09:49:39Z

@wuqingzhou828

Sorry, I don't understand your reply. Could you please explain ?
Does this change solve the problem with decoding sample ?

# Numpy array which contains decoded RGB Surface
frame = np.ndarray(
    dtype=np.uint8,
    shape=(surf_dst.HostSize))

RomanArzumanyan added a commit that referenced this issue Sep 19, 2024

issue #106: add debug output

cd63599

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

problem when running decode sample #106

problem when running decode sample #106

incomingflyingbrick commented Sep 18, 2024

RomanArzumanyan commented Sep 18, 2024

incomingflyingbrick commented Sep 18, 2024

RomanArzumanyan commented Sep 18, 2024 •

edited

Loading

incomingflyingbrick commented Sep 18, 2024 •

edited by RomanArzumanyan

Loading

RomanArzumanyan commented Sep 19, 2024

incomingflyingbrick commented Sep 20, 2024 •

edited by RomanArzumanyan

Loading

RomanArzumanyan commented Sep 20, 2024 •

edited

Loading

incomingflyingbrick commented Sep 20, 2024 •

edited by RomanArzumanyan

Loading

wuqingzhou828 commented Nov 19, 2024 •

edited by RomanArzumanyan

Loading

RomanArzumanyan commented Nov 19, 2024 •

edited

Loading

RomanArzumanyan commented Nov 20, 2024 •

edited

Loading

wuqingzhou828 commented Nov 21, 2024

RomanArzumanyan commented Nov 21, 2024

problem when running decode sample #106

problem when running decode sample #106

Comments

incomingflyingbrick commented Sep 18, 2024

RomanArzumanyan commented Sep 18, 2024

incomingflyingbrick commented Sep 18, 2024

RomanArzumanyan commented Sep 18, 2024 • edited Loading

incomingflyingbrick commented Sep 18, 2024 • edited by RomanArzumanyan Loading

RomanArzumanyan commented Sep 19, 2024

incomingflyingbrick commented Sep 20, 2024 • edited by RomanArzumanyan Loading

RomanArzumanyan commented Sep 20, 2024 • edited Loading

incomingflyingbrick commented Sep 20, 2024 • edited by RomanArzumanyan Loading

wuqingzhou828 commented Nov 19, 2024 • edited by RomanArzumanyan Loading

RomanArzumanyan commented Nov 19, 2024 • edited Loading

RomanArzumanyan commented Nov 20, 2024 • edited Loading

wuqingzhou828 commented Nov 21, 2024

RomanArzumanyan commented Nov 21, 2024

RomanArzumanyan commented Sep 18, 2024 •

edited

Loading

incomingflyingbrick commented Sep 18, 2024 •

edited by RomanArzumanyan

Loading

incomingflyingbrick commented Sep 20, 2024 •

edited by RomanArzumanyan

Loading

RomanArzumanyan commented Sep 20, 2024 •

edited

Loading

incomingflyingbrick commented Sep 20, 2024 •

edited by RomanArzumanyan

Loading

wuqingzhou828 commented Nov 19, 2024 •

edited by RomanArzumanyan

Loading

RomanArzumanyan commented Nov 19, 2024 •

edited

Loading

RomanArzumanyan commented Nov 20, 2024 •

edited

Loading