TIOVXDLPreProc: High input resolutions fails allocating TIVX memory

[dchvs99]

gst-launch-1.0  videotestsrc ! video/x-raw,width=7680,height=4320 ! tiovxdlpreproc ! application/x-tensor-tiovx ! fakesink
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=4) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
 33764.526450 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
 33764.526533 s:  VX_ZONE_INIT:Enabled
 33764.526545 s:  VX_ZONE_ERROR:Enabled
 33764.526554 s:  VX_ZONE_WARNING:Enabled
 33764.526978 s:  VX_ZONE_INIT:[tivxInitLocal:111] Initialization Done !!!
 33764.527190 s:  VX_ZONE_INIT:[tivxHostInit:48] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
MEM: ERROR: Alloc failed with status = 12 !!!
 33764.534308 s:  VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
 33764.534342 s:  VX_ZONE_ERROR:[ownAllocTensorBuffer:83] Could not allocate tensor memory
MEM: ERROR: Alloc failed with status = 12 !!!
 33764.534378 s:  VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
 33764.534395 s:  VX_ZONE_ERROR:[ownAllocTensorBuffer:83] Could not allocate tensor memory
MEM: ERROR: Alloc failed with status = 12 !!!
 33764.534426 s:  VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
 33764.534472 s:  VX_ZONE_ERROR:[ownAllocTensorBuffer:83] Could not allocate tensor memory
MEM: ERROR: Alloc failed with status = 12 !!!
 33764.534517 s:  VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
MEM: ERROR: Alloc failed with status = 12 !!!
 33764.534917 s:  VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
 33764.534929 s:  VX_ZONE_ERROR:[ownAllocImageBuffer:309] could not allocate memory
 33764.534938 s:  VX_ZONE_ERROR:[ownCopyAndMapCheckParams:672] image allocation failed
MEM: ERROR: Alloc failed with status = 12 !!!
 33764.535063 s:  VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data stream error.
Additional debug info:
../gstreamer-1.16.3/libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
 33764.536528 s:  VX_ZONE_WARNING:[vxReleaseContext:1012] Found a reference 0xffffa9b2a998 of type 00000813 at external count 1, internal count 0, releasing it
 33764.536546 s:  VX_ZONE_WARNING:[vxReleaseContext:1016] Unreleased reference name = object_array_104
 33764.536570 s:  VX_ZONE_WARNING:[vxReleaseContext:1018] Releasing reference now as a part of garbage collection
 33764.536825 s:  VX_ZONE_WARNING:[vxReleaseContext:1012] Found a reference 0xffffa9b2ab38 of type 00000813 at external count 1, internal count 0, releasing it
 33764.536836 s:  VX_ZONE_WARNING:[vxReleaseContext:1016] Unreleased reference name = object_array_106
 33764.536843 s:  VX_ZONE_WARNING:[vxReleaseContext:1018] Releasing reference now as a part of garbage collection
 33764.536951 s:  VX_ZONE_INIT:[tivxHostDeInit:56] De-Initialization Done for HOST !!!
 33764.541269 s:  VX_ZONE_INIT:[tivxDeInitLocal:155] De-Initialization Done !!!
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
IPC: Deinit ... !!!
IPC: DeInit ... Done !!!
MEM: Deinit ... !!!
MEM: Alloc's: 5 alloc's of 995328052 bytes 
MEM: Free's : 5 free's  of 995328052 bytes 
MEM: Open's : 0 allocs  of 0 bytes 
MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!

[dchvs99]

Regarding this issue:

The TIVXMemBufferAlloc depends not only on the frame's width & height but also in the format and data format type.

The supported resolution for the frames goes from 1 - 8192. That represents allocations per buffer as exemplified below:

RGB

width * height * data channels = buffer size
8192 * 8192 * 3 = 201.33 M

NV12

8192 * 8192 * 1.5 = 100.66 M

Now, these elements supports dynamic buffer pool size that goes from 2 - 16. And the default value is 2 for input and 2 for output; 4 buffers.
If I stress the plugin to handle the maximum resolution, even using the 4 buffers will run out of memory (the most it seems to handle at this resolutions it's ~ 2 buffers).

We also have to consider the use case of TIOVXDLPreProc using float32 data-type, which adds up more memory usage, per example:

RGB

8192 * 8192 * 3 * 4 = 805.31 M

The most memory consuming scenario would be:

RGB format with Float32 data type and the 16 buffer input/output pool demand:

tivxmemBufferAlloc = w * h * datachannel * datatype * (input bpool size + output bpool size)
         = 8192 * 8192 * 3 * 4 * (16 + 16) = 25769.80 M

Please let me know your thoughts about this constraint.

cc: @shyam-j , @shyamj , @rrcarlosrodriguez

[m-herrera]

This issue can be reproduced in the modules by setting the test application to use the same parameters and duplicating the number of allocs to simulate a pool size of 2, the following patch shows those changes.

---
 test/app_tiovx_dl_pre_proc_module_test.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/test/app_tiovx_dl_pre_proc_module_test.c b/test/app_tiovx_dl_pre_proc_module_test.c
index 1e751e6..c366bbe 100644
--- a/test/app_tiovx_dl_pre_proc_module_test.c
+++ b/test/app_tiovx_dl_pre_proc_module_test.c
@@ -66,8 +66,8 @@
 #define APP_BUFQ_DEPTH   (1)
 #define APP_NUM_CH       (1)
 
-#define IMAGE_WIDTH      (640)
-#define IMAGE_HEIGHT     (480)
+#define IMAGE_WIDTH      (7680)
+#define IMAGE_HEIGHT     (4320)
 
 typedef struct {
 
@@ -163,7 +163,7 @@ static vx_status app_init(AppObj *obj)
         dlPreProcObj->input.height = IMAGE_HEIGHT;
 
         dlPreProcObj->output.bufq_depth = APP_BUFQ_DEPTH;
-        dlPreProcObj->output.datatype = VX_TYPE_UINT8;
+        dlPreProcObj->output.datatype = VX_TYPE_FLOAT32;
         dlPreProcObj->output.num_dims = 3;
         /* This can be updated based on NCHW or NHWC */
         dlPreProcObj->output.dim_sizes[0] = IMAGE_WIDTH;
@@ -282,6 +282,8 @@ static vx_status app_run_graph(AppObj *obj)
 
     /* These can be moved to app_init() */
     allocate_image_buffers(&dlPreProcObj->input, inAddr, inSizes);
+    allocate_image_buffers(&dlPreProcObj->input, inAddr, inSizes);
+    allocate_tensor_buffers(&dlPreProcObj->output, outAddr, outSizes);
     allocate_tensor_buffers(&dlPreProcObj->output, outAddr, outSizes);
 
     bufq = 0;

After running the test app it show the same error:

MEM: ERROR: Alloc failed with status = 12 !!!