The debug binary now show how much memory it will try to allocate on the heap

[antonionevado]

Currently there is no way of knowing how much memory a network model requires for activations on deployment.

Motivation and Context

It is possible to know the required memory from the Tensorflow model but will not match with the actual required amount of memory due to buffer reuse and quantization. This PR will make the debug binary (lm_xxx.elf) to print a message that show how much memory will be allocated on the heap before calling Network::init(). This allows you to have an idea of how much memory will be used and, debug possibly related memory issues.

Description

• Added a new method to Network class that returns the amount of memory in bytes.
• Use this method from mains/main.cpp and print how many megabytes will be required by the model.

How has this been tested?

Tested with 2 of the provided examples:

• classification: lmnet_quantized_cifar10
• object detection: widerface_v5

Screenshots (if appropriate):

For object detection this will show:

This network will try to allocate 10.2753 MB on the heap
-------------------------------------------------------------
Comparison: Default network test  succeeded!!!
-------------------------------------------------------------

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature / Optimization (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.

[primenumber]

There are several temporary buffer allocated by some operators (ex. kn2row_buf in src/func/impl/generic/quantized_conv2d_kn2row.cpp), but these are not counted yet.
I'm working to allocate these hidden temporary buffers in Network::init() ( #473 ), but it may take a while...
Until it is resolved, we should count these by hand...

[antonionevado]

Ah thank you! Didn't know about these buffers.

[antonionevado]

We can wait until they are allocated from Network::init(), if you want.
Or we can add it later and count by hand as you said 😄

[primenumber]

These buffers are allocated by std::make_unique.

I found such buffers in following files:

• src/func/conv2d.cpp
• src/func/*/batch_normalization.cpp
• src/func/impl/generic/quantized_conv2d_kn2row.cpp
• src/func/impl/arm_neon/quantized_conv2d_tiling.cpp
• src/func/impl/x86_avx/quantized_conv2d_tiling.cpp
• src/matrix/multiplication.cpp

[antonionevado]

Ah, yes, seems that for changing layout we need these temporaries...

[primenumber]

When #473 is fixed, buffers may be unified into single buffer, total buffer size is obvious.
It seems waiting for fixed is reasonable way.

[tkng]

To keep consistency of API, it looks better to use verb for the method name. Considering that the responsibility of this function should output debug information, how about renaming to get_debug_info?

[tkng]

Also, I propose to change this API to return string, (or char*), so that we can see the buffer size layer by layer. (Is it possible to change the behavior of this API?)

[antonionevado]

Yes, it's true. The name can be better and we should return more detailed information.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}