Skip to content
This repository has been archived by the owner on Dec 1, 2021. It is now read-only.

The debug binary now show how much memory it will try to allocate on the heap #595

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

The debug binary now show how much memory it will try to allocate on the heap #595

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
81abf5a
The debug binary now show how much memory it will try to allocate on …
antonionevado Nov 7, 2019
73fb550
Merge branch 'master' into show_heap_memory_usage
antonionevado Nov 7, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions dlk/python/dlk/templates/include/network.tpl.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ class SYM_PUBLIC Network
Network();
~Network();

int memory();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep consistency of API, it looks better to use verb for the method name. Considering that the responsibility of this function should output debug information, how about renaming to get_debug_info?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I propose to change this API to return string, (or char*), so that we can see the buffer size layer by layer. (Is it possible to change the behavior of this API?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's true. The name can be better and we should return more detailed information.

bool init();

int get_input_rank();
Expand Down
1 change: 1 addition & 0 deletions dlk/python/dlk/templates/mains/main.tpl.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ int main(int argc, char *argv[])

Measurement::Start("TotalInitTime");
Network nn;
std::cout << "This network will try to allocate " << nn.memory() / 1024.0 / 1024.0 << " MB on the heap" << std::endl;
bool initialized = nn.init();
Measurement::Stop();

Expand Down
26 changes: 26 additions & 0 deletions dlk/python/dlk/templates/src/network.tpl.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,32 @@ Network::~Network()
#endif
}

int Network::memory() {
int total_mem_bytes = 0;
{% for node in graph.non_variables -%}
{% if node.available_buffer == '' -%}
{% for out_k in node.output_ops.keys() -%}
{% if node.output_ops.keys()|length > 1 -%}
total_mem_bytes += sizeof({{ node.dtype.cpptype() }}) * ({{ node.view.size_in_words_as_cpp }}); // {{ node.name + '_' + out_k }}_raw
{% else -%}
total_mem_bytes += sizeof({{ node.dtype.cpptype() }}) * ({{ node.view.size_in_words_as_cpp }}); // {{ node.name }}_raw
{%- endif %}
{%- endfor %}
{% elif node.available_buffer != '' and node.output_ops.keys()|length > 1 -%}
{% for out_k in node.output_ops.keys() -%}
{% if out_k != node.output_ops.keys()|list|first -%}
total_mem_bytes += sizeof({{ node.dtype.cpptype() }}) * ({{ node.view.size_in_words_as_cpp }}); // {{ node.name + '_' + out_k }}_raw
{%- endif %}
{%- endfor %}
{%- endif %}
{%- endfor %}
#if !defined RUN_ON_FPGA
total_mem_bytes += sizeof(QUANTIZED_PACKED) * max_device_input_elems;
total_mem_bytes += sizeof(BIN_CONV_OUTPUT) * max_device_output_elems;
#endif
return total_mem_bytes;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several temporary buffer allocated by some operators (ex. kn2row_buf in src/func/impl/generic/quantized_conv2d_kn2row.cpp), but these are not counted yet.
I'm working to allocate these hidden temporary buffers in Network::init() ( #473 ), but it may take a while...
Until it is resolved, we should count these by hand...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah thank you! Didn't know about these buffers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can wait until they are allocated from Network::init(), if you want.
Or we can add it later and count by hand as you said 😄

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These buffers are allocated by std::make_unique.

I found such buffers in following files:

  • src/func/conv2d.cpp
  • src/func/*/batch_normalization.cpp
  • src/func/impl/generic/quantized_conv2d_kn2row.cpp
  • src/func/impl/arm_neon/quantized_conv2d_tiling.cpp
  • src/func/impl/x86_avx/quantized_conv2d_tiling.cpp
  • src/matrix/multiplication.cpp

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes, seems that for changing layout we need these temporaries...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When #473 is fixed, buffers may be unified into single buffer, total buffer size is obvious.
It seems waiting for fixed is reasonable way.

}

bool Network::init()
{

Expand Down