Heap: Improve debug caller address reporting #8720

mhightower83 · 2022-11-21T18:32:06Z

PR Status: ready for review

Heap: Improve debug caller address reporting to indicate the address in the sketch or library.

Lite refactoring of heap.cpp and expanded comments

Lite refactoring of DEBUG_ESP_OOM printing
Overall refactoring to make it easier to follow.
Changes to support atomic save of OOM debug info
Debug support for C++ "delete" operator - support reporting caller address

Overall avoid identifying the wrapper code as the source of the Heap call. Each outer wrapper passes the caller's address down to the next level.

Improves "caller" address reporting for OOM and Poison events. Previously, some OOM and Poison events reported info pointing to the thick heap wrapper rather than back to the caller. Update abi.cpp, heap.cpp and umm_local.c (umm_poison) to support caller parameters.

Enhanced DEBUG_ESP_OOM to track OOM events when C++ Exceptions: "enabled" is selected. Report more interesting higher-level code addresses as the caller instead of GCC C++ module.

Added DEBUG_ESP_WITHINISR to provide notification for ISRs using C++ "new"/"delete" operators or LIBC calls using _malloc_r, ...

Enhanced poison neighbor reporting to distinguish between allocation vs. neighbor failing poison check.

Support for specifying both UMM_POISON_CHECK and UMM_POISON_CHECK_LITE at the same time did not work as expected. UMM_POISON_CHECK_LITE dominated in the build. Fix: Keep things simple and prevent combining UMM_POISON_CHECK and UMM_POISON_CHECK_LITE. Defines for UMM_POISON_CHECK, UMM_POISON_CHECK_LITE, and UMM_POISON_NONE are checked for mutual exclusivity.

For sketches with extremely limited resources, added UMM_POISON_NONE to prevent UMM_POISON_CHECK_LITE from being added to debug builds.

Moved port related changes for UMM_POISON... from umm_malloc_cfg.h to umm_malloc_cfgport.h.

Draft version remnants removed.

Updated this description to reflect the current PR

lite refactoring of DEBUG_ESP_OOM printing Overall refactoring to make easier to follow. Added ABI changes includes atomic save of OOM debug info Added debug support for C++ "delete" operator Avoid identifying the wrapper code as the source of the Heap call. Each outer wrapper passes the caller's address down to the next level. Improves "caller" address reporting for OOM and Poison events. Previously, some OOM and Poison events reported info pointing to the thick heap wrapper rather than back to the caller. Update abi.cpp, heap.cpp and umm_local.c (umm_poison) to support caller parameter.

sketch or library. Enhanced DEBUG_ESP_OOM to track OOM events when C++ Exceptions: "enabled" is selected. Report more interesting higher level code address as caller instead of GCC C++ module. Enhanced poison neighbor reporting to distinguish between allocation vs. neighbor failing poison check. For draft version, added debug macro HEAP_DEBUG_PROBE_PSFLC_CB to aid in evaluating caller address results.

Added DEBUG_ESP_WITHINISR to provide notification for ISRs using C++ 'new'/'delete' operators or LIBC calls using _malloc_r, ... Moved block of port related changes for UMM_POISON... from `umm_malloc_cfg.h` to `umm_malloc_cfgport.h`. Support for specifying both UMM_POISON_CHECK and UMM_POISON_CHECK_LITE at the same time did not work as expected. UMM_POISON_CHECK_LITE dominated in the build. Fix: Keep things simple prevent combining UMM_POISON_CHECK and UMM_POISON_CHECK_LITE. Defines for UMM_POISON_CHECK, UMM_POISON_CHECK_LITE, and UMM_POISON_NONE are checked for mutual exclusivity. For sketches with extremely limited resources, added UMM_POISON_NONE to prevent UMM_POISON_CHECK_LITE from being added to debug builds. Make UMM_POISON_CHECK_LITE fail messages more specific about the location. At free distinguishes between neighbor and allocation. And, identify the caller's address.

cores/esp8266/core_esp8266_postmortem.cpp

pvPortMalloc(,,,true) case. Improved comments.

…nto pr-heap-refactor3

mhightower83 · 2023-04-18T20:28:51Z

Let me know if you prefer I factor out experimental heap_cb.h.

mcspr · 2023-04-28T12:49:54Z

Let me know if you prefer I factor out experimental heap_cb.h

If it is still a draft, probably makes sense to leave it out of the PR to reduce loc

Updated umm_poison move from umm_malloc_cfg.h to umm_malloc_cfgport.h

in postmortem and printing OOM from Heap wrappers. Fixed crash within postmortem when OOM file is NULL Fixed printing OOM file name when not in ISR.

removed experimental debug extension as requested added forgotten OOM count to the postmortem report

resolved conflicts with umm_malloc_cfg.h and umm_malloc_cfgport.h

mcspr

Updated this description to reflect the current PR state: not ready

Still true?

mcspr · 2024-09-01T10:55:59Z

cores/esp8266/abi.cpp

+void * operator new (std::size_t size)
+{
+    void *p;


(not necessary here, whole file-wide)

There is discrepancy in ptr * style

void* a(); void *b(); void * c(); ... void foo() { void *ptr; } ...

Can feed this file to clang-format afterwards, though.
Original style prefers TYPE* NAME, like most C++ libs in general

should be corrected now

mcspr · 2024-09-01T11:34:35Z

cores/esp8266/heap.cpp

+// The sized deletes are defined in other files.
+#pragma GCC diagnostic ignored "-Wsized-deallocation"


It is just below, though. I don't see a warning here

I finally remembered to remove this one. Some comments I found for Clang indicated that it was needed for older compilers.

mcspr · 2024-09-01T11:41:26Z

cores/esp8266/core_esp8266_postmortem.cpp

-        ets_printf_P(PSTR("\nlast failed alloc call: %08X(%d)@%S:%d\n"),
-            (uint32_t)umm_last_fail_alloc_addr, umm_last_fail_alloc_size,
-            umm_last_fail_alloc_file, umm_last_fail_alloc_line);
+        ets_printf_P(PSTR("\nlast failed alloc call: 0x%08X(%d), File: %S:%d\n"),


%s not %S?

%S, umm_last_fail_alloc_file can be a PROGMEM string
At least some SDK file name strings are in .irom.text.

mcspr · 2024-09-01T12:04:29Z

cores/esp8266/heap.cpp

+#define UMM_MALLOC_FL(s,f,l,c)    umm_poison_malloc(s)
+#define UMM_CALLOC_FL(n,s,f,l,c)  umm_poison_calloc(n,s)
+#define UMM_REALLOC_FL(p,s,f,l,c) umm_poison_realloc_flc(p,s,f,l,c)
+#define UMM_FREE_FL(p,f,l,c)      umm_poison_free_flc(p,f,l,c)


_FL -> _FLC? same as standalone funcs named now

Remove the backport remnants. Cleanup UMM_MALLOC_FL and UMM_CALLOC_FL.
should now be fixed.

mcspr · 2024-09-01T12:10:52Z

cores/esp8266/heap.cpp

+#define UMM_MALLOC_FL(s,f,l,c)    umm_poison_malloc(s)
+#define UMM_CALLOC_FL(n,s,f,l,c)  umm_poison_calloc(n,s)
+#define UMM_REALLOC_FL(p,s,f,l,c) umm_poison_realloc_flc(p,s,f,l,c)
+#define UMM_FREE_FL(p,f,l,c)      umm_poison_free_flc(p,f,l,c)


would it make sense to use struct à la source_location to the reduce number of args?

Nice! That would be convenient.

Thoughts and Concerns:

Can source_location be used from .c modules?

Will function names be stored in PROGMEM or DRAM?

Their example shows a longer (full definition) string than we have used for the function name, which could cause build size issues for debug builds.

Hmm, I am not able to build with #include <source_location>

Detecting libraries used... /home/mhightow/Arduino/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-g++ -D__ets__ -DICACHE_FLASH -U__STRICT_ANSI__ -D_GNU_SOURCE -DESP8266 -Os @/tmp/arduino_build_311930/core/build.opt -I/home/mhightow/Arduino/hardware/esp8266com/esp8266/tools/sdk/include -I/home/mhightow/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip2/include -I/home/mhightow/Arduino/hardware/esp8266com/esp8266/tools/sdk/libc/xtensa-lx106-elf/include -I/tmp/arduino_build_311930/core -c @/home/mhightow/Arduino/hardware/esp8266com/esp8266/tools/warnings/none-cppflags -g -free -fipa-pta -Werror=return-type -mlongcalls -mtext-section-literals -fno-rtti -falign-functions=4 -std=gnu++17 -ffunction-sections -fdata-sections -fexceptions -DMMU_IRAM_SIZE=0xC000 -DMMU_ICACHE_SIZE=0x4000 -DFP_IN_IROM -w -x c++ -E -CC -DNONOSDK22x_190703=1 -DF_CPU=80000000L -DLWIP_OPEN_SRC -DTCP_MSS=536 -DLWIP_FEATURES=1 -DLWIP_IPV6=0 -DDEBUG_ESP_PORT=Serial -DARDUINO=10819 -DARDUINO_ESP8266_NODEMCU_ESP12E -DARDUINO_ARCH_ESP8266 "-DARDUINO_BOARD=\"ESP8266_NODEMCU_ESP12E\"" "-DARDUINO_BOARD_ID=\"nodemcuv2\"" -DLED_BUILTIN=2 -DFLASHMODE_DIO -I/home/mhightow/Arduino/hardware/esp8266com/esp8266/cores/esp8266 -I/home/mhightow/Arduino/hardware/esp8266com/esp8266/variants/nodemcu /tmp/arduino_build_311930/sketch/SrcLoc.ino.cpp -o /dev/null SrcLoc:3:10: fatal error: source_location: No such file or directoryAlternatives for source_location: [] 3 | #include <source_location> | ^~~~~~~~~~~~~~~~~ compilation terminated.

I also noticed that path ... esp8266com/esp8266/tools/sdk/libc/xtensa-lx106-elf/include does not exist on my machine, even after rerunning ./get.py

Not available in C, as it would need default-arg constructed by the caller.
Not available in -std=c++17, part of the -std=c++2a

What I meant was copying the behaviour, at least in structure which is then passed to the func. May be significantly larger in size, on second though :/
i.e. static struct foo { .file = ..., .lineno = ...}; foo.caller = __builtin__(); func(..., &foo);

Plus, we do have a hack around __FILE__ which is not applied to source_location to put strings in progmem as default .rodata is still in ram

cores/esp8266/heap.cpp

mcspr · 2024-09-01T12:19:09Z

cores/esp8266/heap.cpp

+    [[maybe_unused]] const char *file = NULL;
+    [[maybe_unused]] const int line = 0;


Macros below missing 'no-file-line' variants so these can be left out?

should now be fixed.

mcspr · 2024-09-01T12:19:44Z

cores/esp8266/heap.cpp

+//
+#include <bits/c++config.h>
+#include <cstdlib>
+#include "new"


Suggested change

#include "new"

#include <new>

should now be fixed.

mcspr · 2024-09-02T11:14:53Z

cores/esp8266/heap.cpp

+        uint32_t saved_ps = xt_rsil(DEFAULT_CRITICAL_SECTION_INTLEVEL);
+        _umm_last_fail_alloc.addr = caller;
+        _umm_last_fail_alloc.size = size;
+        _umm_last_fail_alloc.file = file;
+        _umm_last_fail_alloc.line = line;
+        print_loc(withinISR(saved_ps), size, file, line, caller);
+        xt_wsr_ps(saved_ps);


Just a refresher, I may be forgetting order of operation rules here...
Since we are between func calls, no need to explicitly put mem barrier to force variable assignment to happen after xt_rsil and before xt_wsr_ps?

Yes, that is my understanding as well.
Also, both macros xt_rsil and xt_wst_ps include a memory barrier, ... : "memory"), so I think we are good.

mcspr · 2024-09-02T11:24:26Z

cores/esp8266/abi.cpp


 extern "C" void __cxa_pure_virtual(void) __attribute__ ((__noreturn__));
 extern "C" void __cxa_deleted_virtual(void) __attribute__ ((__noreturn__));

+#if defined(__cpp_exceptions) && (defined(DEBUG_ESP_OOM) || defined(DEBUG_ESP_PORT) || defined(DEBUG_ESP_WITHINISR))


Shouldn't 'new_handler' be independent of debugging?
since the idea is to allow plain new to sometimes avoid throwing (regardless of whether it builds w/ -fexceptions or without). exception-less builds obviously just call abort for would-be exception
ref new_op.cc and new_opnt.cc, nothrow variant just wraps the other in try catch

Shouldn't 'new_handler' be independent of debugging?

It has been a while since I did this, I think the increased build size for non-debug builds was a consideration.
It would add 136 bytes of IROM to the non-debug build so that it always builds with our revised new_handler.

Will have to recheck too, then. Debug=enabled without exceptions are not handled though?

Plus there are more missing overloads from -std=c++17
https://en.cppreference.com/w/cpp/memory/new/operator_new

Plus there are more missing overloads from -std=c++17

The current master does not support "new" align operations. Compiler reports:
/workdir/repo/gcc-gnu/libstdc++-v3/libsupc++/new_opa.cc:86: undefined reference tomemalign'`

No hits on a quick issue check for memalign.

The library umm_malloc needs an enhancement to support an allocate-aligned option. Also, things will get complicated when incorporating the poison check feature.

I've worked through adding an alignment option to the umm_malloc library. In the process, I have addressed an unreported alignment issue. umm_malloc allocated data addresses always landed on an 8-byte aligned - 4-byte.
Now I need to finish working through all the build options in abi.cpp and heap.cpp.

Cleanup remnants from backport

Fixed postmortem build issue around optional oom count function Made corrections for build macro DEBUG_ESP_WITHINISR

mhightower83 added 5 commits November 17, 2022 11:22

missed items

a716193

Merge branch 'master' into pr-heap-refactor3

6dfd46a

d-a-v added alpha included in alpha release merge-conflict PR has a merge conflict that needs manual correction labels Dec 16, 2022

mhightower83 added 2 commits December 16, 2022 20:08

Merge branch 'master' into pr-heap-refactor3

8b3bb1f

Finish merge cleanuo

ea9befa

mcspr reviewed Dec 17, 2022

View reviewed changes

cores/esp8266/core_esp8266_postmortem.cpp Outdated Show resolved Hide resolved

mhightower83 added 7 commits December 17, 2022 09:42

requested changes

3dd08be

Added missing DRAM fallback to pvPortCallocIram, pvPortZallocIram, and

45f1c3b

pvPortMalloc(,,,true) case. Improved comments.

Add nothrow to aliases

57eb2b3

Merge branch 'master' into pr-heap-refactor3

c2590a7

Merge branch 'pr-heap-refactor3' of github.com:mhightower83/Arduino i…

0619034

…nto pr-heap-refactor3

Merge branch 'master' into pr-heap-refactor3

19da675

Merge branch 'pr-heap-refactor3' of github.com:mhightower83/Arduino i…

af7b962

…nto pr-heap-refactor3

d-a-v removed the merge-conflict PR has a merge conflict that needs manual correction label Dec 19, 2022

mhightower83 added 5 commits December 22, 2022 10:31

Merge branch 'master' into pr-heap-refactor3

9423c8e

Merge branch 'pr-heap-refactor3' of github.com:mhightower83/Arduino i…

471f838

…nto pr-heap-refactor3

Merge branch 'master' into pr-heap-refactor3

d829540

Merge branch 'pr-heap-refactor3' of github.com:mhightower83/Arduino i…

4c96365

…nto pr-heap-refactor3

Merge branch 'master' into pr-heap-refactor3

840e55a

mhightower83 marked this pull request as ready for review April 18, 2023 20:26

Merge branch 'master' into pr-heap-refactor3

95c7e2c

mhightower83 added 3 commits April 28, 2023 06:51

Merge branch 'master' into pr-heap-refactor3

0ce295e

Updated umm_poison move from umm_malloc_cfg.h to umm_malloc_cfgport.h

Use the same format for printing caller, file, line details

4717dcf

in postmortem and printing OOM from Heap wrappers. Fixed crash within postmortem when OOM file is NULL Fixed printing OOM file name when not in ISR.

refactored print_loc()

b083043

removed experimental debug extension as requested added forgotten OOM count to the postmortem report

mhightower83 added 6 commits May 2, 2023 10:55

Merge branch 'master' into pr-heap-refactor3

ecc1b14

Merge branch 'master' into pr-heap-refactor3

a0044d6

Merge branch 'master' into pr-heap-refactor3

01b238a

resolved conflicts with umm_malloc_cfg.h and umm_malloc_cfgport.h

removed unused variable from umm_local.c

b5aa1de

Merge branch 'master' into pr-heap-refactor3

1c99daf

Merge branch 'master' into pr-heap-refactor3

2bbadca

mcspr self-requested a review August 30, 2024 17:10

mcspr reviewed Sep 2, 2024

View reviewed changes

mhightower83 added 3 commits September 3, 2024 12:13

Requested changes

49b48fb

Cleanup remnants from backport

Added missing "new" operators

3786ce6

Fixed postmortem build issue around optional oom count function Made corrections for build macro DEBUG_ESP_WITHINISR

Added missing delete array operators

b9a0e55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Heap: Improve debug caller address reporting #8720

Heap: Improve debug caller address reporting #8720

mhightower83 commented Nov 21, 2022 •

edited

Loading

mhightower83 commented Apr 18, 2023

mcspr commented Apr 28, 2023

mcspr left a comment

mcspr Sep 1, 2024

mhightower83 Sep 3, 2024 •

edited

Loading

mcspr Sep 1, 2024

mhightower83 Sep 7, 2024

mcspr Sep 1, 2024

mhightower83 Sep 3, 2024

mcspr Sep 1, 2024

mhightower83 Sep 3, 2024 •

edited

Loading

mcspr Sep 1, 2024

mhightower83 Sep 3, 2024

mhightower83 Sep 4, 2024

mcspr Sep 4, 2024

mcspr Sep 1, 2024

mhightower83 Sep 3, 2024 •

edited

Loading

mcspr Sep 1, 2024

mhightower83 Sep 3, 2024 •

edited

Loading

mcspr Sep 2, 2024

mhightower83 Sep 3, 2024

mcspr Sep 2, 2024 •

edited

Loading

mhightower83 Sep 3, 2024

mcspr Sep 4, 2024

mhightower83 Sep 7, 2024 •

edited

Loading

mhightower83 Sep 10, 2024

		// The sized deletes are defined in other files.
		#pragma GCC diagnostic ignored "-Wsized-deallocation"

		[[maybe_unused]] const char *file = NULL;
		[[maybe_unused]] const int line = 0;

Heap: Improve debug caller address reporting #8720

Are you sure you want to change the base?

Heap: Improve debug caller address reporting #8720

Conversation

mhightower83 commented Nov 21, 2022 • edited Loading

mhightower83 commented Apr 18, 2023

mcspr commented Apr 28, 2023

mcspr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhightower83 Sep 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhightower83 Sep 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhightower83 Sep 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhightower83 Sep 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcspr Sep 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhightower83 Sep 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhightower83 commented Nov 21, 2022 •

edited

Loading

mhightower83 Sep 3, 2024 •

edited

Loading

mhightower83 Sep 3, 2024 •

edited

Loading

mhightower83 Sep 3, 2024 •

edited

Loading

mhightower83 Sep 3, 2024 •

edited

Loading

mcspr Sep 2, 2024 •

edited

Loading

mhightower83 Sep 7, 2024 •

edited

Loading