KakuteH7 firmware very slow on startup

[sstroobants]

When flashing the KakuteH7 with the latest master build, the FC boots very very slowly (taking up to 20 seconds). The default Betaflight firmware does not have this issue. I've also tested removing a lot of the modules to see if that would change things, but it didn't. Because of this, the ESCs do not get the proper commands on time and thus never finish the startup sequence. Only way now to use it is to power the FC first by usb cable, wait until it has completely booted and only then power the ESCs.

To Reproduce

1. Build the KakuteH7 bootloader (latest master) and flash it using dfu-util.
2. Flash the custom master build using QGroundControl
3. Now the FC takes about 20 seconds to boot, which if using the batteries results in only the first three beeps of the ESCs, but not the two last ones.

Drone:

• Custom drone with the KakuteH7 FC and Tekko32.

[dagar]

@bkueng

[bkueng]

Yes I noticed that as well (#19019). Seems to be something lower-level, and I don't have a jtag setup for the board.

[sstroobants]

Ah I see, I missed that line in that commit. Is there any way I can help looking into this? Or do you have a clue why the ESCs do not start up after this extremely long boot time? Do they go into some kind of standby mode or something when they don't receive the throttle low/high values fast enough? Is there any way to get around that?

[sstroobants]

Hi, I see that in the latest stable release, this is still an issue. Could that be correct @bkueng? So still the ESC's are not initializing after starting up and it takes around +-20 seconds for startup. Let me know if there exists a fix.

[bkueng]

Yes that is correct, it has not been fixed.

[sstroobants]

@bkueng, are there any plans for fixing this in the near future? Is there any way in which I could be of help?

[bkueng]

I probably don't have the time, but you're very welcome to look into this. I'd hook up a debugger to check where in the bootup it hangs.

[sstroobants]

Okay I have attached a black magic probe to the SWD pins of the KakuteH7 and am able to debug using VS Code @bkueng. One issue I have now is that, although I can manually pause, the software does not seem to stop at breakpoint that I set. Do you have any clue why that might be the case? This is making it more difficult for me to find out where it is hanging in the bootup.

[bkueng]

Is this on latest main? In the NuttX source code? Does it work if you set one in a PX4 source file?

[sstroobants]

Yes on latest main and in the NuttX source code. It also doesn't work in a PX4 source file. I have been able to set a breakpoint when I specifically set a hardware breakpoint when I use gdb in the console however, so it seems like it is an issue with how I define the breakpoint in VS Code, maybe I am missing something there?

[bkueng]

Looks like it. I'm not using VS code, so I can't tell.

[dagar]

@sstroobants in vscode did you select the kakuteh7 variant (for cmake configuration)? I use vscode debugging fairly often on various boards, but with a jlink. It could be the blackmagic config within vscode that's not configured properly.

[sstroobants]

Yes I did use the kakuteh7 variant (it was not in the cmake-variants.yaml file, but I added it myself:

holybro_kakuteh7_default:
      short: holybro_kakuteh7
      buildType: MinSizeRel
      settings:
        CONFIG: holybro_kakuteh7_default

I can have another look at the blackmagic config but I am not sure what I could change there to specifically set the hardware breakpoints.

[shupx]

@sstroobants
Did you figure out how to solve this problem? I met the same problem

[vincentpoont2]

@julianoes @bkueng Any idea why there is such a delay? Doesn't happen in betaflight.

VID_20240326_120327_123.mp4.mov

[julianoes]

Is this more than just the 5s of the bootloader? I believe the bootloader USB connected check doesn't work correctly leading to a 5s delay.

[vincentpoont2]

Is this more than just the 5s of the bootloader? I believe the bootloader USB connected check doesn't work correctly leading to a 5s delay.

It seems to take around 20 seconds

[julianoes]

I'll try to reproduce. Best would be to get the output of dmesg after this happens.

[sstroobants]

@sstroobants Did you figure out how to solve this problem? I met the same problem

Hi, no the problem is still there. I wanted to see if I could fix it today but using my KakuteH7 v1.1 is a nightmare with the newer firmware. The latest release won't flash because the flash is too small apparently, and after resizing the firmware by disabling unnecessary modules and rebuilding I have issues with parameters that won't save automatically (looks like they are written to flash by default now? But the flash of the v1.1 is just way too small I suppose..)

I am interested to see if these issues are still there with the newest KakuteH7 V2 or KakuteH7 mini v1.3. But I don't have those to try them out sadly...

[sstroobants]

param_import_fail.txt

for what it's worth; this file gets created at boot. And the shell constantly prints:
ERROR [parameters] param export failed (-27)
INFO [parameters] param auto save unavailable (-27), retrying..

[sstroobants]

param_import_fail.txt

for what it's worth; this file gets created at boot. And the shell constantly prints: ERROR [parameters] param export failed (-27) INFO [parameters] param auto save unavailable (-27), retrying..

This works temporarily by commenting out:
#define FLASH_BASED_PARAMS
in /PX4-Autopilot/boards/holybro/kakuteh7/src/board_config.h

But after a restart the KakuteH7 ends up in a booting loop, for some reason.
EDIT: This happens because I forgot to comment out the $PARAMS_FILE lines in rc.board_defaults.

[sstroobants]

The most annoying thing is that the ESC's don't turn on anymore if they don't receive the command from the FC fast enough. I think that is quite weird actually, and I would be happy with the situation if that gets fixed, and I just have to wait a little bit longer before the whole quad is fully booted.

[sstroobants]

@julianoes @bkueng Any idea why there is such a delay? Doesn't happen in betaflight.
VID_20240326_120327_123.mp4.mov

Interesting that in this example the ESCs only start powering up when the FC is already powered up. Do you know why that might be the case?