Seeed Studio Xiao ESP32C3 cannot package up proper firmware for esp-web-tools usage (ESPTOOL-835)

[mineshaftgap]

Operating System

macos 13.6.6

Esptool Version

v4.8.dev3

Python Version

python3.11

Chip Description

ESP32C3

Device Description

Seeed Studio Xiao ESP32C3

Hardware Configuration

No

How is Esptool Run

Terminal

Full Esptool Command Line that Was Run

/tmp/esptool-4.8.dev3/esptool.py --chip esp32c3 merge_bin \
  -o esp32c3.bin \
  --flash_mode dio \
  --flash_freq 40m \
  --flash_size 4MB \
  0x1000 .pio/build/seeed_xiao_esp32c3/bootloader.bin \
  0x8000 .pio/build/seeed_xiao_esp32c3/partitions.bin \
  0xe000 ~/.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin \
  0x10000 .pio/build/seeed_xiao_esp32c3/firmware.bin

Esptool Output

esptool.py v4.7.0
Flash params set to 0x0220
SHA digest in image updated
Wrote 0x1066a0 bytes to file esp32c3.bin, ready to flash to offset 0x0

More Information

While PlatformIO is able to upload a firmware directly the board when plugged in, trying to package a firmware is not working. I am trying to prepare a firmware to be able to be update with esp-web-tools. Unfortunately no matter what I try, on the console I get:

ESP-ROM:esp32c3-api1-20210207
Build:Feb  7 2021
rst:0x7 (TG0WDT_SYS_RST),boot:0x9 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40053f88
invalid header: 0x434c4557
.... <serious number of times> ...
invalid header: 0x434c4557
invalid header: 0x434c4557

I have also run a few variations of the above command.

Using offset 0x0000 for bootloader.bin since I have seen conflicting information on what it should be:

esptool.py --chip esp32c3 merge_bin \
  -o esp32c3.bin \
  --flash_mode dio \
  --flash_freq 40m \
  --flash_size 4MB \
  0x0000 .pio/build/seeed_xiao_esp32c3/bootloader.bin \
  0x8000 .pio/build/seeed_xiao_esp32c3/partitions.bin \
  0xe000 ~/.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin \
  0x10000 .pio/build/seeed_xiao_esp32c3/firmware.bin

Trying to have all the flash setting use keep:

esptool.py --chip esp32c3 merge_bin \
  -o esp32c3.bin \
  --flash_mode dio \
  --flash_freq keep \
  --flash_size keep \
  0x1000 .pio/build/seeed_xiao_esp32c3/bootloader.bin \
  0x8000 .pio/build/seeed_xiao_esp32c3/partitions.bin \
  0xe000 ~/.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin \
  0x10000 .pio/build/seeed_xiao_esp32c3/firmware.bin

No matter what I try I still get the invalid header.

I appreciate any help you can give, thanks!

Other Steps to Reproduce

No response

I Have Read the Troubleshooting Guide

• I confirm I have read the troubleshooting guide.

[Jason2866]

For C3 the bootloader has to be at 0x0. Use latest dev esptool.py v4.8.2 or v4.8.3 which updates the sha in the image. You are using v4.5.1 which does not support this.
If the bootloader is correct! this command will work:

esptool.py --chip esp32c3 merge_bin \
  -o esp32c3.bin \
  --flash_mode dio \
  --flash_freq 40m \
  --flash_size 4MB \
  0x0000 .pio/build/seeed_xiao_esp32c3/bootloader.bin \
  0x8000 .pio/build/seeed_xiao_esp32c3/partitions.bin \
  0xe000 ~/.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin \
  0x10000 .pio/build/seeed_xiao_esp32c3/firmware.bin

[mineshaftgap]

@Jason2866 Again, thank you for responding. I was already trying with a tag of v4.8.dev3 but just to be sure I downloaded the release of v4.8.dev3 instead of checking it out. Unfortunately what you have suggested still produces the same issue:

curl -L https://github.com/espressif/esptool/archive/refs/tags/v4.8.dev3.tar.gz|tar zxvf - -C /tmp

/tmp/esptool-4.8.dev3/esptool.py --chip esp32c3 merge_bin \
  -o esp32c3.bin \
  --flash_mode dio \
  --flash_freq 40m \
  --flash_size 4MB \
  0x0000 .pio/build/seeed_xiao_esp32c3/bootloader.bin \
  0x8000 .pio/build/seeed_xiao_esp32c3/partitions.bin \
  0xe000 ~/.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin \
  0x10000 .pio/build/seeed_xiao_esp32c3/firmware.bin

output (note version reported as 4.7.0 not v4.8.dev3) :

esptool.py v4.7.0
Flash params set to 0x0220
SHA digest in image updated
Wrote 0x1066a0 bytes to file esp32c3.bin, ready to flash to offset 0x0

Updating description with new info.

[Jason2866]

In this case one of your files is defect or you are using a wrong one.
We are doing just this here https://github.com/arendst/Tasmota/blob/development/pio-tools/post_esp32.py#L251-L275
adding two firmware files (factory and normal firmware)

From an other great project. It just does the merge you need
https://github.com/letscontrolit/ESPEasy/blob/mega/tools/pio/post_esp32.py

[mineshaftgap]

I am quite confused since I am using what PaltformIO is producing as found by running:

jq < .pio/build/seeed_xiao_esp32c3/idedata.json .extra

I get:

{
  "flash_images": [
    {
      "offset": "0x0000",
      "path": ".../.pio/build/seeed_xiao_esp32c3/bootloader.bin"
    },
    {
      "offset": "0x8000",
      "path": ".../.pio/build/seeed_xiao_esp32c3/partitions.bin"
    },
    {
      "offset": "0xe000",
      "path": ".../.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin"
    }
  ],
  "application_offset": "0x10000"
}

[mineshaftgap]

Well I have tried with the .bin files from the Arduino IDE as well to no luck. It also appears that the bootloader.bin, partitions.bin and boot_app0.bin are byte for byte identical to what I was previously using from PlatformIO.

[Jason2866]

Does the C3 you try to flash ever booted successfully?

[mineshaftgap]

Yes whenever I upload directly from PlatformIO or Arduino.

[Jason2866]

Platformio and ArduinoIDE use esptool.py under the hood. The two Platformio Python scripts i have linked merge the files like you do. The merged factory images do work since where provided via WebInstaller and are installed thousand times.
So there has to be a difference overlooking a small detail.

[mineshaftgap]

@Jason2866 I discovered what this was and it was both my user error combined with an over looked test of ESP Web Tools which I will open a ticket for. Thanks again for your help in this matter.

[radimkarnis]

@mineshaftgap I'm glad this got resolved. @Jason2866 thank you very much for your help!