ESP32 flash problems with STM32

[CURTLab]

I have some flash problems using a custom STM32 board with an ESP32 on it.

I'm using the Arduino Framework for both and for the moment I use the Arduino IDE (Export compiled Binary) for the ESP32.

What I want to do is to write a USB mass storage to upload the binary (which works except the flashing).

Sometimes the esp_loader_flash_start fails and sometime esp_loader_flash_write with error 7 (ESP_LOADER_ERROR_INVALID_RESPONSE) and FLASH_WRITE_ERR.

Should I comple it with another IDE instead of Arduino?

Maybe my Arduino implementation of the esp-serial-flasher is incorrect.

I would be thankful for every comment or help!

#include "serial_io.h"

#include <Arduino.h>

static unsigned long s_time_end;
static int s_reset_trigger_pin;
static int s_gpio0_trigger_pin;

#include <USBComposite.h>
extern USBCompositeSerial CompositeSerial;

static void serial_debug_print(const uint8_t *data, uint16_t size, bool write) { }

HardwareSerial *s_serial;

esp_loader_error_t loader_port_serial_init(const loader_serial_config_t *config)
{
  s_reset_trigger_pin = (int)config->reset_trigger_pin;
  s_gpio0_trigger_pin = (int)config->gpio0_trigger_pin;

  if (config->uart_port == 1)
    s_serial = &Serial1;
  else if(config->uart_port == 2)
    s_serial = &Serial2;
  else if(config->uart_port == 3)
    s_serial = &Serial3;
  else
    return ESP_LOADER_ERROR_FAIL;
  
  pinMode(s_reset_trigger_pin, OUTPUT);
  pinMode(s_gpio0_trigger_pin, OUTPUT);
  digitalWrite(s_reset_trigger_pin, 1);
  digitalWrite(s_gpio0_trigger_pin, 1);
  
  s_serial->begin(config->baud_rate);
  //Serial1.begin(9600);
  return ESP_LOADER_SUCCESS;
}

esp_loader_error_t loader_port_serial_write(const uint8_t *data, uint16_t size, uint32_t timeout)
{
  serial_debug_print(data, size, true);

  while (s_serial->available()) s_serial->read();
    
  unsigned long timer = millis();
  while(size) {
    if (s_serial->write(*data) != 1)
      return ESP_LOADER_ERROR_FAIL;
    ++data;
    --size;
    if (millis() - timer > timeout)
      return ESP_LOADER_ERROR_TIMEOUT;
  }
  return ESP_LOADER_SUCCESS;
}

esp_loader_error_t loader_port_serial_read(uint8_t *data, uint16_t size, uint32_t timeout)
{
  serial_debug_print(data, size, false);

  uint8_t *buf = data;
  unsigned long timer = millis();
  
  while(size) {
    if (s_serial->available()) {
      *buf = s_serial->read();
      ++buf;
      --size;
    }
    if ((millis() - timer) > timeout)
      return ESP_LOADER_ERROR_TIMEOUT;
  }
  return (size != 0) ? ESP_LOADER_ERROR_FAIL : ESP_LOADER_SUCCESS;
}

// Set GPIO0 LOW, then
// assert reset pin for 50 milliseconds.
void loader_port_enter_bootloader(void)
{
  digitalWrite(s_gpio0_trigger_pin, 0);
  digitalWrite(s_reset_trigger_pin, 0);
  delay(150);
  digitalWrite(s_reset_trigger_pin, 1);
  delay(150);
  digitalWrite(s_gpio0_trigger_pin, 1);
}

void loader_port_reset_target(void)
{
  digitalWrite(s_gpio0_trigger_pin, 1);
  digitalWrite(s_reset_trigger_pin, 0);
  delay(150);
  digitalWrite(s_reset_trigger_pin, 1);
}

void loader_port_delay_ms(uint32_t ms)
{
  delay(ms);
}

void loader_port_start_timer(uint32_t ms)
{
  s_time_end = millis() + ms;
}

uint32_t loader_port_remaining_time(void)
{
  unsigned long remaining = (s_time_end - millis());
  return (remaining > 0) ? (uint32_t)remaining : 0;
}

void loader_port_debug_print(const char *str)
{
  CompositeSerial.print("\033[36m[D] ");
  CompositeSerial.print(str);
  CompositeSerial.println("\033[m");
}

[MaValki]

Hi,
at the first glance, port layer you wrote seems to be alright. The only line I am not entirely sure about is while (s_serial->available()) s_serial->read(); in loader_port_serial_write function what is its purpose ?

I think that it has nothing to do with Arduino IDE. Perhaps, you could somehow share the whole project, as I am not able to reproduce this issue based on provided snippet of code.

Do you get ESP_LOADER_ERROR_INVALID_RESPONSE from esp_loader_flash_start function when it fails as well?

Another option is to try stm32_example from esp-serial-flasher, to check if your setup works.

[CURTLab]

Hi,

while (s_serial->available()) s_serial->read();

cleans the input buffer (I had problems if i don't clean it).

I've made a repo with the full project for you: https://github.com/CURTLab/ESP32FlasherUSBMassStorage

esp_loader_flash_start failed with ESP_LOADER_ERROR_TIMEOUT.
I think the error from esp_loader_flash_start was due to failed attempts of uploading new binaries.

If I flash the ESP using the Arduino IDE and a serial adapter everything works fine. I can even upload the same binary with my project but as soon as I try to upload a binary that is different from the uploaded one I get the ESP_LOADER_ERROR_INVALID_RESPONSE error during esp_loader_flash_write.

[campbell-seer]

I've had a similar issue with an STM32 and it was caused by the STM32 dropping some bytes from UART. The default STM32 HAL code isn't good for receiving bursts of data of an unknown length because you have to start/stop a receive for every byte. There's a lot of time where we can lose data.

I fixed this by configuring the UART to use a circular DMA buffer and then polling CNDTR to determine how many bytes are available. A 128 byte buffer seems fine for communicating with the ESP32, the largest burst of data comes when connecting and is 112 bytes. I just poll the circular buffer in a loop in loader_port_serial_read.

[CURTLab]

Do you have any example code? I have only found one github repo (https://github.com/MaJerle/stm32-usart-uart-dma-rx-tx) but I have no idea how to implement the CNDTR polling to check the mount of available bytes.

[MaValki]

Although, I have not encountered data loss using polling mode so far, @campbell-seer has a point. It is much more robust with DMA. I could re-implement STM32 port layer to use DMA, but you would have to use STM32 HAL library instead of Arduino.

[MaValki]

I was also trying to compile your project in VS code in order to reproduce the issue, however, I was not able download Arduino library via platformIO.

[Hoel]

I could re-implement STM32 port layer to use DMA, but you would have to use STM32 HAL library instead of Arduino.

That would be much appreciated, i use HAL and the platform is STM32F411, which is somewhat slower than F429, currently it cannot achieve a full programming without error, after 10 attempts at various bitrate, always end up with INVALID_CRC or Packet could not be written! Error 2 at one point or another, tried from 57600 to 460800bps. The STM32F411 is set to 96MHz. The target in my case is ESP8285 with 2MB onboard flash, so its the same as ESP8266.
Also for some reason it does not detect the flash size, while the esptool.py script detects it normally.
I also used this same code with ESP32 and got no problem with flash detection but in this case the platform was ESP32 rather than STM32.

[MaValki]

Hi Hoel,
I answered your first proposal in #12.

Indeed, flash detection failed when ESP8266 was used as target. Thank you for pointing that out.
I have just pushed bugfix solving this issue.

[Hoel]

thank you for the promp reply, where is that bugfix? i would like to implment it directly.

EDIT : OK i found it, it works well now.