Update to latest HALs, various fixes/cleanup

[jessebraham]

Still needs some more thorough testing, but at least for me this resolved the UART issues with the ESP32-C6 and ESP32-S3. We will need to test the other devices and the USB Serial JTAG too, I suppose, so will leave this as a draft for now.

Closes #37 (I hope 😁)

(Sorry about the noise in the TOML files, had forgotten to disable formatting for them and was too lazy to revert the changes 😅)

[jessebraham]

Did some testing today:

Chip	UART	USB Serial JTAG
ESP32	✅	N/A
ESP32-C2 (26MHz)	❌	N/A
ESP32-C2 (40MHz)	✅	N/A
ESP32-C3	✅	✅
ESP32-C6	✅	✅
ESP32-H2	✅	✅
ESP32-S2	✅	N/A
ESP32-S3	✅	✅

Overall looking much better, but there are still a couple nagging issues to resolve. I will try to look into the ESP32-C2 and ESP32-S2 this week.

[jessebraham]

Now uses the latest published HALs. Also spent some time improving the xtask package a bit, you can now optionally specify an output format (JSON or TOML), and both will be written out if no format is specified. Also removed the xtask package from the workspace, as there were some conflicts (we shouldn't have been mixing build targets in the same workspace, anyway).

[SergioGasquez]

Also spent some time improving the xtask package a bit, you can now optionally specify an output format (JSON or TOML), and both will be written out if no format is specified.

I really like this feature so we don't have to manually covert from JSON to TOML for espflash, thanks for doing that!

[jessebraham]

Okay, at least for the ESP32-C2 the issue was that I was testing on a 26MHz version instead of 40MHz (oops 😁). The stub worked successfully with the 40MHz version.

I don't have a 26MHz ESP32 to test on, but I assume the story is the same there.

[bjoernQ]

To make the xtask work on Windows we need to "fix" the workspace path, e.g. .current_dir(workspace.as_os_str().to_str().unwrap().replace("\\\\?\\", ""))

Also for me the xtask doesn't work when run via cargo run .... I need to build it and run the executable 🤷‍♂️

[bjoernQ]

Seems like the dprint feature broke - the UART cfg is not wrapped in an Option anymore and IO isn't imported

[bjoernQ]

I think S2 also doesn't work for me before this commit

I'd say the problem is the linker script.

Especially

/* Fixups for esp32s2 */
SECTIONS {
  .rwdata_dummy (NOLOAD) : ALIGN(4)
  {
    . = ORIGIN(RWDATA) + SIZEOF(.rwtext) + SIZEOF(.rwtext.wifi);
  } > RWDATA
}
INSERT BEFORE .data;

Here we want to reserve the memory taken by text (in IRAM) in DRAM
BUT actually we don't have anything in .rwtext but our code is in .text

I guess the fact it works for other chips (except C6/H2) is just because we are lucky there

To prove my theory we can replace esp32s2_stub.x with

/*INCLUDE "memory.x"*/


ENTRY(ESP32Reset)


VECTORS_SIZE = 0x400;

RESERVE_CACHES = 32k;

/* Specify main memory areas */
MEMORY
{
  vectors_seg ( RX )     : ORIGIN = 0x40020000 + RESERVE_CACHES, len = VECTORS_SIZE /* SRAM0 */
  iram_seg ( RX )        : ORIGIN = 0x40020000 + RESERVE_CACHES + VECTORS_SIZE, len = 192k - RESERVE_CACHES - VECTORS_SIZE /* SRAM0 */
  dram_seg ( RW )        : ORIGIN = 0x3FFB0000 + 0x4000 + RESERVE_CACHES + VECTORS_SIZE, len = 188k - RESERVE_CACHES - VECTORS_SIZE

  /* RTC fast memory (executable). Persists over deep sleep. Only for core 0 (PRO_CPU) */
  rtc_fast_iram_seg(RWX) : ORIGIN = 0x40070000, len = 8k

  /* RTC fast memory (same block as above), viewed from data bus. Only for core 0 (PRO_CPU) */
  rtc_fast_dram_seg(RW)  : ORIGIN = 0x3ff9e000, len = 8k

  /* RTC slow memory (data accessible). Persists over deep sleep. */
  rtc_slow_seg(RW)       : ORIGIN = 0x50000000, len = 8k
}

/* map generic regions to output sections */
REGION_ALIAS("ROTEXT", iram_seg);
REGION_ALIAS("RWTEXT", iram_seg);
REGION_ALIAS("RODATA", dram_seg);
REGION_ALIAS("RWDATA", dram_seg);

REGION_ALIAS("RTC_FAST_RWTEXT", rtc_fast_iram_seg);
REGION_ALIAS("RTC_FAST_RWDATA", rtc_fast_dram_seg);

/* include linker script from esp-hal */
INCLUDE esp32s2.x
INCLUDE rom-functions.x
INCLUDE hal-defaults.x

and do

❯ cargo +esp build --release --features=esp32s2,dprint --target=xtensa-esp32s2-none-elf -Zbuild-std=core -Zbuild-std-features=panic_immediate_abort
   Compiling flasher-stub v0.1.0 (C:\projects\review\jesse\esp-flasher-stub)
    Finished release [optimized + debuginfo] target(s) in 1.83s

esp-flasher-stub on  fixes/updates [!?] is 📦 v0.1.0 via 🦀 v1.76.0-nightly
❯ espflash flash target\xtensa-esp32s2-none-elf\release\flasher-stub --ram --monitor --no-stub
[2023-12-18T13:16:26Z INFO ] Serial port: 'COM16'
[2023-12-18T13:16:26Z INFO ] Connecting...
[2023-12-18T13:16:27Z WARN ] Setting baud rate higher than 115,200 can cause issues
Chip type:         esp32s2 (revision v0.0)
Crystal frequency: 40MHz
Flash size:        4MB
Features:          WiFi, No Embedded Flash, No Embedded PSRAM, ADC and temperature sensor calibration in BLK2 of efuse V1
MAC address:       7c:df:a1:4a:58:5a
[00:00:00] [========================================]       1/1       0x4002ABE0
[00:00:00] [========================================]       1/1       0x40028000
[00:00:00] [========================================]       2/2       0x40028400
[00:00:00] [========================================]       1/1       0x4002ABD8
[00:00:00] [========================================]       1/1       0x4002ABDC
[00:00:00] [========================================]       1/1       0x3FFBC400
Commands:
    CTRL+R    Reset chip
    CTRL+C    Exit

�OHAI�

tada ... there is the greeting from the stub (and I also see logs on UART1)

UNFORTUNATELY that still doesn't make the stub work in espflash for me ... but at least I see some debug messages on UART1 now

[jessebraham]

Also for me the xtask doesn't work when run via cargo run .... I need to build it and run the executable 🤷‍♂️

The xtask package probably shouldn't have been in the workspace in the first place, as we were mixing build targets. Not actually sure why/how it worked before. Anyway, updating esp-hal resulting in dependency conflicts which I was unable to resolve, so I just removed it from the workspace.

Thanks for the info regarding the S2, can at least try to dig into this when I get some time now 😁

[jessebraham]

I've applied the two fixes suggested by @bjoernQ and the stub is now working! Thanks for that, this should be ready for final review now.

[SergioGasquez]

Suggested change

	In order to generate the JSON stub files for one or more devices, you can again use the `xtask` package:
	In order to generate the JSON and TOML stub files for one or more devices, you can again use the `xtask` package:

[SergioGasquez]

Did another round of testing and everything seems to be working fine:

Chip	UART	USB Serial JTAG
ESP32	✅	N/A
ESP32-C2 (40MHz)	✅	N/A
ESP32-C3	✅	✅
ESP32-C6	✅	✅
ESP32-H2	✅	✅
ESP32-S2	✅	N/A
ESP32-S3	✅	✅