ESP32: Something strange about dram2_seg

[bjoernQ]

When running anything on the second core, data in dram2_seg gets modified

To reproduce this change the multicore.rs example to this:

//! This shows how to spawn a task on the second core.
//! The first core will print the value of a counter which is incremented by the
//! second core.

#![no_std]
#![no_main]

use core::cell::RefCell;

use critical_section::Mutex;
use esp32_hal::{
    clock::ClockControl,
    cpu_control::{CpuControl, Stack},
    peripherals::{Peripherals, TIMG1},
    prelude::*,
    timer::{Timer, Timer0, TimerGroup},
};
use esp_backtrace as _;
use esp_println::println;
use nb::block;

static mut APP_CORE_STACK: Stack<8192> = Stack::new();

#[entry]
fn main() -> ! {
    fill(127);
    check(127);

    let peripherals = Peripherals::take();
    let system = peripherals.SYSTEM.split();
    let clocks = ClockControl::boot_defaults(system.clock_control).freeze();

    let timer_group0 = TimerGroup::new(peripherals.TIMG0, &clocks);
    let mut timer0 = timer_group0.timer0;

    let timer_group1 = TimerGroup::new(peripherals.TIMG1, &clocks);
    let mut timer1 = timer_group1.timer0;

    timer0.start(1u64.secs());
    timer1.start(500u64.millis());

    let counter = Mutex::new(RefCell::new(0));

    let mut cpu_control = CpuControl::new(system.cpu_control);
    let cpu1_fnctn = || {
        cpu1_task(&mut timer1, &counter);
    };
    let _guard = cpu_control
        .start_app_core(unsafe { &mut APP_CORE_STACK }, cpu1_fnctn)
        .unwrap();

    loop {
        block!(timer0.wait()).unwrap();

        let count = critical_section::with(|cs| *counter.borrow_ref(cs));
        println!("Hello World - Core 0! Counter is {}", count);
        check(127);
    }
}

fn cpu1_task(
    timer: &mut Timer<Timer0<TIMG1>>,
    counter: &critical_section::Mutex<RefCell<i32>>,
) -> ! {
    println!("Hello World - Core 1!");
    loop {
        block!(timer.wait()).unwrap();

        critical_section::with(|cs| {
            let new_val = counter.borrow_ref_mut(cs).wrapping_add(1);
            *counter.borrow_ref_mut(cs) = new_val;
        });
    }
}

fn fill(value: u8) {
    for i in 0x3FFE4350..(0x3FFE4350 + 111 * 1024) {
        unsafe {
            (i as *mut u8).write_volatile(value);
        }
    }
}

fn check(value: u8) {
    let mut low = 0;
    let mut high = 0;
    let mut count = 0;
    for i in 0x3FFE4350..(0x3FFE4350 + 111 * 1024) {
        if unsafe { (i as *mut u8).read_volatile() } != value {
            count += 1;

            if low == 0 {
                low = i;
            }

            high = i;
        }
    }
    println!("{count} bytes wrong, first {low:x}, last {high:x}");
}

Output is

0 bytes wrong, first 0, last 0
Hello World - Core 1!
Hello World - Core 0! Counter is 1
3868 bytes wrong, first 3ffe4350, last 3ffe7e03
Hello World - Core 0! Counter is 3
3868 bytes wrong, first 3ffe4350, last 3ffe7e03
Hello World - Core 0! Counter is 5
3868 bytes wrong, first 3ffe4350, last 3ffe7e03
Hello World - Core 0! Counter is 7
3868 bytes wrong, first 3ffe4350, last 3ffe7e03
Hello World - Core 0! Counter is 9
3868 bytes wrong, first 3ffe4350, last 3ffe7e03

Even changing cpu_control.rs to just run this on the second core doesn't make a difference

core::arch::global_asm!(
    "
    .section .rwtext
    .global _core1
    _core1:
    j _core1
    "
);

I guess this is the root cause of esp-rs/esp-wifi-sys#412

[bjoernQ]

The problem in #1016 might be related to this

[bjoernQ]

Ok seems like we need to reserve some RAM for the initial stack of app core.

Looking at https://github.com/espressif/esp-idf/blob/a7d1da94b9a70e9dfb8b48ebba0fdd0da5b2658b/components/esp_rom/esp32/ld/esp32.rom.ld#L1317

We see

PROVIDE ( __stack_app = 0x3ffe7e30 );

That should be the initial stack for core-1.

Our linker script includes this:

  reserved_rom_data_pro  : ORIGIN = 0X3FFE0000, len = 1088
  reserved_rom_data_app  : ORIGIN = 0X3FFE3F20, len = 1072

  dram2_seg              : ORIGIN = 0x3FFE4350, len = 111k /* the rest of DRAM after the rom data segments in the middle */

So dram2_seg includes that

[MabezDev]

That stack size seems quite small - I'm still trying to figure out the length but on S3 they were 8K stacks.

[MabezDev]

Okay, so the stack for each core is:

PROVIDE ( __stack = 0x3ffe3f20 );
PROVIDE ( __stack_app = 0x3ffe7e30 );
PROVIDE ( _stack_sentry = 0x3ffe1320 );
PROVIDE ( _stack_sentry_app = 0x3ffe5230 );

__stack - _stack_sentry which yields 11264 bytes for each core. Which is guess why we're overwriting some stuff by starting the second core :D.

[MabezDev]

I opened #1289, hopefully, this fixes this :).