idf_as_lib example does not compile for esp32s3 (IDFGH-6453)

[kevin-june]

Environment

• Development Kit: ESP32-S3-DevKitC-1
• Kit version (for WroverKit/PicoKit/DevKitC): v1?
• Module or chip used: ESP32-S3-WROOM-1
• IDF version (run git describe --tags to find it): v5.0-dev-758-g07bfc09d0c
• Build System: idf.py and CMake/Ninja and CMake/Make
• Compiler version (run xtensa-esp32-elf-gcc --version to find it): xtensa-esp32-elf-gcc (crosstool-NG esp-2021r2) 8.4.0
• Operating System: macOS
• (Windows only) environment type: N/A
• Using an IDE?: No - building at the CLI
• Power Supply: USB

Problem Description

The idf_as_lib example does not compile for the s3 chip:

[ 97%] Linking C executable idf_as_lib.elf
CMakeFiles/idf_as_lib.elf.dir/main.c.obj: in function `app_main':
main.c:(.text+0x5d): dangerous relocation: call8: call target out of range: spi_flash_get_chip_size
main.c:(.text+0x9b): dangerous relocation: call8: call target out of range: vTaskDelay
main.c:(.text+0xb1): dangerous relocation: call8: call target out of range: __getreent
main.c:(.text+0xbd): dangerous relocation: call8: call target out of range: esp_restart
collect2: error: ld returned 1 exit status
make[2]: *** [idf_as_lib.elf] Error 1

My understanding of the issue is described below, should you find it useful.

I think that the issue stems from how the IDF's toolchain file indirectly sets compiler flags. The toolchain file adds flags to an IDF variable:

list(APPEND compile_options "-mlongcalls")

which is later used to set compile options. However, the call to __build_set_default_build_specifications() clears compile_options:

function(__build_set_default_build_specifications)
    # ..
    unset(c_compile_options)

This occurs before add_compile_options() or idf_build_set_property(COMPILE_OPTIONS) has been called, so the settings from the toolchain file seem to be lost.

This behavior does not occur with a traditional idf.py build because that approach overrides project() and sets the toolchain file after __build_set_default_build_specifications() is called.

I'm not sure if it is best practice, but it seems to be possible to call add_compile_options() in the toolchain file.

Note that this problem also exists for the esp32 example, but there is no compilation error.

Expected Behavior

Compilation should succeed; compiler flags from the toolchain file should apply to all compiled files.

Actual Behavior

Compilation fails; compiler flags from the toolchain file are not applied to all compiled files.

Steps to reproduce

1. Extend idf_as_lib/CMakeLists.txt to support the esp32s3 chip (see code sample below)
2. mkdir build && cd build
3. cmake .. -DCMAKE_TOOLCHAIN_FILE=$IDF_PATH/tools/cmake/toolchain-esp32s3.cmake -DTARGET=esp32s3 -GNinja
4. make

Code to reproduce this issue

In idf_as_lib/CMakeLists.txt:

if("${TARGET}" STREQUAL "esp32")
  # ..
elseif("${TARGET}" STREQUAL "esp32s3")
    include($ENV{IDF_PATH}/tools/cmake/idf.cmake)
    idf_build_process(esp32s3
                    COMPONENTS esp32s3 freertos esptool_py
                    SDKCONFIG ${CMAKE_CURRENT_LIST_DIR}/sdkconfig
                    BUILD_DIR ${CMAKE_BINARY_DIR})
endif()

# Link the static libraries to the executable
if("${TARGET}" STREQUAL "esp32")
  # ..
elseif("${TARGET}" STREQUAL "esp32s3")
    target_link_libraries(${elf_file} idf::esp32s3 idf::freertos idf::spi_flash)
    idf_build_executable(${elf_file})
    # This is a workaround
    # target_compile_options(${elf_file} PRIVATE "-mlongcalls")
endif()

Debug Logs

When compiled with Make:

[ 97%] Linking C executable idf_as_lib.elf
CMakeFiles/idf_as_lib.elf.dir/main.c.obj: in function `app_main':
main.c:(.text+0x5d): dangerous relocation: call8: call target out of range: spi_flash_get_chip_size
main.c:(.text+0x9b): dangerous relocation: call8: call target out of range: vTaskDelay
main.c:(.text+0xb1): dangerous relocation: call8: call target out of range: __getreent
main.c:(.text+0xbd): dangerous relocation: call8: call target out of range: esp_restart
collect2: error: ld returned 1 exit status
make[2]: *** [idf_as_lib.elf] Error 1
make[1]: *** [CMakeFiles/idf_as_lib.elf.dir/all] Error 2
make: *** [all] Error 2

Ninja fails similarly.

[igrr]

Hi @kevin-june, this issue is probably related to our recent attempt to fix #7507. In 0baf2c4, we changed the way toolchain files set the compiler options, and while doing that we have overlooked idf_as_lib use case. Additionally, idf_as_lib example wasn't tested in our CI properly, so we didn't notice the issue.

While we are working on a fix you can either revert commit 0baf2c4 in your copy of IDF, or check out the commit right before it (961407c).

[atanisoft]

@igrr This issue seems to impact the IDF v4.4 docker image as well, is there an ETA for fixing this issue in the docker container as well?

example: https://github.com/atanisoft/ESP32CommandStation/runs/4667689780?check_suite_focus=true

[igrr]

@atanisoft this seems to be a different issue, at least the commit which introduced this particular problem hasn't been backported to release/v4.4.
I'll try reproducing the issue in 4.4 based on your project.

[atanisoft]

Ok, I'm not 100% certain then what might cause it as I am able to build successfully locally but not with the tagged docker. Perhaps the docker is pulling in master rather than the 4.4 branch?

[igrr]

@atanisoft it looks like you are reusing the build directory between builds with different IDF versions. CMAKE_C_FLAGS and similar variables get cached in build/CMakeCache.txt. So the build running for v5.0 (IDF master branch) sets the CMAKE_C_FLAGS in a way which is not compatible with v4.4 and older releases.
Please try to add a step to your CI script to remove the build directory between builds (and maybe sdkconfig as well, for good measure).

[atanisoft]

it looks like you are reusing the build directory between builds with different IDF versions.

Ahh, you might be right. I adjusted the actions run to use parallel jobs and haven't tested v4.4 since then. I've enabled it now and testing: https://github.com/atanisoft/ESP32CommandStation/runs/4692894859?check_suite_focus=true

[igrr]

This has been fixed by @o-marshmallow in commit cb90544.

[kevin-june]

Thanks! This works for me and seems to pass the -mlongcalls from the toolchain file as expected.