Errors across all images (RDT-773)

[kdschlosser]

I have added the following my Github actions

- uses: espressif/esp-idf-ci-action@v1
      with:
        esp_idf_version: v5.0.4
        target: esp32s3

I need to perform multiple builds, so the "command" parameter is completely useless to me and it defaulting to running idf.py build causes problems. I do not call idf.py at all in my build and because I am building several times I should be able to have it work properly without specifying a build command. I just need it to install everything, set the board and run export. Nothing more.

Ubuntu 22.04 image

Run espressif/esp-idf-ci-action@v1
  with:
    esp_idf_version: v5.0.4
    target: esp32s3
    command: idf.py build
  env:
    pythonLocation: /Users/runner/hostedtoolcache/Python/3.10.11/arm64
    PKG_CONFIG_PATH: /Users/runner/hostedtoolcache/Python/3.10.11/arm64/lib/pkgconfig
    Python_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.10.11/arm64
    Python2_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.10.11/arm64
    Python3_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.10.11/arm64
Run export IDF_TARGET=$(echo "esp32s3" | tr '[:upper:]' '[:lower:]' | tr -d '_-')
  export IDF_TARGET=$(echo "esp32s3" | tr '[:upper:]' '[:lower:]' | tr -d '_-')
  docker run -t -e IDF_TARGET="${IDF_TARGET}" -v "${GITHUB_WORKSPACE}:/app/kdschlosser/lvgl_micropython" \
  -w "/app/kdschlosser/lvgl_micropython/" espressif/idf:v5.0.4 \
  /bin/bash -c 'git config --global --add safe.directory "*" && idf.py build'
  shell: /bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    pythonLocation: /Users/runner/hostedtoolcache/Python/3.10.11/arm64
    PKG_CONFIG_PATH: /Users/runner/hostedtoolcache/Python/3.10.11/arm64/lib/pkgconfig
    Python_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.10.11/arm64
    Python2_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.10.11/arm64
    Python3_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.10.11/arm64
/Users/runner/work/_temp/2ac0c02b-13e7-4e61-8725-651d91167d83.sh: line 2: docker: command not found
Error: Process completed with exit code 127.

Windows (latest)

Run espressif/esp-idf-ci-action@v1
Run export IDF_TARGET=$(echo "esp32s3" | tr '[:upper:]' '[:lower:]' | tr -d '_-')
Unable to find image 'espressif/idf:v5.0.4' locally
v5.0.4: Pulling from espressif/idf
docker: no matching manifest for windows/amd64 10.0.20348 in the manifest list entries.
See 'docker run --help'.
Error: Process completed with exit code 125.

macOS (latest)

Run espressif/esp-idf-ci-action@v1
Run export IDF_TARGET=$(echo "esp32s3" | tr '[:upper:]' '[:lower:]' | tr -d '_-')
/Users/runner/work/_temp/2ac0c02b-13e7-4e61-8725-651d91167d83.sh: line 2: docker: command not found
Error: Process completed with exit code 127.

Is only Linux supported? There is nothing that states it is only supported when using the Ubuntu image. I am not sure why it is not working properly for any image.

[kumekay]

Hello @kdschlosser

Do I understand correctly, that you are trying to use this action to install ESP-IDF to the GitHub Action machine on which the workflow runs?

If so, then it's not how this action works. It is a simple wrapper around the official docker image for Linux: https://hub.docker.com/r/espressif/idf. So it expects the project code will be mounted to the docker container and executed in the context of the docker container.

So it only works on a Linux-based 'runs-on' machine and doesn't install anything to the machine itself.

What is the goal of your workflow? Do you want to test the build process of your project on different operating systems?

[liamHowatt]

@kdschlosser you might draw inspiration from this: lvgl/lvgl@ef4b99b

As kumekay said, you can use the IDF docker image in an Action but it's Ubuntu runners only.

[kdschlosser]

What I am saying is This would be handy if it didn't build the project. It should be up to the user to build the project and not the action. It calling idf.py build after it installs is incorrect behavior. What if the user needs to run the idf.py script for things like adding components? or to set some config variables? The assumption is being made that as soon as it installs the user wants to compile their project and that is not always the case.

[kdschlosser]

There is also nothing stating that this only works for an Ubuntu build and not the rest of the OS's that Github actions supports.

[kumekay]

@kdschlosser I understand your point. The primary purpose of this action is to simplify the process of building ESP-IDF projects in CI. It does not install ESP-IDF into the CI environment. While we could create a separate action for that, it falls outside the scope of this action. To help reduce any confusion, I’ve opened a PR to update the description: #46

Installing dependencies for ESP-IDF is platform-dependent, which makes it challenging to generalize. However, it should be possible to write the CI jobs in your specific case. You can refer to the manual installation guide here: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/index.html#manual-installation

[kdschlosser]

The ESP-IDF documentation is not updated and it doesn't install as they say it should especially on Windows. I have managed to get it working for Linux and for macOS but Windows doesn't work right.

Thank you for correcting the description. If I knew that this only worked with Linux builds I would have not spent the time to add it to my CI only to discover it's not going to work. It would have saved me some time. As we know time is one of those things that can never be replaced.

If you would be interested in working jointly to write an action that would get the IDF setup for Linux, macOS and Windows I would be more than happy to help. I have an in depth knowledge of Windows and also how the IDF build system runs. What I am not all that versed on is the GitHub actions CI. Let me know if you are interested is sharing some ideas on this. I would be more than willing to get involved to make it happen.

[kumekay]

@kdschlosser We are working on a new cross-platform installer https://github.com/espressif/idf-im-cli which should make it much simpler task

[kdschlosser]

That would be absolutely fantastic.

Is there really a need to use MinGW? it seems like there is a binary made of the Xtensa compiler for Windows and CMake is already cross platform.

[kdschlosser]

The other issue is the use of DOSKEY in the ESP-IDF scripts. DOSKEY is not persistent between environments.

[kumekay]

Is there really a need to use MinGW? it seems like there is a binary made of the Xtensa compiler for Windows and CMake is already cross platform.

To build our binaries of GCC for Xtensa we need to use some C runtime. For windows, there are generally 2 options available: mingw crt and msvc-crt. So we chose to statically link mingw-crt. Do you experience any issues with that?

[kdschlosser]

it's just another build requirement that doesn't need to be there. It is also using a library that is a sinking ship so to speak. It isn't really being actively developed anymore. There are bug fixes being done but not much more than that. In a year or 2 it could end up causing issues. I know the reason why it was chosen which is it was the easiest and fastest way to get everything up and running. MSVC is a pain to work with I get that. The majority of the pain come from locating the various parts and pieces across the different versions.

I know that the ESP-IDF uses a lot of python in it's build system. I wrote a python library that makes setting up an MSVC build environment a snap to do. Might be something worth taking a look at... it's on the Python Package Index (pypi) and the library is called pyMSVC. It doesn't use the vcvars*.bat files to set up an environment, those have been proven to be unreliable at setting up a proper build environment. The script leverages an undocumented COM interface in Visual Studio Installer and it collects most of the information using that, the rest of the information it gets from the registry. It works for Visual Studio 2010 to current, Visual C++ Build tools from 10.0 to current. Windows SDK's from 6.1 to current. .NET 3.5 to current. F#, CMAKE, Ninja, etc... The list of things it detects is pretty long. It was written for helping when compiling Python extension modules on Windows using setuptools or distutils but I wrote it so it can be used outside of Python as well.