Creating a customized framework for the ESP-IDF results in a Python error #1418

MrSurly · 2024-06-06T16:35:53Z

What kind of issue is this?

PlatformIO Core.
If you’ve found a bug, please provide an information below.

Configuration

Operating system: Ubuntu Linux 22.04

PlatformIO Version (platformio --version): PlatformIO Core, version 6.1.15

Description of problem

N.B.: This was originally a forum post, but that got zero traction.

I have a custom framework (based on ESP32 IDF) being pulled from a repository, named esp-idf-ssr. It’s defined in my platformio.ini file (see below). I also have a custom board located in boards/tc.json in the top level of my project.

The tc board does show up for the espressif32 platform when you issue pio boards.

The tc.json file is identical as the esp32dev board, except I added the custom framework.

Problems:

Custom framework with stock board:

When using espidf-ssr framework with esp32dev board:

Error: This board doesn't support espidf-ssr framework!

Custom framework with custom board:

When using espidf-ssr framework with tc board:

Processing TC (platform: espressif32 @ 6.5.0; framework: espidf-ssr; board: tc)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/tc.html
PLATFORM: Espressif 32 (6.5.0) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-espidf-ssr @ 3.50201.0+sha.70c9b04 
 - tool-esptoolpy @ 1.40501.0 (4.5.1) 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
AssertionError: :
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/main.py", line 173:
    env.SConscript("$BUILD_SCRIPT")
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 612:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 279:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/epoulsen/.platformio/platforms/espressif32/builder/main.py", line 346:
    target_elf = env.BuildProgram()
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 61:
    env.ProcessProgramDeps()
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 121:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 347:
    SConscript(env.GetFrameworkScript(name), exports="env")
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/pioplatform.py", line 58:
    assert p.frameworks and framework in p.frameworks
============================================================================== [FAILED] Took 0.15 seconds ==============================================================================

Environment    Status    Duration
-------------  --------  ------------
TC             FAILED    00:00:00.149
========================================================================= 1 failed, 0 succeeded in 00:00:00.149 =========================================================================

Stock framework, but override framework-espidf with custom URL:

platformio.ini snippets:

platform = espressif32 @ 6.5.0
platform_packages = framework-espidf@ <redacted>/esp-idf-ssr.git#v5.2.1-ssr
framework = espidf
board = esp32dev

Here, I’m simply trying to override the existing framework-espidf for use with the esp32dev board. I get:

TypeError: can only concatenate str (not "NoneType") to str:
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/main.py", line 173:
    env.SConscript("$BUILD_SCRIPT")
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 612:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 279:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/epoulsen/.platformio/platforms/espressif32/builder/main.py", line 346:
    target_elf = env.BuildProgram()
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 61:
    env.ProcessProgramDeps()
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 121:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 347:
    SConscript(env.GetFrameworkScript(name), exports="env")
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 676:
    return method(*args, **kw)
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 612:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 279:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/epoulsen/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 1251:
    ensure_python_venv_available()
  File "/home/epoulsen/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 1224:
    venv_dir = get_idf_venv_dir()
  File "/home/epoulsen/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 1178:
    env.subst("$PROJECT_CORE_DIR"), "penv", ".espidf-" + idf_version

How can I use my custom framework? I only added the custom board thinking I needed to do that to avoid the This board doesn't support espidf-ssr framework! error

Update 1

I found what’s causing this (sorta)

In [platform]/espressif32/builder/frameworks/espidf.py function get_idf_venv_dir, the call to get_original_version fails because the version is 3.50201.0+sha.8ef99ab. It has the +sha... tacked onto the end, and get_original_version won’t parse it because it has 3 dots (.) in it:

def get_original_version(version):
    if version.count(".") != 2:
        return None
    _, raw = version.split(".")[:2]
    if int(raw) <= 99:
        return None
    if int(raw) <= 9999:
        return "%s.%s" % (raw[:-2], int(raw[-2:]))
    return "%s.%s.%s" % (raw[:-4], int(raw[-4:-2]), int(raw[-2:]))

Update 2

The version data is ultimately coming from .../packages/framework-espidf/.git/.piopm But I cannot figure out what writes this file. I did a search for the string .piopm in the ~/.platformio directory – got a few hits, but nothing related to what creates this file.

Steps to Reproduce

[Detailed description above]

Actual Results

Crashes with Python assertion

Expected Results

To work as a custom framework

If problems with PlatformIO Build System:

The content of platformio.ini:

[env] ## Defaults
monitor_speed = 115200
platform = espressif32 @ 6.5.0
platform_packages = 
    framework-espidf-ssr @ git@<redacted>/esp-idf-ssr.git#v5.2.1-ssr
framework = espidf-ssr

board = tc

Source file to reproduce issue:

N/A

The text was updated successfully, but these errors were encountered:

Szybet · 2024-06-23T19:16:34Z

I also have this issue ;/

Szybet · 2024-06-24T16:37:47Z

The workaround for me is just to point to a release zip file for example

gojimmypi · 2024-06-30T12:31:01Z

I have a similar issue. Here's my platformio.ini file:

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = espidf
monitor_speed = 115200
 
platform_packages = 
    framework-espidf @ file:///SysGCC/esp32/esp-idf/v5.2

The ESP-IDF in that directory is known to be good, and works with both the idf.py command, as well as from VisualGDB.

When I delete all the related directories in C:\Users\gojimmypi\.platformio\.cache and and C:\Users\gojimmypi\.platformio\packages\framework-espidf to install fresh, this is the error I see:

pio run
Processing esp32dev (platform: espressif32; board: esp32dev; framework: espidf)
------------------------------------------------------------------------------------------------------------------------
Tool Manager: Installing platformio/toolchain-xtensa-esp-elf @ 13.2.0+20230928
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: toolchain-xtensa-esp-elf@13.2.0+20230928 has been installed!
Tool Manager: Installing espressif/tool-xtensa-esp-elf-gdb @ ~11.2.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: tool-xtensa-esp-elf-gdb@11.2.0+20230208 has been installed!
Tool Manager: Installing espressif/tool-riscv32-esp-elf-gdb @ ~11.2.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: tool-riscv32-esp-elf-gdb@11.2.0+20220823 has been installed!
Tool Manager: Installing file:///SysGCC/esp32/esp-idf/v5.2
Tool Manager: framework-espidf@0.0.0+20240630140639 has been installed!
Tool Manager: Resolving dependencies...
AssertionError: Traceback (most recent call last):
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\__main__.py", line 103, in main
    cli()  # pylint: disable=no-value-for-parameter
    ^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\cli.py", line 85, in invoke
    return super().invoke(ctx)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\run\cli.py", line 145, in cli
    process_env(
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\run\cli.py", line 208, in process_env
    ).process()
      ^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\run\processor.py", line 81, in process
    install_project_env_dependencies(
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\commands\install.py", line 132, in install_project_env_dependencies
    _install_project_env_platform(project_env, options),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\commands\install.py", line 149, in _install_project_env_platform
    PlatformPackageManager().install(
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\manager\platform.py", line 62, in install
    p.install_required_packages(force=force)
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\platform\_packages.py", line 76, in install_required_packages
    self.install_package(name, force=force)
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\platform\_packages.py", line 70, in install_package
    return self.pm.install(spec or self.get_package_spec(name), force=force)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\manager\_install.py", line 47, in install
    pkg = self._install(spec, skip_dependencies=skip_dependencies, force=force)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\manager\_install.py", line 130, in _install
    self.install_dependencies(pkg)
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\manager\_install.py", line 143, in install_dependencies
    self.install_dependency(dependency)
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\manager\_install.py", line 155, in install_dependency
    dependency_compatibility = PackageCompatibility.from_dependency(dependency)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\meta.py", line 79, in from_dependency
    assert isinstance(dependency, dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

============================================================

An unexpected error occurred. Further steps:

* Verify that you have the latest version of PlatformIO using
  `python -m pip install -U platformio` command

* Try to find answer in FAQ Troubleshooting section
  https://docs.platformio.org/page/faq/index.html

* Report this problem to the developers
  https://github.com/platformio/platformio-core/issues

============================================================

I also left a comment in #1398 (comment) that has a similar topic, noting that this platformio.ini file in the same project directory compiles successfully:

[env:esp32dev]
platform = espressif32@6.3.0
board = esp32dev
framework = espidf
monitor_speed = 115200

So clearly the problem seems to be with the install using this line:

platform_packages = 
    framework-espidf @ file:///SysGCC/esp32/esp-idf/v5.2

My objective here is to leverage some of the latest features I am working on for wolfSSL in the esp-tls layer as noted in espressif/esp-idf#13966 - specifically those that likely won't be in a published ESP-IDF release in the immediate future.

gojimmypi · 2024-06-30T12:54:15Z

oh, and after posting my prior comment, I see in #1398 (comment) from @Jason2866:

It is not possible to use the espressif IDF repo as source nor a file or directory

Are there no alternatives to installing a specific ESP-IDF version in PlatformIO?

I was looking at the documentation that had this example for the arduino platform for something similar:

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino
platform_packages =
    platformio/framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git

@Jason2866: Perhaps if I create a release on my branch, I can do something like this from @johnboiles?

framework-espidf @ https://github.com/tasmota/esp-idf/releases/download/v4.4.2/esp-idf-v4.4.2.zip

as noted in #889 (comment) ?

Jason2866 · 2024-06-30T13:05:28Z

Yes, and I am the guy responsible for Tasmota related to all framework stuff.
So the link you show is work from me ;-)
If you fork ping me from there and i will help you to get a release done without that you have to reinvent the wheel

Jason2866 · 2024-07-07T12:57:14Z

@MrSurly You can try my PR. You have to add a package.json in your forked IDF branch.

gojimmypi · 2024-07-08T15:21:23Z

For reference a related conversation continued in gojimmypi/esp-idf#1 regarding creating a custom ESP-IDF release.

Resolves platformio#1418

gojimmypi mentioned this issue Jun 30, 2024

Update to ESP-IDF v5.2.2 #1398

Closed

ivankravets transferred this issue from platformio/platformio-core Jul 6, 2024

Jason2866 mentioned this issue Jul 7, 2024

more failsafe IDF version get #1420

Merged

valeros closed this as completed in #1420 Jul 8, 2024

valeros closed this as completed in f77dd2c Jul 8, 2024

jonathandreyer pushed a commit to jonathandreyer/platform-espressif32 that referenced this issue Nov 14, 2024

Better handling of IDF package version (platformio#1420)

fb98123

Resolves platformio#1418

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Creating a customized framework for the ESP-IDF results in a Python error #1418

Creating a customized framework for the ESP-IDF results in a Python error #1418

MrSurly commented Jun 6, 2024

Szybet commented Jun 23, 2024

Szybet commented Jun 24, 2024

gojimmypi commented Jun 30, 2024

gojimmypi commented Jun 30, 2024

Jason2866 commented Jun 30, 2024

Jason2866 commented Jul 7, 2024

gojimmypi commented Jul 8, 2024

Creating a customized framework for the ESP-IDF results in a Python error #1418

Creating a customized framework for the ESP-IDF results in a Python error #1418

Comments

MrSurly commented Jun 6, 2024

Configuration

Description of problem

Problems:

Custom framework with stock board:

Custom framework with custom board:

Stock framework, but override framework-espidf with custom URL:

Update 1

Update 2

Steps to Reproduce

Actual Results

Expected Results

If problems with PlatformIO Build System:

Szybet commented Jun 23, 2024

Szybet commented Jun 24, 2024

gojimmypi commented Jun 30, 2024

gojimmypi commented Jun 30, 2024

Jason2866 commented Jun 30, 2024

Jason2866 commented Jul 7, 2024

gojimmypi commented Jul 8, 2024