Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Auto-generated __MACOSX folder in tool archive causes core install to fail: "Cannot install platform: searching package root dir: no unique root dir in archive #1790

Closed
3 tasks done
kittaakos opened this issue Jul 2, 2022 · 2 comments · Fixed by #2162
Closed
3 tasks done

Auto-generated __MACOSX folder in tool archive causes core install to fail: "Cannot install platform: searching package root dir: no unique root dir in archive #1790

kittaakos opened this issue Jul 2, 2022 · 2 comments · Fixed by #2162
Assignees
@alessio-perugini
Labels
conclusion: resolved Issue was resolved os: macos Specific to macOS operating system topic: code Related to content of the project itself type: imperfection Perceived defect in any part of project

Comments

@kittaakos
Copy link
Contributor

kittaakos commented Jul 2, 2022 •
edited by per1234
Loading

Describe the problem

I could not install heltec-esp8266:esp8266 with IDE2. It's probably a CLI issue but let's make sure.

Downloading packages
Installing heltec-esp8266:xtensa-lx106-elf-gcc@2.5.0-4-b40a506
heltec-esp8266:xtensa-lx106-elf-gcc@2.5.0-4-b40a506 installed
Installing heltec-esp8266:mkspiffs@2.5.0-4-b40a506
heltec-esp8266:mkspiffs@2.5.0-4-b40a506 installed
Installing heltec-esp8266:mklittlefs@2.5.0-4-69bd9e6
heltec-esp8266:mklittlefs@2.5.0-4-69bd9e6 installed
Installing heltec-esp8266:python3@3.7.2-post1
heltec-esp8266:python3@3.7.2-post1 installed
Installing platform heltec-esp8266:esp8266@0.0.4
Failed to install platform: heltec-esp8266:esp8266.
Error: 13 INTERNAL: Cannot install platform: installing platform heltec-esp8266:esp8266@0.0.4: searching package root dir: no unique root dir in archive, found '/Users/a.kitta/Library/Arduino15/tmp/package-3672416258/__MACOSX' and '/Users/a.kitta/Library/Arduino15/tmp/package-3672416258/esp8266-0.0.4'

To reproduce

I have the following URLs

http://digistump.com/package_digistump_index.json,http://www.leonardomiliani.com/repository/package_leonardomiliani.com_index.json,https://ambasat.com/boards/package_ambasat-1.com_index.json,https://arduino.esp8266.com/stable/package_esp8266com_index.json,https://dl.espressif.com/dl/package_esp32_index.json,https://files.seeedstudio.com/arduino/package_seeeduino_boards_index.json,https://mcudude.github.io/MightyCore/package_MCUdude_MightyCore_index.json,https://rabidprototypes.com/arduino/package_rabidprototypes_index.json,https://raw.githubusercontent.com/Lauszus/Sanguino/master/package_lauszus_sanguino_index.json,https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json,https://resource.heltec.cn/download/package_CubeCell_index.json,https://resource.heltec.cn/download/package_heltec_esp32_index.json,https://resource.heltec.cn/download/package_heltec_esp8266_index.json

Install the platform from the IDE2.

Expected behavior

The archives produced by the macOS built-in archive utility can be used in Arduino boards packages.

Arduino IDE version

HEAD

Operating system

macOS

Operating system version

12.3.1

Additional context

Error log:

daemon INFO {"error":"installing platform heltec-esp8266:esp8266@0.0.4: searching package root dir: no unique root dir in archive, found '/Users/a.kitta/Library/Arduino15/tmp/package-3672416258/__MACOSX' and '/Users/a.kitta/Library/Arduino15/tmp/package-3672416258/esp8266-0.0.4'","level":"error","msg":"Cannot install platform","platform":{"ID":"heltec-esp8266:esp8266","Installed":"0.0.4","Latest":"0.0.4","Name":"Heltec ESP8266 Series Dev-boards"},"time":"2022-07-02T23:04:57+02:00"}

root ERROR Request install failed with error: 13 INTERNAL: Cannot install platform: installing platform heltec-esp8266:esp8266@0.0.4: searching package root dir: no unique root dir in archive, found '/Users/a.kitta/Library/Arduino15/tmp/package-3672416258/__MACOSX' and '/Users/a.kitta/Library/Arduino15/tmp/package-3672416258/esp8266-0.0.4' Error: 13 INTERNAL: Cannot install platform: installing platform heltec-esp8266:esp8266@0.0.4: searching package root dir: no unique root dir in archive, found '/Users/a.kitta/Library/Arduino15/tmp/package-3672416258/__MACOSX' and '/Users/a.kitta/Library/Arduino15/tmp/package-3672416258/esp8266-0.0.4'
    at Object.callErrorFromStatus (/Users/a.kitta/dev/git/arduino-ide/node_modules/@grpc/grpc-js/build/src/call.js:31:26)
    at Object.onReceiveStatus (/Users/a.kitta/dev/git/arduino-ide/node_modules/@grpc/grpc-js/build/src/client.js:349:49)
    at Object.onReceiveStatus (/Users/a.kitta/dev/git/arduino-ide/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181)
    at /Users/a.kitta/dev/git/arduino-ide/node_modules/@grpc/grpc-js/build/src/call-stream.js:187:78
    at processTicksAndRejections (node:internal/process/task_queues:78:11)

I have installed another heltec platform.

Screen Shot 2022-07-02 at 23 12 21

Related

Issue checklist

  • I searched for previous reports in the issue tracker
  • I verified the problem still occurs when using the latest nightly build
  • My report contains all necessary details
@kittaakos kittaakos added type: imperfection Perceived defect in any part of project status: waiting for information More information must be provided before work can proceed labels Jul 2, 2022
@kittaakos kittaakos self-assigned this Jul 2, 2022
@kittaakos kittaakos mentioned this issue Jul 2, 2022
Merged
4 tasks
@per1234 per1234 transferred this issue from arduino/arduino-ide Jul 3, 2022
@per1234
Copy link
Contributor

per1234 commented Jul 3, 2022 •
edited
Loading

This caused by the heltec-esp8266:esp8266@0.0.4 boards platform archive structure violating a limitation imposed by Arduino CLI:

$ arduino-cli version
arduino-cli.exe  Version: git-snapshot Commit: 9b6c9c46 Date: 2022-07-03T15:00:54Z

$ export ARDUINO_BOARD_MANAGER_ADDITIONAL_URLS=https://resource.heltec.cn/download/package_heltec_esp8266_index.json

$ arduino-cli core update-index

[...]

$ arduino-cli.exe core install heltec-esp8266:esp8266@0.0.4
Downloading packages...
heltec-esp8266:xtensa-lx106-elf-gcc@2.5.0-4-b40a506 already downloaded
heltec-esp8266:mkspiffs@2.5.0-4-b40a506 already downloaded
heltec-esp8266:mklittlefs@2.5.0-4-69bd9e6 already downloaded
heltec-esp8266:python3@3.7.2-post1 already downloaded
heltec-esp8266:esp8266@0.0.4 downloaded
Installing heltec-esp8266:xtensa-lx106-elf-gcc@2.5.0-4-b40a506...
heltec-esp8266:xtensa-lx106-elf-gcc@2.5.0-4-b40a506 installed
Installing heltec-esp8266:mkspiffs@2.5.0-4-b40a506...
heltec-esp8266:mkspiffs@2.5.0-4-b40a506 installed
Installing heltec-esp8266:mklittlefs@2.5.0-4-69bd9e6...
heltec-esp8266:mklittlefs@2.5.0-4-69bd9e6 installed
Installing heltec-esp8266:python3@3.7.2-post1...
heltec-esp8266:python3@3.7.2-post1 installed
Installing platform heltec-esp8266:esp8266@0.0.4...
Error during install: Cannot install platform: installing platform heltec-esp8266:esp8266@0.0.4: searching package root dir: no unique root dir in archive, found 'E:\deleteme\arduino-cli\directories\data\tmp\package-2598651179\__MACOSX' and 'E:\deleteme\arduino-cli\directories\data\tmp\package-2598651179\esp8266-0.0.4'

$ wget --output-document="/tmp/esp8266-0.0.4.zip" $(curl $ARDUINO_BOARD_MANAGER_ADDITIONAL_URLS | jq -r .packages[0].platforms[0].url)

[...]

$ unzip -d "/tmp/esp8266-0.0.4" "/tmp/esp8266-0.0.4.zip"

[...]

$ tree -A -F -L 2 "/tmp/esp8266-0.0.4/"
/tmp/esp8266-0.0.4/
├── __MACOSX/
│   └── esp8266-0.0.4/
└── esp8266-0.0.4/
    ├── ISSUE_TEMPLATE.md
    ├── LICENSE
    ├── POLICY.md
    ├── boards.txt
    ├── bootloaders/
    ├── cores/
    ├── keywords.txt
    ├── libraries/
    ├── package.json
    ├── platform.txt
    ├── programmers.txt
    ├── tests/
    ├── tools/
    └── variants/

Note that there are two folders in the root of the platform archive ZIP file. Like this:

esp8266-0.0.4.zip
├── __MACOSX/
└── esp8266-0.0.4/

Arduino CLI allows only a single root folder, which contains the platform:

esp8266-0.0.4.zip
└── esp8266-0.0.4/
    ├── ISSUE_TEMPLATE.md
    ├── LICENSE
    ├── POLICY.md
    ├── boards.txt
    ├── bootloaders/
    ├── cores/
    ├── keywords.txt
    ├── libraries/
    ├── package.json
    ├── platform.txt
    ├── programmers.txt
    ├── tests/
    ├── tools/
    └── variants/

This __MACOSX folder is an annoying quirk of the native macOS archiver. The folder is hidden on macOS, so the users often are not even aware that the structure of the ZIP files they produce has been tampered with.

The same issue previously occurred when installing libraries via arduino-cli lib install --zip-path when the ZIP file had this structure. An exception for the folder in the library archives was added to Arduino CLI (#1228).

Arduino IDE 1.x allows this extra __MACOSX folder in the platform archive.
ⓘ If testing with Arduino IDE 1.x, you may need to use the new package index URL, as specified in the heltec-esp8266:esp8266 platform's installation instructions, as I found that the outdated package index URL mentioned in the report is not compatible with Arduino IDE 1.x ("Error downloading https://resource.heltec.cn/download/package_heltec_esp8266_index.json").

So I think it would be worth adding support to Arduino CLI for this platform archive structure. If not, the package index specification could be made to more clearly describe the required archive structure:

https://arduino.github.io/arduino-cli/dev/package_index_json-specification/#installation-archive-structure

Related

@per1234 per1234 added topic: code Related to content of the project itself and removed status: waiting for information More information must be provided before work can proceed labels Jul 3, 2022
@kittaakos kittaakos removed their assignment Sep 15, 2022
@per1234 per1234 changed the title Cannot install platform: searching package root dir: no unique root dir in archive, found Auto-generated __MACOSX folder in tool archive causes core install to fail: "Cannot install platform: searching package root dir: no unique root dir in archive Mar 29, 2023
@per1234 per1234 added the os: macos Specific to macOS operating system label Mar 29, 2023
@cmaglie
Copy link
Member

cmaglie commented Apr 27, 2023

We may add the exception for __MACOSX and other folder names that seems to appear in the archives artifacts (maybe hidden files with . prefix too?). The exceptions will be added as part of the specification.

@alessio-perugini alessio-perugini linked a pull request Apr 28, 2023 that will close this issue
Ignore __MACOSX folder in tool archives when launching core install #2162
Merged
6 tasks
@per1234 per1234 added the conclusion: resolved Issue was resolved label Nov 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alessio-perugini alessio-perugini

Labels
conclusion: resolved Issue was resolved os: macos Specific to macOS operating system topic: code Related to content of the project itself type: imperfection Perceived defect in any part of project
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Ignore __MACOSX folder in tool archives when launching core install arduino/arduino-cli
4 participants
@cmaglie @kittaakos @alessio-perugini @per1234