Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failure to query Azure Sphere compiler due to -B unhandled and due to shell escaping in custom configuration args #8962

Closed
Colengms opened this issue Mar 2, 2022 · 3 comments
Closed

Failure to query Azure Sphere compiler due to -B unhandled and due to shell escaping in custom configuration args #8962

Colengms opened this issue Mar 2, 2022 · 3 comments
Assignees
@Colengms
Labels
bug Feature: Configuration An issue related to configuring the extension or IntelliSense fixed Check the Milestone for the release in which the fix is or will be available. Language Service
Milestone
1.9.3 (insiders3)

Comments

@Colengms
Copy link
Collaborator

Colengms commented Mar 2, 2022

In the process of looking into an IntelliSense issue the Azure Sphere team reported, I noticed some incorrect include paths in the TU.

Log Diagnostics output 
-------- Diagnostics - 3/2/2022, 12:39:40 PM
Version: 1.9.2
Current Configuration:
{
    "name": "Linux",
    "includePath": [
        "${workspaceFolder}/**"
    ],
    "defines": [],
    "compilerPath": "/usr/bin/clang",
    "cStandard": "c11",
    "cppStandard": "c++14",
    "intelliSenseMode": "linux-clang-x64",
    "compilerArgs": [],
    "intelliSenseModeIsExplicit": false,
    "cStandardIsExplicit": false,
    "cppStandardIsExplicit": false,
    "mergeConfigurations": false,
    "compilerPathIsExplicit": false,
    "configurationProvider": "ms-vscode.cmake-tools",
    "browse": {
        "path": [
            "${workspaceFolder}/**"
        ],
        "limitSymbolsToIncludedHeaders": true
    }
}
Custom browse configuration: 
{
    "browsePath": [
        "/home/colen/repos/blink/Blink/HardwareDefinitions/mt3620_rdb/inc",
        "/opt/azurespheresdk/HardwareDefinitions/inc",
        "/opt/azurespheresdk/HardwareDefinitions/inc/hw",
        "/opt/azurespheresdk/Sysroots/11/usr/include",
        "/home/colen/repos/blink/Blink",
        "/home/colen/repos/blink/Blink/out/ARM-Debug/CMakeFiles",
        "/home/colen/repos/blink/Blink/out/ARM-Debug",
        "/home/colen/repos/blink/Blink/out/ARM-Debug/approotBlink/bin"
    ],
    "compilerPath": "/opt/azurespheresdk/Sysroots/11/tools/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-musleabi/arm-poky-linux-musleabi-g++",
    "compilerArgs": []
}
Custom configurations:
[ /home/colen/repos/blink/Blink/main.c ]
{
    "defines": [
        "_POSIX_C_SOURCE"
    ],
    "standard": "c11",
    "includePath": [
        "/home/colen/repos/blink/Blink/HardwareDefinitions/mt3620_rdb/inc",
        "/opt/azurespheresdk/HardwareDefinitions/inc",
        "/opt/azurespheresdk/HardwareDefinitions/inc/hw",
        "/opt/azurespheresdk/Sysroots/12/usr/include"
    ],
    "compilerPath": "/opt/azurespheresdk/Sysroots/12/tools/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-musleabi/arm-poky-linux-musleabi-gcc",
    "compilerArgs": [
        "-B",
        "\"/opt/azurespheresdk/Sysroots/12/tools/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-musleabi\"",
        "-march=armv7ve",
        "-mthumb",
        "-mfpu=neon-vfpv4",
        "-mfloat-abi=hard",
        "-mcpu=cortex-a7",
        "--sysroot=\"/opt/azurespheresdk/Sysroots/12\"",
        "-g",
        "-std=c11",
        "-Wstrict-prototypes",
        "-Wno-pointer-sign",
        "-ggdb",
        "-O0",
        "-fPIC",
        "-ffunction-sections",
        "-fdata-sections",
        "-fno-strict-aliasing",
        "-fno-omit-frame-pointer",
        "-fno-exceptions",
        "-Wall",
        "-Wswitch",
        "-Wempty-body",
        "-Wconversion",
        "-Wreturn-type",
        "-Wparentheses",
        "-Wno-format",
        "-Wuninitialized",
        "-Wunreachable-code",
        "-Wunused-function",
        "-Wunused-value",
        "-Wunused-variable",
        "-Werror=implicit-function-declaration",
        "-fstack-protector-strong"
    ]
}
Translation Unit Mappings:
[ /home/colen/repos/blink/Blink/main.c ]:
    /home/colen/repos/blink/Blink/main.c
Translation Unit Configurations:
[ /home/colen/repos/blink/Blink/main.c ]:
    Process ID: 13587
    Memory Usage: 30 MB
    Includes:
        /home/colen/repos/blink/Blink/HardwareDefinitions/mt3620_rdb/inc
        /opt/azurespheresdk/HardwareDefinitions/inc
        /opt/azurespheresdk/HardwareDefinitions/inc/hw
        /opt/azurespheresdk/Sysroots/12/usr/include
        /usr/include/x86_64-linux-gnu/c++/9
        /usr/include/c++/9
        /usr/local/include
        /usr/lib/llvm-9/lib/clang/9.0.1/include
        /usr/include/x86_64-linux-gnu
        /usr/include
    Defines:
        _POSIX_C_SOURCE
    Standard Version: c11
    IntelliSense Mode: linux-gcc-x64
Total Memory Usage: 30 MB

------- Workspace parsing diagnostics -------
Number of files discovered (not excluded): 16919

Note that there are "x86_64" related include paths in the TU, despite no such paths in the received custom configuration.

When I query the compiler manually (Using the command line that was logged), I get:

colen@COLENSHOMEPC:~/repos/blink/Blink$ "/opt/azurespheresdk/Sysroots/12/tools/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-musleabi/arm-poky-linux-musleabi-gcc" -B -march=armv7ve -mthumb -mfpu=neon-vfpv4 -mfloat-abi=hard -mcpu=cortex-a7 --sysroot="/opt/azurespheresdk/Sysroots/12" -g -std=c11 -Wstrict-prototypes -Wno-pointer-sign -ggdb -O0 -fPIC -ffunction-sections -fdata-sections -fno-strict-aliasing -fno-omit-frame-pointer -fno-exceptions -Wall -Wswitch -Wempty-body -Wconversion -Wreturn-type -Wparentheses -Wno-format -Wuninitialized -Wunreachable-code -Wunused-function -Wunused-value -Wunused-variable -Werror=implicit-function-declaration -fstack-protector-strong  -Wp,-v -E -dM -x c   /dev/null
ignoring nonexistent directory "/opt/azurespheresdk/Sysroots/12/usr/local/include"
ignoring nonexistent directory "/opt/exp23/3.1.9/sysroots/x86_64-pokysdk-linux/usr/arm-poky-linux-musleabi/include"
ignoring nonexistent directory "/opt/exp23/3.1.9/sysroots/x86_64-pokysdk-linux/usr/lib/arm-poky-linux-musleabi/gcc/arm-poky-linux-musleabi/9.3.0/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/azurespheresdk/Sysroots/12/usr/include/
End of search list.
#define __SSP_STRONG__ 3
...

The additional paths in the TU appear to be coming from clang, which is the 'default' detected compiler. It's unclear why those paths would be getting merged into this TU, as they are for an entirely different compiler and architecture.
@Colengms Colengms added Language Service bug Feature: Configuration An issue related to configuring the extension or IntelliSense labels Mar 2, 2022
@Colengms
Copy link
Collaborator Author

Colengms commented Mar 2, 2022

There are actually 2 bugs here. It looks like that CMake is giving us a non-standard -B arg, which that compiler appears to support. The gcc and clang specs indicate this is a single arg (-Bprefix) not an arg pair (-B prefix). But clang is actually rather lenient and supports many similarly defined options also as arg pairs (not sure about gcc), so we can address this by also supporting it as an arg pair.

The second issue is a CMake Tools issue (if it's breaking up the args into an array based on parsing a command line from CMake. It would instead be a CMake issue if it's providing it already in the form of an array of args). It's providing cpptools with the following compiler args:

"\"/opt/azurespheresdk/Sysroots/12/tools/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-musleabi\"",

"--sysroot=\"/opt/azurespheresdk/Sysroots/12\"",

Because these are being provided in an arg array and not a shell command line, they should not contain shell escaping. The quotes in both of these args are instances of shell escaping. Quotes are used in shell command lines to ensure a space does not get misinterpreted as delimiting multiple arguments. But when args are passed via an API as an array of individual arguments, they should not contain shell escaping. The quotes are instead interpreted literally, as if \" were specified via command line).

(We have our own similar issues with this distinction: #8649 ) We need to be diligent about adding/removing shell escaping between uses of command lines vs. arg arrays. Currently, I believe we also have a known issue in which shell escaping will occur when executing processes on Windows (where we build a command line out of the args) but not on Linux (where we invoke an API using an array of args). That may need to be addressed in addition to the CMake Tools issue, in order for this issue to be addressed on both Windows and Linux.

@Colengms Colengms changed the title Default compiler's system include paths incorrectly added to TU Failure to query Azure Sphere compiler due to -B unhandled and due to shell escaping in custom configuration args Mar 2, 2022
@Colengms Colengms mentioned this issue Mar 2, 2022
Closed
@Colengms
Copy link
Collaborator Author

Colengms commented Mar 2, 2022

This is the result of the query when done programmatically, with extra quotes present due to being present in the custom configuration args:

ignoring nonexistent directory ""/opt/azurespheresdk/Sysroots/12"/usr/local/include"
ignoring nonexistent directory "/opt/exp23/3.1.9/sysroots/x86_64-pokysdk-linux/usr/arm-poky-linux-musleabi/include"
ignoring nonexistent directory ""/opt/azurespheresdk/Sysroots/12"/usr/include/"
ignoring nonexistent directory "/opt/exp23/3.1.9/sysroots/x86_64-pokysdk-linux/usr/lib/arm-poky-linux-musleabi/gcc/arm-poky-linux-musleabi/9.3.0/include"
#include "..." search starts here:
#include <...> search starts here:
End of search list.
cc1: error: no include path in which to search for stdc-predef.h

@Colengms Colengms self-assigned this Mar 2, 2022
This was referenced Mar 2, 2022
Closed
Closed
@Colengms Colengms added this to the 1.9.3 (insiders3) milestone Mar 9, 2022
@Colengms Colengms added the fixed Check the Milestone for the release in which the fix is or will be available. label Mar 9, 2022
@Colengms
Copy link
Collaborator Author

Colengms commented Mar 9, 2022

Using this issue to track the mishandling of -B. The quoting issue is being addressed in 1.10 via #8963 .

@github-actions github-actions bot locked and limited conversation to collaborators Apr 24, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@Colengms Colengms

Labels
bug Feature: Configuration An issue related to configuring the extension or IntelliSense fixed Check the Milestone for the release in which the fix is or will be available. Language Service
Projects
None yet
Milestone
1.9.3 (insiders3)
Development

No branches or pull requests
1 participant
@Colengms