-
Notifications
You must be signed in to change notification settings - Fork 403
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OpenTelemetry C++ SDK build with vcpkg / OTLP-gRPC: 260 symbols MAX_PATH limitation #701
Comments
At this time I think we should capture this detail somewhere in |
@maxgolov how about using vcpkg's manifests feature (a vcpkg.json file describing dependencies)? And optionally a custom registry if there are packages that aren't in the official vcpkg registry (a custom git repo with custom ports)? When using manifests, the packages are built in the vcpkg directory (used via vcpkg.exe integrate install) which could be in e.g. C:\vcpkg to have space for long file paths. (This for example is the path where Azure Hosted Windows Agents have pre-installed vcpkg) |
@meastp - it should indeed work fine for the usual build pipelines, like Azure Hosted agents or GitHub Action runners. Maybe that is the best option. I think what we do with |
@maxgolov If you use I think it would be a better option to either use dependencies as submodules etc. OR use vcpkg with manifests (and custom registries if needed) - not a combination. This is more in line with the documentation of vcpkg, and I think the experience would be smoother also :) |
I think we can find creative ways of handling it without Presently the issue is only happening with gRPC, and gRPC is the main component that pulls so many other deps, that it's easier to handle the build and installation of it on Windows with |
@maxgolov I'm not just talking about how to solve this specific issue, but using vcpkg more in line with how the documentation suggests it be used :) If you want to explore this more, I'm happy to help :) Keep in mind that However, I think that you want to support multiple different build systems, and this might be a least common denominator type of solution. Is this correct? |
This was, unfortunately, breaking some of my other projects. I personally do not do this on my dev box, i.e. prefer having separate build trees / separate snapshots of |
This is NOT OpenTelemetry C++ SDK issue. But rather a build issue with ninja, described here:
ninja-build/ninja#1900
Problem Description
Imagine a project that builds OpenTelemetry C++ SDK as a dependent git submodule in its own build tree. OpenTelemetry C++ SDK on Windows is easy to build using vcpkg. However, there is a file path length limitation of 260 symbols that is coming into play here if you are building with OTLP+gRPC enabled.
The following layout:
C:\agent\worker\MyInstrumentationFrameworkProject\third_party\opentelemetry-cpp\tools\vcpkg\buildtrees\grpc
fails to build miserably while trying to install OTLP-gRPC on Windows with the following error:
ninja: error: Stat(C:/agent/worker/MyInstrumentationFrameworkProject/third_party/opentelemetry-cpp/tools/vcpkg/buildtrees/grpc/src/2948941d75-78cfc6f1b1.clean/src/core/ext/upbdefs-generated/envoy/extensions/filters/network/http_connection_manager/v3/http_connection_manager.upbdefs.c): Filename longer than 260 characters
Since it is rather hard to predict where the source code is going to be located, especially if it gets pulled into another project built somewhere on a build pipeline as a submodule... we do not have much options. We may try to assume that a custom
%TEMP%
variable is set to short path. And we hostvcpkg
build tree inside%TEMP%
rather than hosting it inside thetools/vcpkg
directory? That may work. However, this approach has some security implications. Ideally the full build has to be isolated in a single work tree, without needing any external directories, especially external temporary directories which may be harder to isolate.Immediate recommendation we can give that the origin name of
opentelemetry
working directory MUST BE shorter than260 - 5 - strlen("/tools/vcpkg/buildtrees/grpc/src/2948941d75-78cfc6f1b1.clean/src/core/ext/upbdefs-generated/envoy/extensions/filters/network/http_connection_manager/v3/http_connection_manager.upbdefs.c")
, that is no longer than 71 symbols.In the above example - the path C:/agent/worker/MyInstrumentationFrameworkProject/third_party/opentelemetry-cpp/ is already too long. As it will fail
vcpkg install gRPC:x64-windows
installation steps within the same build tree, because it is exceeding the allowed base path length limit. There could be other issues coming into play later on: as new extensions could be added to gRPC source, there is no such limitation on Unix OS, and filenames could be even longer than the example above.Quick recap - TL;DR
Don't use names that are "too long" as base path for building OpenTelemetry C++ SDK on Windows.
Use something short, like
C:\otel\
- is a good name. And then thevcpkg gRPC
build required for OTLP - passes okay. 😁The text was updated successfully, but these errors were encountered: