-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OTel Collector Builder: Go "monorepo" support #9252
Comments
mx-psi
added a commit
that referenced
this issue
May 3, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description When running tests, waiting for `downloadModules()` to fail 3 times when that's expected adds time to the test run. This updates tests to only attempt downloading once. Note: if there's a network failure that could cause `downloadModules()` to fail when it should normally succeed. Also the wording here is `retries` when in actuality it's the number of attempts. I didn't change this to keep the log wording the same, but I can change the wording if that's preferable. <!-- Issue number if applicable --> #### Link to tracking issue this will help for adding tests for #9252 and #9896 <!--Describe what testing was performed and which tests were added.--> #### Testing Tests ran --------- Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>
andrzej-stencel
pushed a commit
to andrzej-stencel/opentelemetry-collector
that referenced
this issue
May 27, 2024
…10035) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description When running tests, waiting for `downloadModules()` to fail 3 times when that's expected adds time to the test run. This updates tests to only attempt downloading once. Note: if there's a network failure that could cause `downloadModules()` to fail when it should normally succeed. Also the wording here is `retries` when in actuality it's the number of attempts. I didn't change this to keep the log wording the same, but I can change the wording if that's preferable. <!-- Issue number if applicable --> #### Link to tracking issue this will help for adding tests for open-telemetry#9252 and open-telemetry#9896 <!--Describe what testing was performed and which tests were added.--> #### Testing Tests ran --------- Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>
mx-psi
added a commit
that referenced
this issue
Aug 22, 2024
A continuation of #9253 and #9631 Description: Adds a `--skip-new-go-module` flag to the OTC builder. This enables users working in an existing go module environment (say, a "monorepo") to update the module they have, vs forcing the use of a new module. With the new support inside an existing Go module, a collector main package can be generated using a go:generate directive. For example, in the directory where I want my collector built, the file generate.go has this line: //go:generate builder --skip-new-go-module --skip-compilation --strict-versioning --config=./build-config.yaml In the same directory, the build-config.yaml describes the collector to build. The builder generates the other files in the same directory. At this point, normal Go workflows can be used to update indirect dependencies. Link to tracking Issue: #9252 Testing: Will add unit tests in the next few days. Documentation: Yes. --------- Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
The only supported way to build an OTel Collector requires generating a "fresh" go.mod file for the main package. This creates a challenging workflow for users of a "monorepo", which is to describe a single go.mod file used to build a large code base. This can also create a security concern; users who are familiar with Go module tooling may expect to update go.mod files using the Go toolchain. However, when they do, the next invocation of the
builder
will wipe out any changes that were made to the go.mod files.Describe the solution you'd like
The proposed solution is to offer a mode in which a new go.mod file is not created. Instead, when this mode is selected, each of the modules that would have been required will be individually updated. There is a concept of the "enclosing module" which is determined by the Go toolchain. When the builder is invoked in this new mode, it will
go get
each module using the version in the build configuration, only when the configured module is newer than the one in the enclosing Go module, with emphasis on use of semantic version comparison to control whether an update is performed.We can imagine two steps, the first already described: with a
--skip-new-go-module
flag the builder willgo get
each component instead of generating a new go.mod file. The second, nice-to-have but not essential feature called for, for users of a monorepo, is to use the configuration of the enclosing module to update the build configuration by editing the yaml and writing a new configuration. Thisbuilder sync
command, named for its simlarity to thego work sync
command, would rewrite the build configuration using the go module version that is found in the enclosing module for every component. That would allow the build configuration to be distributed and re-built outside of the monorepo, using the intended "current" versions that were in effect inside the monorepo.Describe alternatives you've considered
In the case where as user has an OTel Collector build and wants to update one of the indirect dependencies, instead of a simple
go get
command, they can continue with the status quo. They will add areplace
statement to the configuration and re-invoke the builder. This is error-prone for users who are not familiar with the collector build process.The text was updated successfully, but these errors were encountered: