Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Toolchain refactoring #26

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

chancila
Copy link

This is kind of a big change. Key takeaways:

  • Use a single toolchain type, having 3 for 3 different tools is unnecessary, especially since everything is downloaded together
  • Expose the toolchain using a provider (BufToolchainInfo), this is how toolchains typically are used
  • Refactor toolchain download to prepare toolchains for all execution platforms, due to how toolchain lazy loading works the actual toolchain files are not fetched until it's needed, registering a toolchain does not cause all of it's attributes to be evaluated (namely toolchain.toolchain which contains the ToolchainInfo with the BufToolchainInfo)
  • Due to the above, it's possible to run buf rules in different execution platforms for host (ie if using remote execution)

@CLAassistant
Copy link

CLAassistant commented Aug 15, 2022

CLA assistant check
All committers have signed the CLA.

@chancila
Copy link
Author

Are there any tests for validating these rules? there seem to not be

Copy link
Member

@srikrsna-buf srikrsna-buf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @chancila. This is awesome. Thank you for the PR. I definitely like the provider approach. I've added some preliminary comments. I'll add a more detailed review soon.

toolchain_info = platform_common.ToolchainInfo(
cli = ctx.executable.cli,
return platform_common.ToolchainInfo(
buf = BufToolchainInfo(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can also include the old option to avoid breaking changes and mark it deprecated.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am not sure how this would be breaking, the previous api was not public

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is public and is depended on

buf/internal/toolchain.bzl Show resolved Hide resolved
@@ -33,7 +33,7 @@ def _buf_lint_test_impl(ctx):
files_to_include = []
if ctx.file.config != None:
files_to_include.append(ctx.file.config)
return protoc_plugin_test(ctx, proto_infos, ctx.executable._protoc, ctx.toolchains[_TOOLCHAIN].cli, config, files_to_include)
return protoc_plugin_test(ctx, proto_infos, ctx.executable._protoc, ctx.toolchains[_TOOLCHAIN].buf.protoc_lint_tool, config, files_to_include)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buf_breaking_test and buf_dependencies should also be updated in a similar way.

@@ -23,7 +23,7 @@ _DOC = """
For more info please refer to the [`buf_lint_test` section](https://docs.buf.build/build-systems/bazel#buf-lint-test) of the docs.
"""

_TOOLCHAIN = str(Label("//tools/protoc-gen-buf-lint:toolchain_type"))
_TOOLCHAIN = "@rules_buf//buf:toolchain_type"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't using @rules_buf fail when the imported repository is not named that?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the rule repo name should be consistent, it's part of your API essentially...

@@ -1,17 +0,0 @@
# Copyright 2021-2022 Buf Technologies, Inc.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The general approach thats followed is to use a tools directory for the toolchains, so we want to keep this.

Copy link
Author

@chancila chancila Aug 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally the public API should be in buf/
toolchains are part of your public api

do you want to keep these for backwards compatibility sake?

it's also a bit strange because there's no public api for instantiating toolchains so these toolchain types are defacto not useable publicly

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it depends for example rules_java has it defined in a similar way: https://github.com/bazelbuild/rules_java/blob/master/toolchains/

@srikrsna-buf
Copy link
Member

Hey @chancila! If you can sign the CLA, I'll take up #26 (comment) and resolve the merge conflicts to get this merged.

@chancila
Copy link
Author

thanks, signed

@honnix
Copy link

honnix commented Oct 21, 2024

Any plan to ship this? It's gonna help a lot with remote execution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants