Refactor compiler configuration repositories #900

comius · 2022-12-22T07:50:47Z

Add capabilities.bzl to Kotlin compiler repository. Generate kt_kotlinc_options rule from it, that is filter out options that are not available in a specific version.

This change makes a narrow interface between Kotlin compiler and Kotlin rules. It will make it possible to release and use new versions of Kotlin compiler independently from Kotlin rules.

The release of Kotlin compiler should include the new capabilities.bzl file.

The capabilities.bzl files were generated by grepping respective version of Kotlin compiler sources for @Argument annotation (some of them are not documented). The legacy file was generated from Kotlin compiler 1.3.0 additionally removing options that weren't available in src/legacy.

There are some minor differences from previous src/legacy - javac opts are the same for versions.

Works toward: #660

restingbull · 2022-12-22T17:06:25Z

This makes multiple kotlin version interactions more difficult, if not impossible. This is a common usecase for burgeoning mono-repos. It's come up with java numerous times, and I've seen it a few other places.

The repository approach was chosen to allow providers (and common code) to live in the core repo, while versioned code can reference that and reuse as necessary. Flattening it means that only a single version of kotlin can be accessible at a time.

That said, the capabilities work is much better than the opts kludge.

* Convert strings to label with the correct repository. * Convert label to string before passing to register_toolchains * Add labels to select * Stringify Labels to support older bazels

restingbull · 2022-12-22T17:31:12Z

I'm guessing bzlmod hates the idea of multiple-repos in a module. Of course, much of this was to work around the lack of bzlmod... erg.

comius · 2022-12-23T06:14:35Z

bzlmod controls versions "transitively". In the graph of repository/module dependencies you can have several modules depend on Kotlin rules and maybe even Kotlin compiler. bzlmod will select a single version (of rules and compiler) for all of them. Top level module can choose to additional control/override the versions, but not versions that specific modules use.

I'm not sure how would multiple versions of Kotlin play together. Everything is fine until Kotlin rules don't interact / depend on each-other. If any interaction happens, let's say two Kotlin libraries depend on each other, but have a different version of Kotlin rules. The problem would be 2 different versions of providers. Starlark would say that those are not the same provider and prevent those dependencies to be made.

The interaction might be a bit better using 2 different versions of a compiler, providing they use the same conventions. But in general I think it's a bad idea to use 2 versions, either with bzlmod or in a monorepo.

In principle you can't ask bzlmod from Kotlin rules, what version of the compiler you've got and then create from it a second helper repository/extension. capabilities.bzl are close to this though. You could also put version.bzl into the compiler, that would allow you to ask.

Another thing that can happen now is, that the interface for kt_javac_options can change, because the version of the compiler changed. It might be better to keep the attributes fixed and only report a warning if something is unrecognized by the compiler.

comius · 2022-12-23T06:40:26Z

There are two more things to mention.

With bzlmod toplevel module can set 2 different (or multiple) versions of Kotlin compiler for specific users - that's a feature envisioned to break diamond dependency problem.

And also toolchains could be used to support several versions of the compiler at the same time.

restingbull · 2023-01-04T14:39:19Z

There are two more things to mention.

With bzlmod toplevel module can set 2 different (or multiple) versions of Kotlin compiler for specific users - that's a feature envisioned to break diamond dependency problem.

And also toolchains could be used to support several versions of the compiler at the same time.

I had a few minutes to think this over while sitting in SLC airport.... (oi.)

This approach is fine (actually, it's substantially better), with a few tweaks. The bzlmod will actually make multi-version easier, but we don't need to worry about that at this exact moment (the rough sketch is that each legacy version can be an extension).

I have a few notes on the layout...

I've been moving the starlark implementation out of kotlin and into src/main/starlark. The kotlin directory should be considered the public API, while the src/main/starlark the implementation. This is long term push to improve our test coverage and make the packaging more sensible. In practice, let's not move the options implementation to the kotlin directory.
I loathe to give up the rkt_* directories, because it was nice to have a strict separation. However, keeping them seems more trouble that it's worth for the capabilities files. Versioning is hard, let's go shopping, etc, etc.
[nice to have] Can you add a script to scripts that pulls the flags from a kotlinc jar? If not, I think I know how to handle it.

restingbull · 2023-01-04T14:40:09Z

kotlin/internal/opts.javac.bzl

@@ -1,5 +1,19 @@
-load("@dev_io_bazel_rules_kotlin//src/main/starlark/core/options:derive.bzl", "derive")
-load("@dev_io_bazel_rules_kotlin//src/main/starlark/core/options:convert.bzl", "convert")
+# Copyright 2022 The Bazel Authors. All rights reserved.


Move to src/main/starlark and import from there.

restingbull · 2023-01-04T14:41:00Z

src/main/starlark/core/repositories/capabilities_1.4.bzl.com_github_jetbrains_kotlin.bazel

@@ -0,0 +1,125 @@
+# Copyright 2022 The Bazel Authors. All rights reserved.


Move these to a capabilities directory? It's getting crowded in here.

This simplifies the option resolution for head. Multiple kotlin toolchains can be more simply handled via bzlmod.

chrislovecnm · 2023-07-11T14:54:53Z

Any update on this PR?

restingbull · 2023-07-12T21:27:09Z

Implemented in #963

comius changed the title ~~Refactor rkt repostiories~~ Refactor compiler configuration repositories Dec 22, 2022

comius marked this pull request as ready for review December 22, 2022 11:25

comius requested review from cgruber, timpeut and restingbull as code owners December 22, 2022 11:25

This was referenced Dec 22, 2022

Support bzlmod #902

Closed

Release rules_kotlin as a Bazel module #660

Open

comius and others added 2 commits December 22, 2022 16:34

Upgrade dependencies (bazelbuild#901)

0675c01

Bump the Bazel version to Bazel 6 official (bazelbuild#898)

87507da

restingbull self-assigned this Dec 22, 2022

Convert strings to label with the correct repository (bazelbuild#896)

a7e57a4

* Convert strings to label with the correct repository. * Convert label to string before passing to register_toolchains * Add labels to select * Stringify Labels to support older bazels

comius added 7 commits December 23, 2022 07:41

Refactor rkt repostiories

6b3908f

Address buildifier warnings

7ad9766

Fix failures

e053466

Fix stardoc

42e0dbe

Fix mismatches in options.

1f21a2d

Regenerate docs

e6a253b

Buildifier fixes

d90ecd2

restingbull reviewed Jan 4, 2023

View reviewed changes

restingbull added a commit that referenced this pull request Apr 21, 2023

Move to the option strategy presented in #900

57ca6cf

This simplifies the option resolution for head. Multiple kotlin toolchains can be more simply handled via bzlmod.

restingbull added a commit that referenced this pull request Apr 21, 2023

Move to the option strategy presented in #900

7e0d422

This simplifies the option resolution for head. Multiple kotlin toolchains can be more simply handled via bzlmod.

restingbull mentioned this pull request Apr 21, 2023

Move to options capabilities to kotlinc repository #963

Merged

restingbull added a commit that referenced this pull request Apr 23, 2023

Move to the option strategy presented in #900

213f0f7

This simplifies the option resolution for head. Multiple kotlin toolchains can be more simply handled via bzlmod.

restingbull closed this Jul 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor compiler configuration repositories #900

Refactor compiler configuration repositories #900

comius commented Dec 22, 2022 •

edited

Loading

restingbull commented Dec 22, 2022

restingbull commented Dec 22, 2022

comius commented Dec 23, 2022

comius commented Dec 23, 2022

restingbull commented Jan 4, 2023

restingbull Jan 4, 2023

restingbull Jan 4, 2023

chrislovecnm commented Jul 11, 2023

restingbull commented Jul 12, 2023

		@@ -0,0 +1,125 @@
		# Copyright 2022 The Bazel Authors. All rights reserved.

Refactor compiler configuration repositories #900

Refactor compiler configuration repositories #900

Conversation

comius commented Dec 22, 2022 • edited Loading

restingbull commented Dec 22, 2022

restingbull commented Dec 22, 2022

comius commented Dec 23, 2022

comius commented Dec 23, 2022

restingbull commented Jan 4, 2023

restingbull Jan 4, 2023

Choose a reason for hiding this comment

restingbull Jan 4, 2023

Choose a reason for hiding this comment

chrislovecnm commented Jul 11, 2023

restingbull commented Jul 12, 2023

comius commented Dec 22, 2022 •

edited

Loading