Support for -Yscala-release flag

[armanbilge]

This is the new flag in Scala 3.1.2-RC1 that enables targeting Scala 3.0 from the Scala 3.1 compiler. One of the exciting things about sbt-typelevel is that we can remotely roll out this feature to several projects :)

I've been thinking about this, and I propose we add a tlScala3Release setting to the typelevel-settings plugin that defaults to 3.0.

@smarter some projects such as fs2 are already being published against Scala 3.1. Is it okay to target them to Scala 3.0 going forward? Otherwise those projects will have to overwrite this setting.

It also seems there's some fiddling required with POM files. @smarter could you clarify what exactly this is? Thanks!

Seems related to:

dependencyOverrides ++= Seq(
  scalaOrganization.value %% "scala3-library" % scalaVersion.value,
  scalaOrganization.value %% "scala3-library_sjs1" % scalaVersion.value // for Scala.js projects
)

https://docs.scala-lang.org/scala3/reference/language-versions/binary-compatibility.html

[smarter]

First I'd like to insist on the fact that -Yscala-release is an experimental feature (hence the use of -Y, the flag will be renamed to -scala-release when it's deemed stable), it shouldn't be used to publish non-experimental artifacts and it certainly shouldn't be used by default in an sbt plugin.

Is it okay to target them to Scala 3.0 going forward?

Not in general as mentioned in typelevel/cats#4016 (comment)
Also, I think it's up to each project to decide what minimal version of Scala they want to support depending on what features/bugfixes they rely on. If you use -Yscala-release 3.0 you should consider actually testing that your APIs are properly usable from 3.0 (e.g., to make sure there's no inference issues).

It also seems there's some fiddling required with POM files. @smarter could you clarify what exactly this is?

I don't have more details than what's in https://docs.scala-lang.org/scala3/reference/language-versions/binary-compatibility.html, right now sbt doesn't know anything about the flag, so when you're using scalaVersion := 3.1.2-RC1, scalacOptions ++= Seq("-Yscala-release", "3.0"), sbt will declare the 3.1.2-RC1 version of scala3-library as a pom dependency even though in practice 3.0 would be a valid dependency. The plan is to fix this in sbt somehow before -scala-release becomes non-experimental.

[armanbilge]

Thanks for the response :)

Not in general as mentioned in typelevel/cats#4016 (comment)

I see. It wasn't obvious to me that using the new compiler with -Yscala-release flag would be essentially the same as running the old compiler, from a binary-compatibility perspective. In any case, MiMa will catch any issues here.

If you use -Yscala-release 3.0 you should consider actually testing that your APIs are properly usable from 3.0 (e.g., to make sure there's no inference issues).

Huh. I see why this is a good idea, but this sounds potentially quite complicated to actually implement in sbt.

[smarter]

Also to reiterate something mentioned in passing in the doc: even once -scala-release will be considered stable, we make no guarantee that we will forever support every single older version of scala 3, we might drop support for older versions to keep a reasonable maintenance and testing burden (e.g. maybe scala 3.5 will only support -scala-release 3.2 and up).

[smarter]

I see. It wasn't obvious to me that using the new compiler with -Yscala-release flag would be essentially the same as running the old compiler, from a binary-compatibility perspective.

It's not exactly the same, if the new code we generate would be compatible with scala 3.0 we can keep generating it under -Yscala-release 3.0, but if we generate code that relies on APIs that only exist in >= 3.1 then we couldn't do that.

[armanbilge]

Also to reiterate something mentioned in passing in the doc: even once -scala-release will be considered stable, we make no guarantee that we will forever support every single older version of scala 3

This makes sense 👍

It's not exactly the same

Right, which is why it wasn't obvious to me if typelevel/cats#4016 (comment) applies to -Yscala-release as well.

[smarter]

Yeah that particular pr I linked to wouldn't cause mima issues I believe, sorry for the confusion.

[armanbilge]

Actually, does -Yscala-release affect bytecode generation at all? scala/scala3#14156 seems like it was all about Tasty.

[smarter]

Right now I don't think it does but it might although this is controversial, see discussion in scala/scala3#14136

[smarter]

Concrete example of why you can't default to -scala-release 3.0 in general: the fix for scala/scala3#11332 is very likely to require a minor version bump (because old compilers wouldn't be able to interpret tasty containing these weird method calls correctly which would break inlining among other things), it seems cats-effect is interested in this feature but it won't be available with an old version passed to -scala-release

[armanbilge]

Sure, there are all sorts of reasons why a project may want to set -Yscala-release to a version greater than 3.0. But if it's a reasonable default for a large majority of projects then it makes sense to set it in sbt-typelevel while making it easy to override.

[armanbilge]

This is available in:
https://github.com/sbt/sbt/releases/tag/v1.7.0-M1

[armanbilge]

Seems the Scala 3 team is going in a different direction, and this flag won't be available in Scala 3.2. So no point to add support here.

https://github.com/scala/scala-lang/blob/1495481107e3ae5f05acf66160bc6bfd8c65e344/blog/_posts/2022-05-30-long-term-compatibility-plans.md