-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Lots of Builds in the community-build are silently skipping tests and even test-compiles #14391
Comments
Huh, it's weird that sbt logs an error but doesn't fail the task. If ClassLoaderLayeringStrategy.Flat fixes it then I guess we should do this by default but I wonder when this issue started. Does this ring any bell @griggt ? |
I know for |
I noticed it this weekend but ran out of time to investigate. It's on my todo list for tonight. Thanks for opening the ticket @deusaquilus It's a bit concerning that sbt seems to return a success exit code despite the error. |
Well, I'm not sure what is this about, though in case you need us to make any changes in |
This issue first started with the upgrade to sbt 1.6.1 in #14202 https://github.com/lampepfl/dotty/actions/runs/1657330713 The preceding CI run on master was clean https://github.com/lampepfl/dotty/actions/runs/1656594683 |
Older versions used |
@pshirshov izumi-reflect community-build appears to not be running or even compiling tests. It has the following error:
You can see it here: There are a couple ways to fix this. I fixed it by adding the following setting to SBT build.
|
Thanks, I assume something in 1.6.0 is the culprit here but it's not clear what (sbt/sbt#6480 is the only related PR I could find). @eed3si9n: does sbt publish nightly builds these days? |
The fact that our community build apparently still injects the obsolete sbt-dotty plugin might be related since sbt/sbt#6480 removed support for sbt-dotty: https://github.com/lampepfl/dotty/blob/1776d812cfaec31813d5e0865ebdb8654aaf3de0/project/Build.scala#L1591 |
Any objection to discontinuing this practice and manually fixing up in the dotty-staging forks any projects that have not updated upstream? |
@griggt That'd be awesome, go for it! |
@smarter Works for me. Thanks! |
Some local testing shows the issue also manifests with sbt 1.6.0 but not 1.5.8 (we jumped from 1.5.7 to 1.6.1 in #14202) It also appears that not injecting sbt-dotty when using sbt 1.6.x fixes things (at least for shapeless) Will have more time later tonight to investigate further and PR a fix. It still seems to me that sbt should return a non-zero exit code if the tests aren't being run. |
Sorry I haven't gotten around to resurrecting nightlies. I wonder if this is a side effect of no longer trapping the exit code or waiting on all threads in tests. If so forking might fix it, but it's somewhat controversial - sbt/sbt#6413 |
I have a PR up so it won't silently succeed the build - sbt/sbt#6806 |
The sbt-dotty plugin was deprecated with the release of sbt 1.5.0, and removed from the dotty build in b477c42 However it was still being injected into the community build, as many projects therein had not been updated to a recent sbt, or were using a (possibly outdated) version of sbt-dotty either directly or indirectly via some other plugin. This commit discontinues that practice, since sbt 1.6.x no longer plays nice with sbt-dotty (see the linked ticket scala#14391). Many of the community build projects are already updated upstream to use sbt 1.5.x or 1.6.x without sbt-dotty; those are updated to their upstream versions in this PR as appropriate. A handful of community build projects are either unmaintained or do not offer Scala 3 support upstream, or their inclusion in the community build was part of some special effort. Those projects, listed below, have had their dotty-staging forks updated to use sbt 1.6.2 and drop sbt-dotty. - betterfiles - effpi - fastparse - intent - scalap
The sbt-dotty plugin was deprecated with the release of sbt 1.5.0, and removed from the dotty build in b477c42 However it was still being injected into the community build, as many projects therein had not been updated to a recent sbt, or were using a (possibly outdated) version of sbt-dotty either directly or indirectly via some other plugin. This commit discontinues that practice, since sbt 1.6.x no longer plays nice with sbt-dotty (see the linked ticket scala#14391). Many of the community build projects are already updated upstream to use sbt 1.5.x or 1.6.x without sbt-dotty; those are updated to their upstream versions in this PR as appropriate. A handful of community build projects are either unmaintained or do not offer Scala 3 support upstream, or their inclusion in the community build was part of some special effort. Those projects, listed below, have had their dotty-staging forks updated to use sbt 1.6.2 and drop sbt-dotty. - betterfiles - effpi - fastparse - intent - scalap
The sbt-dotty plugin was deprecated with the release of sbt 1.5.0, and removed from the dotty build in b477c42 However it was still being injected into the community build, as many projects therein had not been updated to a recent sbt, or were using a (possibly outdated) version of sbt-dotty either directly or indirectly via some other plugin. This commit discontinues that practice, since sbt 1.6.x no longer plays nice with sbt-dotty (see the linked ticket scala#14391). Many of the community build projects are already updated upstream to use sbt 1.5.x or 1.6.x without sbt-dotty; those are updated to their upstream versions in this PR as appropriate. A handful of community build projects are either unmaintained or do not offer Scala 3 support upstream, or their inclusion in the community build was part of some special effort. Those projects, listed below, have had their dotty-staging forks updated to use sbt 1.6.2 and drop sbt-dotty. - betterfiles - effpi - fastparse - intent - scalap
It looks like lots of Builds not running or even compiling their tests. When they get to testing this happens:
As similar issues happens for some builds trying to use ZTestFramework:
When this error occurs in a project, none of the tests will be run or even compiled. It is effectively a total disabling of testing code for the given project.
I first saw these in the my quill PR builds:
Some quick grepping (attached) shows me this is happening for the following projects:
in community-build-a
in community-build-c
I solved this problem by adding something slightly different to my settings:
When I added this, the error went away.
This could all be totally wrong and my grepping could be incorrect too but I've tried this with the quill build. I tried running the community build with and actual compile error in my scalatests and without the classLoaderLayeringStrategy parameter. The build passes as though nothing is wrong.
I think that project owners should maybe have a look at their builds and see if this is happening for them.
grepping_results.txt
The text was updated successfully, but these errors were encountered: