Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StackOverflowError when using Java code that creates types with LazyRef #21521

Closed
wjoel opened this issue Aug 30, 2024 · 6 comments · Fixed by #21566
Closed

StackOverflowError when using Java code that creates types with LazyRef #21521

wjoel opened this issue Aug 30, 2024 · 6 comments · Fixed by #21566

Comments

@wjoel
Copy link
Contributor

wjoel commented Aug 30, 2024

Compiler version

3.5.0, works on 3.4.2, 3.4.3.

Minimized code

I have unfortunately not been able to minimize this to a self contained example.

//> using scala 3.5.0
//> using dep io.github.jwharm.javagi:gtk:0.10.2

import org.gnome.gtk.{Unit as _, *}

object Main {
  Box.builder()
    .setOrientation(Orientation.VERTICAL)
    .setHalign(Align.START)
    .setValign(Align.BASELINE)
    .se
    .build()
    .asInstanceOf[Box]
}

Note that .se (which does not exist, and should be a compiler error) is intentional, and triggers the stack overflow.
Scala 3.4.2 fails with a reasonable error message:

 7 |  Box.builder()
 8 |    .setOrientation(Orientation.VERTICAL)
 9 |    .setHalign(Align.START)
10 |    .setValign(Align.BASELINE)
11 |    .se
   |  ^
   |  value se is not a member of org.gnome.gtk.Box#Builder[? <: org.gnome.gtk.Box#Builder[?]]#B#B² - did you mean B.ne?
   |
   |  where:    B  is a type in trait Builder² with bounds <: io.github.jwharm.javagi.gobject.Builder³[B]
   |            B² is a type in class Builder⁴ with bounds <: org.gnome.gtk.Widget#Builder⁴[B²]

Output (click arrow to expand)

  Exception while compiling /home/wjoel/dev/scala3-playground/Box.java, /home/wjoel/dev/scala3-21:47:04 [1004/45939]
, /home/wjoel/dev/scala3-playground/Orientable.java, /home/wjoel/dev/scala3-playground/Widget.java

  An unhandled exception was thrown in the compiler.
  Please file a crash report here:
  https://github.com/scala/scala3/issues/new/choose
  For non-enriched exceptions, compile with -Xno-enrich-error-messages.

     while compiling: <no file>
        during phase: parser
                mode: Mode()
     library version: version 2.13.14
    compiler version: version 3.5.0
            settings: -classpath /home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-l
ibrary_3/3.5.0/scala3-library_3-3.5.0.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwha
rm/javagi/gtk/0.10.2/gtk-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala
-library/2.13.14/scala-library-2.13.14.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwh
arm/javagi/gdk/0.10.2/gdk-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/ja
vagi/gsk/0.10.2/gsk-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/g
dkpixbuf/0.10.2/gdkpixbuf-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/ja
vagi/gio/0.10.2/gio-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/p
ango/0.10.2/pango-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/pan
gocairo/0.10.2/pangocairo-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/ca
irobindings/cairo/1.18.3/cairo-1.18.3.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwha
rm/javagi/graphene/0.10.2/graphene-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/
jwharm/javagi/gmodule/0.10.2/gmodule-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/githu
b/jwharm/javagi/gobject/0.10.2/gobject-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/git
hub/jwharm/javagi/harfbuzz/0.10.2/harfbuzz-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io
/github/jwharm/javagi/glib/0.10.2/glib-0.10.2.jar -d /home/wjoel/dev/scala3-playground/.scala-build/scala3-playgroun
d_1a8a7b68eb/classes/main -java-output-version 22 -sourceroot /home/wjoel/dev/scala3-playground

Exception in thread "main" java.lang.StackOverflowError
        at dotty.tools.dotc.typer.ImplicitRunInfo$collectParts$2$.traverse(Implicits.scala:681)
        at dotty.tools.dotc.typer.ImplicitRunInfo$collectParts$2$.apply(Implicits.scala:686)
        at dotty.tools.dotc.typer.ImplicitRunInfo.recur$1(Implicits.scala:762)
        at dotty.tools.dotc.typer.ImplicitRunInfo.iscopeRefs$1(Implicits.scala:706)
        at dotty.tools.dotc.typer.ImplicitRunInfo.addCompanions$1(Implicits.scala:739)
        at dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1$$anonfun$1(Implicits.scala:757)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.mutable.LinkedHashSet.foreach(LinkedHashSet.scala:128)
        at dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1(Implicits.scala:757)
        at dotty.tools.dotc.typer.ImplicitRunInfo.recur$1(Implicits.scala:763)
        at dotty.tools.dotc.typer.ImplicitRunInfo.iscopeRefs$1(Implicits.scala:706)
        at dotty.tools.dotc.typer.ImplicitRunInfo.addCompanions$1(Implicits.scala:739)
        at dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1$$anonfun$1(Implicits.scala:757)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.mutable.LinkedHashSet.foreach(LinkedHashSet.scala:128)
        at dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1(Implicits.scala:757)
        at dotty.tools.dotc.typer.ImplicitRunInfo.recur$1(Implicits.scala:763)
        at dotty.tools.dotc.typer.ImplicitRunInfo.iscopeRefs$1(Implicits.scala:706)
        at dotty.tools.dotc.typer.ImplicitRunInfo.addCompanions$1(Implicits.scala:739)
        at dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1$$anonfun$1(Implicits.scala:757)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.mutable.LinkedHashSet.foreach(LinkedHashSet.scala:128)
        at dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1(Implicits.scala:757)
        at dotty.tools.dotc.typer.ImplicitRunInfo.recur$1(Implicits.scala:763)
        at dotty.tools.dotc.typer.ImplicitRunInfo.iscopeRefs$1(Implicits.scala:706)
        at dotty.tools.dotc.typer.ImplicitRunInfo.addCompanions$1(Implicits.scala:739)
        at dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1$$anonfun$1(Implicits.scala:757)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.mutable.LinkedHashSet.foreach(LinkedHashSet.scala:128)
        at dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1(Implicits.scala:757)
        at dotty.tools.dotc.typer.ImplicitRunInfo.recur$1(Implicits.scala:763)
        at dotty.tools.dotc.typer.ImplicitRunInfo.iscopeRefs$1(Implicits.scala:706)
        at dotty.tools.dotc.typer.ImplicitRunInfo.addCompanions$1(Implicits.scala:739)
        at dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1$$anonfun$1(Implicits.scala:757)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.mutable.LinkedHashSet.foreach(LinkedHashSet.scala:128)
        at dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1(Implicits.scala:757)
        at dotty.tools.dotc.typer.ImplicitRunInfo.recur$1(Implicits.scala:763)
        at dotty.tools.dotc.typer.ImplicitRunInfo.iscopeRefs$1(Implicits.scala:706)
        at dotty.tools.dotc.typer.ImplicitRunInfo.addCompanions$1(Implicits.scala:739)
        at dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1$$anonfun$1(Implicits.scala:757)
... and so on
@wjoel wjoel added itype:bug itype:crash stat:needs triage Every issue needs to have an "area" and "itype" label labels Aug 30, 2024
wjoel added a commit to wjoel/scala3 that referenced this issue Aug 30, 2024
@Gedochao Gedochao added area:typer and removed stat:needs triage Every issue needs to have an "area" and "itype" label labels Sep 2, 2024
@EugeneFlesselle
Copy link
Contributor

Myself and @hamzaremmal were not able to reproduce the issue. In any of the 4 configurations with either Scala 3.5.0 or the latest nightly, and with either Java 17 or Java 22; we get the following error:

-- [E008] Not Found Error: /Users/eugeneflesselle/Documents/dotty/tests/playground/example.scala:11:5 
 7 |  Box.builder()
 8 |    .setOrientation(Orientation.VERTICAL)
 9 |    .setHalign(Align.START)
10 |    .setValign(Align.BASELINE)
11 |    .se
   |  ^
   |value se is not a member of org.gnome.gtk.Box#Builder[? <: org.gnome.gtk.Box#Builder[?]]#B#B².
   |Extension methods were tried, but the search failed with:
   |
   |    Recursion limit exceeded.
   |    Maybe there is an illegal cyclic reference?
   |    If that's not the case, you could also try to increase the stacksize using the -Xss JVM option.
   |    For the unprocessed stack trace, compile with -Xno-decode-stacktraces.
   |    A recurring operation is (inner to outer):
   |    
   |      collectParts of org.gnome.gtk.Widget#Builder[
   |      org.gnome.gtk.Box#Builder[? <: org.gnome.gtk.Box#Builder[?]]#B#B]
   |      collectParts of  <:
   |      org.gnome.gtk.Widget#Builder[
   |        org.gnome.gtk.Box#Builder[? <: org.gnome.gtk.Box#Builder[?]]#B#B]
   |
   |where:    B  is a type in trait Builder² with bounds <: io.github.jwharm.javagi.gobject.Builder³[B]
   |          B² is a type in class Builderwith bounds <: org.gnome.gtk.Widget#Builder⁴[B²]

The stack trace in the original issue points to code using handleRecursive whose goal is precisely to catch stack overflows caused by theses cyclic references. I'm not sure how the unprocessed stacks trace got reported without the -Xno-decode-stacktraces option.

@wjoel
Copy link
Contributor Author

wjoel commented Sep 4, 2024

Thanks for looking into this, @EugeneFlesselle! That is odd. It shouldn't matter, but I'm on Linux, Fedora 40.

$ scala-cli -version
Scala CLI version: 1.5.0
Scala version (default): 3.5.0
$ java -version
openjdk version "22.0.2" 2024-07-16
OpenJDK Runtime Environment Zulu22.32+15-CA (build 22.0.2+9)
OpenJDK 64-Bit Server VM Zulu22.32+15-CA (build 22.0.2+9, mixed mode, sharing)

And with this in reprod-i21521.sh

#!/bin/bash

mkdir i21521
cd i21521
echo '//> using scala 3.5.0
//> using dep io.github.jwharm.javagi:gtk:0.10.2

import org.gnome.gtk.{Unit as _, *}

object Main {
  Box.builder()
    .setOrientation(Orientation.VERTICAL)
    .setHalign(Align.START)
    .setValign(Align.BASELINE)
    .se
    .build()
    .asInstanceOf[Box]
}' > main.scala && scala-cli compile .

I get:

wjoel@apollo:~/dev$ ./reprod-i21521.sh 
Compiling project (Scala 3.5.0, JVM (22))
Error: Encountered a StackOverflowError coming from the compiler. You might need to restart your Bloop build server:
dotty.tools.dotc.typer.ImplicitRunInfo$collectParts$2$.traverse(Implicits.scala:681)
dotty.tools.dotc.typer.ImplicitRunInfo$collectParts$2$.apply(Implicits.scala:686)
dotty.tools.dotc.typer.ImplicitRunInfo.recur$1(Implicits.scala:762)
dotty.tools.dotc.typer.ImplicitRunInfo.iscopeRefs$1(Implicits.scala:706)
dotty.tools.dotc.typer.ImplicitRunInfo.addCompanions$1(Implicits.scala:739)
dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1$$anonfun$1(Implicits.scala:757)
scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
scala.collection.mutable.LinkedHashSet.foreach(LinkedHashSet.scala:128)
dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1(Implicits.scala:757)
dotty.tools.dotc.typer.ImplicitRunInfo.recur$1(Implicits.scala:763)
dotty.tools.dotc.typer.ImplicitRunInfo.iscopeRefs$1(Implicits.scala:706)
dotty.tools.dotc.typer.ImplicitRunInfo.addCompanions$1(Implicits.scala:739)
dotty.tools.dotc.typer.ImplicitRunInfo.collectCompanions$1$$anonfun$1(Implicits.scala:757)
...

Restarting bloop (it wasn't actually started when I ran this the first time, but just in case, I tried again after scala --power bloop exit) does not make any difference.

That being said, there might be a (false) cyclic reference hidden in the LazyRefs. I say false, because it works with other versions of Scala 3, and it works in Java. Does my PR take care of your supposedly cyclic reference and stack overflow being reported?

@EugeneFlesselle
Copy link
Contributor

@wjoel Ah! I was able to reproduce it, thank you for the additional details. Could you try it with bloop disabled completely --server=false ? It prevents the overflow from occurring on my end.

Does my PR take care of your supposedly cyclic reference and stack overflow being reported?

It does, I'll add a review over on the PR directly.

@wjoel
Copy link
Contributor Author

wjoel commented Sep 5, 2024

I still get an overflow, it's just printed differently (no suggestion to restart the bloop server, for example, and with more details):

  Exception while compiling /home/wjoel/dev/i21521/main.scala

  An unhandled exception was thrown in the compiler.
  Please file a crash report here:
  https://github.com/scala/scala3/issues/new/choose
  For non-enriched exceptions, compile with -Xno-enrich-error-messages.

     while compiling: <no file>
        during phase: parser
                mode: Mode()
     library version: version 2.13.14
    compiler version: version 3.5.0
            settings: -classpath /home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.5.0/scala3-library_3-3.5.0.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/gtk/0.10.2/gtk-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.14/scala-library-2.13.14.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/gdk/0.10.2/gdk-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/gsk/0.10.2/gsk-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/gdkpixbuf/0.10.2/gdkpixbuf-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/gio/0.10.2/gio-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/pango/0.10.2/pango-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/pangocairo/0.10.2/pangocairo-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/cairobindings/cairo/1.18.3/cairo-1.18.3.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/graphene/0.10.2/graphene-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/gmodule/0.10.2/gmodule-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/gobject/0.10.2/gobject-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/harfbuzz/0.10.2/harfbuzz-0.10.2.jar:/home/wjoel/.cache/coursier/v1/https/repo1.maven.org/maven2/io/github/jwharm/javagi/glib/0.10.2/glib-0.10.2.jar -d /home/wjoel/dev/i21521/.scala-build/i21521_cef53d4a2d/classes/main -sourceroot /home/wjoel/dev/i21521

Exception in thread "main" java.lang.StackOverflowError
	at dotty.tools.dotc.typer.ImplicitRunInfo$collectParts$2$.traverse(Implicits.scala:681)
	at dotty.tools.dotc.typer.ImplicitRunInfo$collectParts$2$.apply(Implicits.scala:686)
	at dotty.tools.dotc.typer.ImplicitRunInfo.recur$1(Implicits.scala:762)
	at dotty.tools.dotc.typer.ImplicitRunInfo.iscopeRefs$1(Implicits.scala:706)
...

@EugeneFlesselle
Copy link
Contributor

With --server=false appended to reprod-i21521.sh, I consistently get the cyclic reference error message, on MacOS with Scala 3.5.0, and Java 17. @Gedochao Is anyone on another system (still with bloop disabled) able to reproduce the issue ?

@wjoel
Copy link
Contributor Author

wjoel commented Sep 7, 2024

I've bisected this, and the bad commit is 9d88c80, "Drop redundant butNot = Param clause in isAnchor". Undoing that,

modified   compiler/src/dotty/tools/dotc/typer/Implicits.scala
@@ -636,7 +636,7 @@ trait ImplicitRunInfo:
   private def isAnchor(sym: Symbol) =
     sym.isClass && !isExcluded(sym)
     || sym.isOpaqueAlias
-    || sym.is(Deferred)
+    || sym.is(Deferred, butNot = Param)
     || sym.info.isMatchAlias
 
   private def computeIScope(rootTp: Type): OfTypeImplicits =

... I get the same error message as on 3.4.2. So I'm guessing that butNot = Param is in fact not redundant, and the removal causes some LazyRefs to be created where they shouldn't?

EugeneFlesselle added a commit that referenced this issue Sep 14, 2024
This reverts commit 9d88c80. 
Closes #21521

The `ClassTypeParamCreationFlags` include both `TypeParam` and `Deferred`.
In effect, a class type parameter was considered an anchor for implicit search, 
by `sym.is(Deferred)` as a sufficient condition.

For a failing example, one can try asserting:
```scala
|| sym.is(Deferred).ensuring(_ == sym.is(Deferred, butNot = Param))
```
in `ImplicitRunInfo#isAnchor` and a test with `summon[Ordering[Int]]`.

In that example, at least, the flags happen to be set by
`Scala2Unpickler#readDisambiguatedSymbol`:

https://github.com/scala/scala3/blob/614170f4545ea6da8f07e0c4b0f2fdfe01377270/compiler/src/dotty/tools/dotc/core/unpickleScala2/Scala2Unpickler.scala#L560
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants