Deadlock with ScalaCheck 1.13.x, scalaz-concurrent, and Scala 2.12.0-RC2 #290
Comments
|
Same result even with a different worker count: deadlocks with w = 1 and 2 |
|
Thread dump? (logical or actual deadlock?) |
|
This throws after 10 second delay at object initialization time: object ResponseSpec extends Properties("Response") {
property("stack safety") = {
import ExecutionContext.Implicits.global
val f = Future(1)
Await.result(f, 10.seconds) == 1
}
}Not sure if it's related or not but seems suspicious. JStack shows the same/similar stack trace as the hang in the original ResponseSpec: |
|
hmm, I don't understand. why is remotely in the stack trace if not in the Cheers, On Oct 29, 2016 6:31 PM, "Michael Pilquist" notifications@github.com
|
|
@viktorklang Only because the test class is in the "remotely" package. Here's a further simplification which shows the same hang in jstack: I think what's happening here is that when we run the tests via ScalaCheck, the properties are being instantiated as part of object initialization of ResponseSpec. However, we have a separate thread calling back in to ResponseSpec from the thread pool, and it looks like that thread spin-waits on ResponseSpec object initialization to complete (or otherwise blocks in a way that's not visible to JVM). So overall, I don't think this is a bug, just an undesirable result of running futures from an object initializer. |
|
One more simplification -- same hang here: object Foo {
import ExecutionContext.Implicits.global
val x = Await.result(Future(1), 1000.seconds)
def main(args: Array[String]) = println(x)
}Dump: |
|
Yep, @mpilquist says it well. See also http://stackoverflow.com/questions/15176199/scala-parallel-collection-in-object-initializer-causes-a-program-to-hang/15176433#15176433. This was true before 2.12. New in 2.12 is that locking behavior is now different when lambdas are involved, as the lambda body is lifted to a member of the enclosing class (and thus an invocation blocks on the object's initialization lock), instead of a method in a separate (anonymous) class. |
|
Adrian, this new behavior sounds plenty risky. Cheers, On Oct 29, 2016 7:57 PM, "Adriaan Moors" notifications@github.com wrote:
|
|
A sympathy comment: I hit the same issue looking at https://issues.scala-lang.org/browse/SI-10007 and you scratch your head looking at thread dumps. |
|
Viktor, it's a consequence of the new lambda encoding, which is also how Java does it. I don't know how we could change this and still use |
|
To make this more concrete: object O {
def f = (x: Int) => x
f(1)
}compiles to: public final class O {
public static Function1<Object, Object> f() {
return O$.MODULE$.f();
}
}
public final class O$ {
public static O$ MODULE$;
public static { new O$(); }
private O$() {
MODULE$ = this;
this.f().apply$mcII$sp(1);
}
public scala.Function1<Object, Object> f() { return x -> x; }
} |
|
I probably over-interpreted it. (I am a contributor to the improved lazy init so I'd of course want it as Cheers, On Oct 30, 2016 2:47 AM, "Adriaan Moors" notifications@github.com wrote:
|
|
So, to summarize: It's not a problem in ScalaCheck per se, but it is exhibited by the fact that the ScalaCheck runner
Is that assessment correct? |
|
The simplest change which would fix this issue (for the end-user, in this case @larsrh) is something like this: - property("stack safety") = {
+ property("stack safety") = P {
import ExecutionContext.Implicits.global
fromFuture(Future(1)).run == 1
}
}
+
+object P {
+ def apply(body: => Boolean): Prop =
+ Prop(_ => Prop.Result(if (body) Prop.True else Prop.False))
+}@larsrh @rickynils Do you think ScalaCheck should support something like this? If |
yep, as far as I understand -- all access indirects through the module field, which is locked during initialization, and re-entry (e.g., by invoking a lambda which is now implemented as a method defined in the module class, and is thus invoked through the module field) will lock (from another thread) |
|
I guess we'll see how much of a problem this will become—it's not going to Are there any chances of getting the improved lazy val SIP retrofitted onto On Sun, Oct 30, 2016 at 8:30 PM, Adriaan Moors notifications@github.com
Cheers, |
IMHO this should be the default behaviour (lazy evaluation), because I assume quite a few people will hit this issue. |
|
Note this works: object ResponseSpec extends Properties("Response") {
def fromFuture[A](f: Future[A])(implicit E: ExecutionContext): Task[A] =
Task.async { cb => f.onComplete {
case Success(a) => cb(\/.right(a))
case Failure(e) => cb(\/.left(e))
}}
property("stack safety") = Ext.suspend {
import ExecutionContext.Implicits.global
fromFuture(Future(1)).run == 1
}
}
object Ext {
def suspend[P <% Prop](p: => P): Prop = Prop { prms => (p: Prop)(prms) }
} |
|
I don't think there's any other way to solve it in ScalaCheck 1.13.x -- the main culprit is the Theoretically, we could change that (and maybe other conversions) to be by-name like: |
|
@mpilquist We could keep that method but make it non-implicit, and add a new by-name implicit. That would maintain binary compatibility and fix the issue I think. |
|
@non oooo good idea! |
|
I'll try it out now, unless you want to? |
|
Cool, I hope it is fixable. On Mon, Oct 31, 2016 at 5:21 PM, Erik Osheim notifications@github.com
Cheers, |
|
So, the simple fix did not work. I tried creating If I could get |
|
@non How about this instead? |
|
@mpilquist Oh, that looks great! Does that pass |
|
Yep, mima passes |
|
|
|
|
Fixes #290 (deadlock under 2.12 during property initialization)
(cherry picked from commit 4dd1a15)
|
FYI: I don't think this problem is inherent to invoke dynamic and lambdas. I'm adding version by @mpilquist to Dotty test suite scala/scala3#2283 |
After trying to port remotely to Scala 2.12.x, I found that some of the tests involving
scala.concurrent.Futureweren't terminating. The most likely cause is a deadlock. The issue doesn't appear in Scala 2.10.x or 2.11.x, and also not when running ScalaCheck properties through ScalaTest. It only manifests when using the plain ScalaCheck runner and 2.12.0-RC2.A small reproduction project, including instructions, can be found here.
As of now I'm unsure whether this is a problem with
scalaz.concurrent.Task,scala.concurrent.Future, or ScalaCheck (or a combination thereof). I'm opening this issue so that others who might have the same problem can find it.The text was updated successfully, but these errors were encountered: