Crash with "VerifyError: Expecting a stackmap frame" when loading Android classes

[cypressious]

I'm using the gradle plugin which can be found here and I'm getting the following crash during compilation:

15:54:46.846 [INFO] [system.out] Error! Failed to transform some classes
15:54:46.851 [INFO] [system.out] java.lang.RuntimeException: java.lang.IllegalAccessException: no such method: my.package.android.view.MarkerMapFragment.lambda$init$11(View)void/invokeSpecial
15:54:46.857 [INFO] [system.out]        at net.orfjackal.retrolambda.LambdaReifier.reifyLambdaClass(LambdaReifier.java:42)
15:54:46.862 [INFO] [system.out]        at net.orfjackal.retrolambda.LambdaUsageBackporter$InvokeDynamicInsnConvertingMethodVis
itor.backportLambda(LambdaUsageBackporter.java:165) 
15:54:46.866 [INFO] [system.out]        at net.orfjackal.retrolambda.LambdaUsageBackporter$InvokeDynamicInsnConvertingMethodVis
itor.visitInvokeDynamicInsn(LambdaUsageBackporter.java:154)
15:54:46.870 [INFO] [system.out]        at net.orfjackal.retrolambda.asm.ClassReader.readCode(ClassReader.java:1439)
15:54:46.875 [INFO] [system.out]        at net.orfjackal.retrolambda.asm.ClassReader.readMethod(ClassReader.java:1017)
15:54:46.880 [INFO] [system.out]        at net.orfjackal.retrolambda.asm.ClassReader.accept(ClassReader.java:693)
15:54:46.886 [INFO] [system.out]        at net.orfjackal.retrolambda.asm.ClassReader.accept(ClassReader.java:506)
15:54:46.891 [INFO] [system.out]        at net.orfjackal.retrolambda.LambdaUsageBackporter.transform(LambdaUsageBackporter.java
:22)                                                
15:54:46.896 [INFO] [system.out]        at net.orfjackal.retrolambda.Main$1.transform(Main.java:46)
15:54:46.899 [INFO] [system.out]        at net.orfjackal.retrolambda.BytecodeTransformingFileVisitor.visitFile(BytecodeTransfor
mingFileVisitor.java:25)                            
15:54:46.903 [INFO] [system.out]        at net.orfjackal.retrolambda.BytecodeTransformingFileVisitor.visitFile(BytecodeTransfor
mingFileVisitor.java:11)                            
15:54:46.908 [INFO] [system.out]        at java.nio.file.Files.walkFileTree(Files.java:2667)
15:54:46.914 [INFO] [system.out]        at java.nio.file.Files.walkFileTree(Files.java:2739)
15:54:46.919 [INFO] [system.out]        at net.orfjackal.retrolambda.Main.visitFiles(Main.java:60)
15:54:46.925 [INFO] [system.out]        at net.orfjackal.retrolambda.Main.main(Main.java:43)
15:54:46.929 [INFO] [system.out] Caused by: java.lang.IllegalAccessException: no such method: my.package.android.view.Ma
rkerMapFragment.lambda$init$11(View)void/invokeSpecial
15:54:46.934 [INFO] [system.out]        at java.lang.invoke.MemberName.makeAccessException(MemberName.java:872)
15:54:46.938 [INFO] [system.out]        at java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:993)
15:54:46.943 [INFO] [system.out]        at java.lang.invoke.MethodHandles$Lookup.resolveOrFail(MethodHandles.ja9):137
15:54:46.949 [INFO] [system.out]        at java.lang.invoke.MethodHandles$Lookup.findSpecial(MethodHandles.java:997)
15:54:46.955 [INFO] [system.out]        at net.orfjackal.retrolambda.Types.toMethodHandle(Types.java:42)
15:54:46.961 [INFO] [system.out]        at net.orfjackal.retrolambda.Types.asmToJdkType(Types.java:19)
15:54:46.966 [INFO] [system.out]        at net.orfjackal.retrolambda.LambdaReifier.callBootstrapMethod(LambdaReifier.java:106)
15:54:46.970 [INFO] [system.out]        at net.orfjackal.retrolambda.LambdaReifier.reifyLambdaClass(LambdaReifier.java:37)
15:54:46.974 [INFO] [system.out]        ... 14 more 
15:54:46.978 [INFO] [system.out] Caused by: java.lang.VerifyError: Expecting a stackmap frame at branch target 11
Exception Details:                                  
  Location:
    com/google/android/gms/maps/SupportMapFragment.getMap()Lcom/google/android/gms/maps/GoogleMap; @6: ifnonnull
  Reason:
    Expected stackmap frame at this location.
  Bytecode:
    0000000: 2ab6 0023 4c2b c700 0501 b02b b900 3601
    0000010: 004d a700 0d4e bb00 0c59 2db7 0033 bf2c
    0000020: c700 0501 b02a b400 0ec6 0018 2ab4 000e
    0000030: b600 20b9 0035 0100 2cb9 0035 0100 a500
    0000040: 0f2a bb00 0559 2cb7 001f b500 0e2a b400
    0000050: 0eb0
  Exception Handler Table:
    bci [11, 18] => handler: 21

15:54:46.984 [INFO] [system.out]        at java.lang.invoke.MethodHandleNatives.resolve(Native Method)
15:54:46.990 [INFO] [system.out]        at java.lang.invoke.MemberName$Factory.resolve(MemberName.java:965)
15:54:46.997 [INFO] [system.out]        at java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:990)
15:54:47.003 [INFO] [system.out]        ... 20 more 

It seems that any class that extends Google's SupportMapFragment from the Google Play Services and uses any lambdas will produce this crash.

Here's a minimal app that produces the crash: https://gist.github.com/cypressious/5ccfe84a31fd836fe125

[naixx]

Could a constructor be the cause?

[cypressious]

What exactly about the constructor? As far as I can see, the super class has one zero args constructor. Source is not visible, because the code is proprietary.

[naixx]

I meant that lambda is declared at initialization. But checked that and can't confirm that.

[cypressious]

Now I understand. Yeah, it crashes no matter where the lambda is defined. I originally had it inside a method but put it in the class body to make the example shorter.

[luontola]

I'm unable to build your example project:

C:\DEVEL\Retrolambda\5ccfe84a31fd836fe125>gradle

FAILURE: Build failed with an exception.

* Where:
Build file 'C:\DEVEL\Retrolambda\5ccfe84a31fd836fe125\build.gradle' line: 10

* What went wrong:
A problem occurred evaluating root project '5ccfe84a31fd836fe125'.
> Plugin with id 'com.android.application' not found.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 2.014 secs

Also where can I get the SupportMapFragment.class file? Please attach your file or give an URL to the exactly same version. From the error message it sounds like it's not a fully valid Java class file.

[luontola]

I noticed that the build mentions ProGuard, in which case it could be related to this: http://stackoverflow.com/questions/19928927/obfuscation-causes-verifyerror-expecting-a-stackmap-frame

Try removing Retrolamdba and ProGuard one-by-one from the project and see whether it's possible to execute the file under Java 8.

[cypressious]

Building without ProGuard still doesn't work. I uploaded a complete project you should be able to (not) compile: https://github.com/cypressious/RetrolambdaFailure

[luontola]

Thanks, I'm now getting the same crash.

I've only managed to figure out that the com.google.android.gms.maps.SupportMapFragment#getMap method is missing a stackmap table. But that class is compiled for Java 6, where stackmap tables should be optional. I don't understand why Java 8 anyways requires them.

I'm getting the same crash also when running that class on Java 8 without Retrolamdba, i.e. ./gradlew clean compileDebugJava followed by:

C:\DEVEL\Retrolambda\RetrolambdaFailure\app>java -cp C:\DEVEL\Retrolambda\RetrolambdaFailure\app\build\intermediates\exp
loded-aar\com.google.android.gms\play-services\5.0.77\classes.jar;C:\Users\ORFJackal\AppData\Local\Android\android-studi
o\sdk\extras\android\m2repository\com\android\support\support-v4\19.1.0\support-v4-19.1.0.jar;C:\DEVEL\Retrolambda\Retro
lambdaFailure\app\build\retrolambda\debug;C:\Users\ORFJackal\AppData\Local\Android\android-studio\sdk\platforms\android-
19\android.jar;C:\DEVEL\Retrolambda\RetrolambdaFailure\app\build\retrolambda\debug com.cypressworks.retrolambdafailure.M
arkerMapFragment
Exception in thread "main" java.lang.VerifyError: Expecting a stackmap frame at branch target 11
Exception Details:
  Location:
    com/google/android/gms/maps/SupportMapFragment.getMap()Lcom/google/android/gms/maps/GoogleMap; @6: ifnonnull
  Reason:
    Expected stackmap frame at this location.
  Bytecode:
    0000000: 2ab6 0023 4c2b c700 0501 b02b b900 3601
    0000010: 004d a700 0d4e bb00 0c59 2db7 0033 bf2c
    0000020: c700 0501 b02a b400 0ec6 0018 2ab4 000e
    0000030: b600 20b9 0035 0100 2cb9 0035 0100 a500
    0000040: 0f2a bb00 0559 2cb7 001f b500 0e2a b400
    0000050: 0eb0
  Exception Handler Table:
    bci [11, 18] => handler: 21

        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
        at java.lang.Class.getMethod0(Unknown Source)
        at java.lang.Class.getMethod(Unknown Source)
        at sun.launcher.LauncherHelper.validateMainClass(Unknown Source)
        at sun.launcher.LauncherHelper.checkAndLoadMain(Unknown Source)

Even just this is enough to reproduce the issue:

package com.cypressworks.retrolambdafailure;

import com.google.android.gms.maps.SupportMapFragment;

public class MarkerMapFragment extends SupportMapFragment {

    public static void main(String[] args) {
        new MarkerMapFragment();
    }
}

[luontola]

I can think of three solutions of which only one is realistic:

1. Persuade Google to release Android libraries with the stackmap frames in place. (Maybe they are missing because of them using ProGuard on SupportMapFragment.)
2. Persuade Oracle to make Java allow missing stackmap frames for Java 6 bytecode. (I don't understand why this isn't so.[1])
3. Change gradle-retrolambda to run Retrolambda with the -noverify option. (The -XX:-UseSplitVerifier option doesn't anymore work with Java 8.)

[1] It seems intentional, because the bytecode spec says in §4.7.4 "In a class file whose version number is greater than or equal to 50.0, if a method's Code attribute does not have a StackMapTable attribute, it has an implicit stack map attribute." Version 50.0 means Java 6, so it seems that according to the spec stack map frames have been required since Java 6, but the JVM has been checking them only since Java 7. But in Java 7 release notes it says "Classfiles with version number 51 are exclusively verified using the type-checking verifier, and thus the methods must have StackMapTable attributes when appropriate. For classfiles with version 50, the Hotspot JVM would (and continues to) failover to the type-inferencing verifier if the stackmaps in the file were missing or incorrect. This failover behavior does not occur for classfiles with version 51 (the default version for Java SE 7). Any tool that modifies bytecode in a version 51 classfile must be sure to update the stackmap information to be consistent with the bytecode in order to pass verification." Did they suddenly make Java 8 even stricter about that?

[luontola]

Crap. To answer my rethorical question in the previous comment, this it what it reads in the JVM spec §4.10.1. Verification by Type Checking, the relevant words highlighted by me:

If, and only if, a class file's version number equals 50.0, then if the type checking fails, a Java Virtual Machine implementation may choose to attempt to perform verification by type inference (§4.10.2).

This is a pragmatic adjustment, designed to ease the transition to the new verification discipline. Many tools that manipulate class files may alter the bytecodes of a method in a manner that requires adjustment of the method's stack map frames. If a tool does not make the necessary adjustments to the stack map frames, type checking may fail even though the bytecode is in principle valid (and would consequently verify under the old type inference scheme). To allow implementors time to adapt their tools, Java Virtual Machine implementations may fall back to the older verification discipline, but only for a limited time.

Seems that Java 8 was the limit.

[luontola]

Actually I had a fourth idea of a workaround:

Use Retrolambda's Java agent to add stack map frames to all classes with bytecode version 50.0. It needs to be done only at runtime inside Java 8 VM - the updated classes don't need to be saved.

[luontola]

I'll try to go for the workaround number 4. It's safer than using the -noverify option.

Some help with this would let me do it faster: http://stackoverflow.com/questions/24927993/what-kind-of-java-code-requires-stackmap-frames

[cypressious]

Sadly I know nothing about this kind of stuff, but I upvoted your question.

[luontola]

Phew, finally found out the reason.

Version 5.0.77 of play-services is marked to be Java 7 bytecode (version 51.0), but it's missing the stackmap frames. In other words the bytecode is broken and must be rejected by every JVM that conforms to the spec. Version 4.4.52 of play-services contains Java 6 bytecode (version 50.0) and even though it's missing the stackmap frames, that may be allowed by JVMs.

The example project can be fixed by specifying the version to use:

 dependencies {
-    compile 'com.google.android.gms:play-services:+'
+    compile 'com.google.android.gms:play-services:4.4.+'
     compile 'com.android.support:support-v4:19.1.0'
 }

So this is a bug in play-services and it should be fixed there. As a temporary workaround it might be good to allow configuring custom JVM options in gradle-retrolambda so that users may add the -noverify option when necessary.

[naixx]

Great to hear that! 👍

[vinc3m1]

@orfjackal is there a way to run with -noverify at the moment (using gradle-retrolambda) in case we need to use play services 5+?

[evant]

@vinc3m1 gradle-retrolambda has been updated to support this, check the README for details.

[jaredsburrows]

@orfjackal +1 Thanks for the -noverify.