JVM crash while loading a scaled MeshCollisionShape

[stephengold]

Prior discussion: https://hub.jmonkeyengine.org/t/jvm-crash-in-stranded/47354/27

I reproduced the crash on Linux using DebugSp natives and the "pc-screen.gltf" model. Here is a native stack trace from GDB:

#0  0x000070d6e48da4a1 in btQuantizedBvh::reportAabbOverlappingNodex (
    this=0x0, nodeCallback=0x70d6e4dfc5d0, aabbMin=..., aabbMax=...)
    at /home/travis/build/stephengold/Libbulletjme/src/main/native/bullet3/BulletCollision/BroadphaseCollision/btQuantizedBvh.cpp:323
#1  0x000070d6e46dbc49 in btBvhTriangleMeshShape::processAllTriangles (
    this=0x70d654101db0, callback=0x70d6e4dfc6f0, aabbMin=..., aabbMax=...)
    at /home/travis/build/stephengold/Libbulletjme/src/main/native/bullet3/BulletCollision/CollisionShapes/btBvhTriangleMeshShape.cpp:326
#2  0x000070d6e4684cf1 in btTriangleMeshShape::localGetSupportingVertex (
    this=0x70d654101db0, vec=...)
    at /home/travis/build/stephengold/Libbulletjme/src/main/native/bullet3/BulletCollision/CollisionShapes/btTriangleMeshShape.cpp:174
#3  0x000070d6e4684868 in btTriangleMeshShape::recalcLocalAabb (
    this=0x70d654101db0)
    at /home/travis/build/stephengold/Libbulletjme/src/main/native/bullet3/BulletCollision/CollisionShapes/btTriangleMeshShape.cpp:62
#4  0x000070d6e46849e5 in btTriangleMeshShape::setLocalScaling (
    this=0x70d654101db0, scaling=...)
    at /home/travis/build/stephengold/Libbulletjme/src/main/native/bullet3/BulletCollision/CollisionShapes/btTriangleMeshShape.cpp:115
#5  0x000070d6e46dbd17 in btBvhTriangleMeshShape::setLocalScaling (
    this=0x70d654101db0, scaling=...)
    at /home/travis/build/stephengold/Libbulletjme/src/main/native/bullet3/BulletCollision/CollisionShapes/btBvhTriangleMeshShape.cpp:335
#6  0x000070d6e4829936 in Java_com_jme3_bullet_collision_shapes_CollisionShape_setLocalScaling (pEnv=0x70d71c4b4290, shapeId=124065835654576, scaleVector=0x70d6e4dfc948) at /home/travis/build/stephengold/Libbulletjme/src/main/native/glue/com_jme3_bullet_collision_shapes_CollisionShape.cpp:255

And here's the Java stack from the crash log:

j  com.jme3.bullet.collision.shapes.CollisionShape.setLocalScaling(JLcom/jme3/math/Vector3f;)V+0
j  com.jme3.bullet.collision.shapes.CollisionShape.setScale(Lcom/jme3/math/Vector3f;)V+84
j  com.jme3.bullet.collision.shapes.MeshCollisionShape.setScale(Lcom/jme3/math/Vector3f;)V+2
j  com.jme3.bullet.collision.shapes.MeshCollisionShape.createShape()V+181
j  com.jme3.bullet.collision.shapes.MeshCollisionShape.read(Lcom/jme3/export/JmeImporter;)V+113
j  com.jme3.export.binary.BinaryImporter.readObject(I)Lcom/jme3/export/Savable;+203
j  com.jme3.export.binary.BinaryInputCapsule.readSavable(Ljava/lang/String;Lcom/jme3/export/Savable;)Lcom/jme3/export/Savable;+81
j  com.jme3.bullet.collision.shapes.infos.ChildCollisionShape.read(Lcom/jme3/export/JmeImporter;)V+64
j  com.jme3.export.binary.BinaryImporter.readObject(I)Lcom/jme3/export/Savable;+203
j  com.jme3.export.binary.BinaryInputCapsule.resolveIDs([Ljava/lang/Object;)[Lcom/jme3/export/Savable;+42
j  com.jme3.export.binary.BinaryInputCapsule.readSavableArray(Ljava/lang/String;[Lcom/jme3/export/Savable;)[Lcom/jme3/export/Savable;+68
j  com.jme3.export.binary.BinaryInputCapsule.readSavableArrayList(Ljava/lang/String;Ljava/util/ArrayList;)Ljava/util/ArrayList;+65
j  com.jme3.bullet.collision.shapes.CompoundCollisionShape.read(Lcom/jme3/export/JmeImporter;)V+19
j  com.jme3.export.binary.BinaryImporter.readObject(I)Lcom/jme3/export/Savable;+203
j  com.jme3.export.binary.BinaryInputCapsule.resolveIDs([Ljava/lang/Object;)[Lcom/jme3/export/Savable;+42
j  com.jme3.export.binary.BinaryInputCapsule.readStringSavableMap(Ljava/lang/String;Ljava/util/Map;)Ljava/util/Map;+75
j  com.jme3.scene.Spatial.read(Lcom/jme3/export/JmeImporter;)V+250
j  com.jme3.scene.Node.read(Lcom/jme3/export/JmeImporter;)V+79
j  com.jme3.export.binary.BinaryImporter.readObject(I)Lcom/jme3/export/Savable;+203
j  com.jme3.export.binary.BinaryImporter.load(Ljava/io/InputStream;Lcom/jme3/export/ReadListener;Ljava/io/ByteArrayOutputStream;)Lcom/jme3/export/Savable;+631
j  com.jme3.export.binary.BinaryImporter.load(Ljava/io/InputStream;)Lcom/jme3/export/Savable;+4
j  com.jme3.export.binary.BinaryImporter.load(Lcom/jme3/asset/AssetInfo;)Ljava/lang/Object;+17
j  com.jme3.export.binary.BinaryLoader.load(Lcom/jme3/asset/AssetInfo;)Ljava/lang/Object;+27
j  com.jme3.asset.DesktopAssetManager.loadLocatedAsset(Lcom/jme3/asset/AssetKey;Lcom/jme3/asset/AssetInfo;Lcom/jme3/asset/AssetProcessor;Lcom/jme3/asset/cache/AssetCache;)Ljava/lang/Object;+21
j  com.jme3.asset.DesktopAssetManager.loadAsset(Lcom/jme3/asset/AssetKey;)Ljava/lang/Object;+190
j  com.jme3.asset.DesktopAssetManager.loadModel(Lcom/jme3/asset/ModelKey;)Lcom/jme3/scene/Spatial;+2
j  com.jme3.asset.DesktopAssetManager.loadModel(Ljava/lang/String;)Lcom/jme3/scene/Spatial;+9

[stephengold]

The crash occurs while loading a collision shape using Libbulletjme v21.0.0 that was saved using that same native library on the same platform.
The crash occurs during the 2nd invocation of MeshCollisionShape.read().

Excerpt from MeshCollisionShape.createShape():

        boolean buildBvh = false;
        long meshId = nativeMesh.nativeId();
        long shapeId = createShape(useCompression, buildBvh, meshId);
        setNativeId(shapeId);

        setContactFilterEnabled(enableContactFilter);
        setScale(scale); // <----- crash occurs here!

Just before the crash, the value of scale is (16.336132, 1, 5.044612).
When btTriangleMeshShape::setLocalScaling() is invoked, m_bvh == NULL.
btTriangleMeshShape::setLocalScaling() must re-calculate the shape's AABB.
The recalculation invokes btTriangleMeshShape::localGetSupportingVertex(), which invokes
btBvhTriangleMeshShape::processAllTriangles().

Line 326 of processAllTriangles() dereferences m_bvh, which is still NULL. That's what crashes the JVM.

It ought to be possible to iterate over all mesh triangles without using BVH, but that's not how btBvhTriangleMeshShape is written.

[stephengold]

I added a test app to Minie: 3802942

[stephengold]

The fix is in: dcb979d

[stephengold]

joliver82 has confirmed the fix on Linux and Windows.