Premature GC of Pointer object when invoking getWideString #664
Comments
|
Writing m to a static variable prevents the crash: The reduced test case still fails today. (I tested with Win7 32-bit, Java 1.8.0_92, and JNA 4.2.2.) It isn't obvious to me how much freedom the JVM has in deciding when to finalize an object. (i.e. I couldn't tell whether the JVM was behaving correctly in this case.) |
|
At this point I'd agree with @rprichard I ran some tests and was only able to reproduce it with the 32 bit VM on windows. It looks indeed like a GC issue, as running it with the "-Xms1G" option (effectively preventing GC), the problem is not reproduced. The problem is also not reproduced with the 64bit VM, but the 64bit version runs with a different VM configuration (server), which most probably does less aggressive GC. A slightly modified version gives better error information: Native.setProtected(false);
// This code below is reduced from a different test. (I forget which.)
new Memory(4);
// The code below comes from NativeTest.testLongStringGeneration.
StringBuilder buf = new StringBuilder();
final int MAX = 2000000;
for (int i=0;i < MAX;i++) {
buf.append('a');
}
String s1 = buf.toString();
Memory m = new Memory((MAX + 1)*Native.WCHAR_SIZE);
m.setWideString(0, s1);
m.getWideString(0);This results in the following hs_err.log: https://gist.github.com/matthiasblaesing/efab6ce5f744c2146484adfab7010f2b |
|
Another update: I replace Memory with a subclass reporting finalization: public class MemoryWithFinalizer extends Memory {
public MemoryWithFinalizer(long size) {
super(size);
}
public MemoryWithFinalizer() {
}
@Override
protected void finalize() {
System.out.println("FINALIZER");
super.finalize();
}
}While I did not expect clear results, I got them. With the sample from #664 (comment) I get this (both finalizers, the one of the first dummy allocation and the real allocation for the string are run): |
That finalizer is known to prematurely free memory, in at least one case[1]. I noticed a non-reproducible "Invalid memory error" from a call to winpty_open that I speculate is a similar problem. [1] java-native-access/jna#664
matthiasblaesing
changed the title
|
Ok - so the GC is more aggressive, than is good for us, so we need a strategy to prevent GC of the Pointer object while the Native#getWideString method is running. Some options I see:
@twall I'd vote for variant 3 - would you agree? |
|
As long as it can be done without adversely affecting Direct mode performance. The field extraction (peer pointer value) was originally moved out of native to improve performance (avoiding vm interactions from native code). Native code already supports extracting the "peer" field from a Pointer object; the main issue is that you don't want to require that since it's additional "talk to the VM" overhead within the native code. Variant 1 might work without a static call; you just need something in the I don't want to bend over too far backward without understanding the rationale going on within the VM that says it's OK to GC an object when you're still in the middle of executing one of that object's member functions. That to me seems wrong. |
I ran across a presentation on Java finalization by Hans Boehm (dated 2004?), http://www.hboehm.info/misc_slides/java_finalizers.pdf. Slide 31 is interesting:
Fixes are provided on slide 34, which include volatile fields/statics or use of locking. On the next slide, Boehm chooses to use locks, but I wonder if JNA would prefer to avoid locking for better performance. In any case, the locking is only used to prevent premature finalization, so there's no deadlock concern. He also created an earlier slide deck that might be worth examining, http://www.hboehm.info/popl03/slides.pdf. Two slides mention a premature finalization problem:
From the second slide:
|
|
Would applying a fix to all JNA's Pointer-accepting methods even completely Maybe it's better to document the issue and make clear what some options On Tue, Jul 5, 2016 at 2:51 PM, Ryan Prichard notifications@github.com
|
|
I decided to check performance and implemented to variants, that both should fix the primary problem (premature GC in method call of Memory class). The two testbranches (see below) intend to stabilize the method calls in Pointer. Testvariants:T1 => only PrematureGCTest added to reproduce the problem, used as reference reference Systems:S1 => Windows 10 64Bit, VirtualBox, Oracle JDK 8u92 64 Bit Testing:3 Invokations of PerformanceTest without measurement Raw data:All combinations of Systems + Testvariants were run. The numbers are provided in an OpenDocument table file: http://www.doppel-helix.eu/Analyse.ods Observations:
At this point I'd go with the pass_pointer_to_native branch. From my perspective this looks like the most stable approach. Once an object as passed the barrier into native called (even if that code calls back into java at some point) from the GC perspective the object can't be collected, as the native code is black box it can't reason about. |
|
Another option (basicly picking up @twall 's suggestion: Make the user protect its objects. It is asked many times and it looks manageable (complete sample, see end of this message). The implementation of GCProtection is compatible with java 1.5 (according to java doc, Closeable is present since then), the usage in try-with-resource is compatible 1.7. package com.sun.jna;
import java.io.Closeable;
import java.util.IdentityHashMap;
public class PrematureGCTest {
public static void main(String[] args) {
Native.setProtected(false);
// This code below is reduced from a different test. (I forget which.)
new MemoryWithFinalizer(4);
for (int MAX = 50; MAX < 2000000; MAX += 100) {
// The code below comes from NativeTest.testLongStringGeneration.
StringBuilder buf = new StringBuilder();
for (int i = 0; i < MAX; i++) {
buf.append('a');
}
String s1 = buf.toString();
Memory m = new MemoryWithFinalizer((MAX + 1) * Native.WCHAR_SIZE);
try (GCProtection gcp = new com.sun.jna.GCProtection(m)) {
m.setWideString(0, s1);
m.getWideString(0);
}
}
System.out.println("OK");
}
}
class MemoryWithFinalizer extends Memory {
public MemoryWithFinalizer(long size) {
super(size);
}
public MemoryWithFinalizer() {
}
@Override
protected void finalize() {
System.out.println("FINALIZER");
super.finalize();
}
}
class GCProtection implements Closeable {
private IdentityHashMap<Object, Object> holder = new IdentityHashMap<Object, Object>();
public GCProtection(Object... objects) {
for(Object o: objects) {
holder.put(o, Boolean.TRUE);
}
}
public synchronized void keepAlive() {
// synchronize to prevent GC
}
@Override
protected void finalize() throws Throwable {
synchronized (this) {};
super.finalize();
}
@Override
public void close() {
this.keepAlive();
}
} |
|
That looks reasonable. On Sat, Jul 16, 2016 at 1:12 PM, matthiasblaesing notifications@github.com
|
|
I'll pull back that last suggestion. I had some time to think about this and the longer I think about it, there surer I get that the Pointer object indeed should be passed to native. @twall please have a look at my sample implementation: The helper class proposed in my last comment would work, but is an ugly work-around the real problem. That solution would add much noise, that makes it harder to get the code right and to read it. While the underlying problem is in the bindings. Are there numbers, that show the high overhead for the C->Java calls? |
|
The thing to test for performance would be the field access on the Pointer The other concern would be ensuring you can still make direct mode calls, On Mon, Aug 29, 2016 at 3:07 PM, matthiasblaesing notifications@github.com
|
|
Ok - some more thought down and measuring done. Please see the complete testset here: All data: http://www.doppel-helix.eu/Pointer_to_Native_Comparison.7z In the comparison:
pointer_to_native was the first implementation - this implementation passes the pointer object instead of the native pointer value to the native side. The drawback is, that a call C->Java is needed to access the field value of the pointer. pointer_native_direct modifies the first approach by passing the pointer and its native value to the C side. While potentially causing longer call times (more arguments need to be marshalled), it removes the necessaty to callback from C to java (in the common case). My interpretation of the results:
The unittests are unaffected by the changes:
From my perspective this looks good. If you agree I'd polish the pointer_native_direct branch, add documentation why the pointer object itself is passed to native (to prevent regression in the future) and that should do the trick. |
|
That's a pretty significant API change (would require a version bump) for On Wed, Aug 31, 2016 at 3:35 PM, matthiasblaesing notifications@github.com
|
|
Don't take this wrong: But what do you suggest as fix? I came up with different approaches and in the end it comes down to the fact, that JNA tries to play hide-and-seek with the garbadge collector. Maybe new developments in the JDK (project panama) will offer better control over the java<->native interaction, but as far a I see it, the only real option is to make the GC aware of the fact, that references to the Pointer objects still exists when calling into native. My reading is, that it is just luck, that the problem is not visible with the primitive types: Assume you have a Memory object and the last call you will make to this object is an integer read (Memory#getInt), now you call Memory#getInt, this delegates to Pointer#getInt and that in turn calls into native code, before it does so it resolves the target address for the read. This could result in a segfault:
From the GCs POV the Pointer object is not needed anymore and so the finalizer can run. If we are unlucky the finalizer finishes before the call to Native#getInt finishes. With respect to the API break: I see one method that breaks and that is Native#getDirectByteBuffer. I'll revert the change to that method. All other methods have default visiblity and so are only visible in the same package. If someone got there he/she is already way out of line. With regard to versioning: I changed the JNA_JNI_VERSION - whether this is minor or major can of course be discussed. You already bumped to 5.0.0 in january in that timeframe no release was done (4.2.2 was branched of before that) and I bumped it to 5.1.0. With respect to the scope of the changes: My take is, that it looks worse than it is. Yes there are many methods changed and yes that can introduce regressions, but in the end there are these groups:
|
|
Matthias, I really appreciate the work you've put into this, and I, too, am I think you've come up with a reasonable accommodation of both reference Previously this wasn't an issue when Pointer#getInt was itself a native Thanks On Thu, Sep 1, 2016 at 5:20 AM, matthiasblaesing notifications@github.com
|
|
public String getWideString(long offset) { public class Keeper An empty keep() might get optimized away. in that case: public class Keeper when a public Class reference is objected to: public class Keeper |
|
@sijskes sorry this won't work or if it does it depends on implementation details. The moment the JVM decides to inline Keeper.keep it can optimize the o.getClass() away, as it can argue, that the this pointer reference can't change its class. As this method is short it is likely to get inlined, getClass itself is a native (I asume an intrisic) so I don't know whether or not hotspot will optimize this away (access to the class info is a structure lookup) but it is probable. |
|
@matthiasblaesing you mean that the ordering is changed? even if we change the 'last' to volatile? So on entry, the o.getClass() is done before the Native.get.. ?? |
|
Is there are wrapper for a global reference? |
|
@matthiasblaesing re ordering, IMHO it depends on how you interpret you are right in assuming o.getClass() as a const within the method, not sure if reordering is allowed here. but possibly because there is no memory access, and therefore consistent (maybe?). |
|
@matthiasblaesing btw, i vote for option 3 when Ref is to intensive. (passing this to jni domain) |
|
With respect to the last comment: I don't see the point. The discussion is around the question whether or not this can be solved on the java level or if the solution has to be in the JNI layer. I don't get the intention. With respect to ordering the JLS for Java SE 8 (section 12.6.2. Interaction with the Memory Model is relevant. An in fact it could be argued, that the store into a static field is enough to protect this:
To force the execution and establish the happens-before you'd need a synchronized block or a volatile write/read pair (without the read there is no order). This was tested above and the performance was worse than passing the whole Pointer object into the C layer and doing a read of the field from C. What is more the object itself needs to be held strong, not its class. For the comparison:
I won't repeat my findings above the "Boehm" variant was nearly always slower than the "Pointer" variant. There is another drawback of the "Boehm" variant: as the synchronization is done on the object itself, this could introduce a deadlock, as other code could also synchronize on the same object. The new numbers are a deeper look and show that in a microbenchmark the differences are much higher/more noticable. The idea behind passing the Pointer object and the pointer value into native code is:
My main concern is that the native parts need recompilation. |
|
With respect to the vote: Which is option 3? There are three options on the table:
"The Boehm" method is nice because it works fully on the java level - but then it cries for deadlocks. |
|
@matthiasblaesing issues are a bad place to discuss things. my appologies. I believe a Ref implementation with fore-mentioned globalref jni calls will keep the this reachable until execution of the finally block. Otherwise the native part needs refactoring. Adding the this pointer to the call to the JNI will keep it reachable until return. |
java-native-access#664) … life-time Motivation: We should not cache the local and remote address as these might change during the life-time of the connection Modifications: - Override methods so we don't cache - Also duplicate ByteBuffer in the QuicConnectionAddress to make things safer. Result: Correctly return ids
(Copied from jna-users forum post on 2015-12-23, https://groups.google.com/forum/#!topic/jna-users/dCPnztnQnRw.)
I compiled JNA myself, and when I ran the test suite, the
testLongStringGeneration(com.sun.jna.NativeTest)test sometimes failed with a "Invalid memory access" error indicating a JVM segfault. Here's what it looks like running JUnit from the command-line:I'm using 32-bit Windows 7 SP1, with jdk1.8.0_60.
I worked on the test a bit, and came up with this standalone test program that segfaults using the official JNA 4.2.1 JAR.
With lots of printf debugging, I think I identified the cause. The final
m.getWideStringcall ultimately callsJava_com_sun_jna_Native_getWideString. Once the JVM reaches this native function, there are no live references remaining to theMemoryobject.newJavaStringcalls back into the JVM,((*env)->NewString), to create the 2MB String object, at which point the JVM finalizes the Memory object, which frees the buffer. The JVM then attempts to read from the freed buffer and segfaults.The text was updated successfully, but these errors were encountered: