Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finalizer vs. NativeFinalizer #52436

Open
DanielSWolf opened this issue May 18, 2023 · 4 comments
Open

Finalizer vs. NativeFinalizer #52436

DanielSWolf opened this issue May 18, 2023 · 4 comments
Assignees
Labels
area-documentation Prefer using 'type-documentation' and a specific area label. area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. library-ffi P3 A lower priority bug or feature request triaged Issue has been triaged by sub team type-question A question about expected behavior or functionality

Comments

@DanielSWolf
Copy link

DanielSWolf commented May 18, 2023

I'm having a hard time understanding the exact differences between Finalizer (docs) and NativeFinalizer (docs). Here's a feature matrix I created, based on the officials documentation and a bit of guesswork.

Finalizer NativeFinalizer
Guaranteed to be called -
Guaranteed to be called in a timely manner - - (edited)
Can be called during synchronous operation -
Guaranteed not to be called prematurely with variable still in scope - ✔ (edited)
Premature execution can be prevented through Finalizable -
Callback can be Dart function object -
Callback can be native function -
  1. Is this matrix correct?
  2. The Finalizer docs make a point of stressing how very unreliable it is. Why would one ever choose Finalizer over NativeFinalizer?
  3. Some native functionality may already be wrapped in Dart code, so the callback for freeing a native resource may be a Dart function. For instance, package:ffi offers handy calloc and calloc.free functions. Given that NativeFinalizer only supports native callback functions, it seems that there is no way to reliably have a non-native free function called -- unless I were to wrap the Dart function in a native function just to be able to pass it to NativeFinalizer. Is this correct?
@lrhn lrhn added the type-question A question about expected behavior or functionality label May 18, 2023
@lrhn
Copy link
Member

lrhn commented May 18, 2023

Ad 1. @dcharkes?

Ad 2.
NativeFinalizer is part of dart:ffi, which many do not, and cannot, use. (Anyone on the web, or creating a package expected to be used cross-platform.)
If "Callback can be Dart function object" is "no", I simply cannot use it for anything I write, because I only write Dart.

The Finalizer class is not dart:ffi specific, and can be used by anyone, anywhere.

  1. See 1.

@lrhn lrhn added area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. library-ffi labels May 18, 2023
@dcharkes
Copy link
Contributor

RE 1: I've edited your matrix.

Finalizable prevents premature finalization. NativeFinalizers can only have Finalizables attached, so NativeFinalizers are guaranteed to not have premature finalization. If you encounter any bugs with that, please report.

"Guaranteed to be called in a timely manner", if you write code that doesn't allocate a whole lot (e.g. only some small integer manipulation in an existing data structure, or just a bunch of async that is doing nothing waiting on some timer to expire) it is entirely possible long stretches of time go without GC. So, the native finalizers are not guaranteed to be running in a timely manner.

RE 2: Indeed, if you only have Dart code, you can only use Finalizer. In that case you don't care about finalizers not being run on isolate shutdown, because the whole heap that the finalizer would manipulate is gone. If you have native resources, e.g. a database-handle, malloc-ed memory, etc., then you need to use NativeFinalizers.

RE 3: Instead, you should have a NativeFinalizer with the free pointer to free your native memory.

unless I were to wrap the Dart function in a native function just to be able to pass it to NativeFinalizer.

You can't callback into Dart from the C function that is being run as native finalizer. You could send a message back to dart asynchronously with a native port.

Your specific ask about using an allocator is being tracked on:

@Jeanno
Copy link

Jeanno commented Jun 16, 2023

3. Some native functionality may already be wrapped in Dart code, so the callback for freeing a native resource may be a Dart function. For instance, package:ffi offers handy calloc and calloc.free functions. Given that NativeFinalizer only supports native callback functions, it seems that there is no way to reliably have a non-native free function called -- unless I were to wrap the Dart function in a native function just to be able to pass it to NativeFinalizer. Is this correct?

Can you explain what is "premature finalization"? The term "finalization" is being used in the docs without being defined clearly what it means in the context.

I still don't fully understand the use cases for Finalizer even with the example provided in Finalizer (docs). If there's a chance where Finalizer is not called, then in the example, it only mitigates the problem of user not closing the Database connection, instead of preventing it entirely?

If that's the case, am I correct to say that it's still on the users to make sure every close and dispose (e.g. ui.Image) are called?

@dcharkes
Copy link
Contributor

If there's a chance where Finalizer is not called, then in the example, it only mitigates the problem of user not closing the Database connection, instead of preventing it entirely?

Maybe we should consider replacing the example with something purely dart.

Yes, that why for native resources (like a sqlite3 database or query handle) we should use NativeFinalizers.

For Dart resources (let's say some hashmap cache) we can use normal finalizers. If the isolate shuts down, all objects will be gone anyway.

Can you explain what is "premature finalization"?

class MyDatabase implements Finalizable {
  Pointer<Void> database;

  MyDatabase() // attaches finalizer
}

void main() {
  final db = MyDatabase(...);
  db.query(...);
}

// Now, the dart compiler might realize it can inline db.query(...);
void main() {
  final db = MyDatabase(...);
  final pointer = db.database;
  someFffCallForQuery(pointer, ...);
}

// And the garbage collector might run before the query.
void main() {
  final db = MyDatabase(...);
  final pointer = db.database;
  // GC runs, collects `db`, and runs the finalizer, which calls a close/release on the native database.
  someFffCallForQuery(pointer, ...); // uses `pointer` after the native database has been closed -> crash
}

If that's the case, am I correct to say that it's still on the users to make sure every close and dispose (e.g. ui.Image) are called?

I don't know if Flutter cleans up it's native resources when the wrapper objects are GCed. In Dart we do for example for File, if you forget to close a File and it's GCed, we close the native file handle. (Though, it's better to just write close as a user, because the file will probably be closed much sooner.)

@a-siva a-siva added area-documentation Prefer using 'type-documentation' and a specific area label. triaged Issue has been triaged by sub team P3 A lower priority bug or feature request labels Dec 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-documentation Prefer using 'type-documentation' and a specific area label. area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. library-ffi P3 A lower priority bug or feature request triaged Issue has been triaged by sub team type-question A question about expected behavior or functionality
Projects
None yet
Development

No branches or pull requests

5 participants