FFIgenPad #1390

TheComputerM · 2024-08-03T18:09:47Z

This is a PR for adding FFIgenPad (a GSoC proposal)

Bulk of the code is directly lifted from ffigen 14-wip, some major changes are
- @Native bindings instead of DynamicLibrary cause of dart2wasm
- use of a wrapper.c (used to make functions that play well with pointers) cause dart2wasm currently doesn't play well with structs
- header_parser/clang_bindings/clang_types.dart and header_parser/clang_bindings/clang_wrapper.dart aim to provide 1:1 parity with clang_types and the Clang class in ffigen respectively, this leads to the only changes in many files being the import paths
parts which are constant and independent, (like most of lib/src/strings.dart) are exported directly from ffigen
The website is located in the web directory, currently uses nodejs.

FFIgenPad also requires a wasm build of libclang, bundled with wrapper.c, which currently is being tracked with git but there are no instructions on how to setup and build it, I'll probably add it soon though.

I’ve reviewed the contributor guide and applied the relevant portions to this PR.

Contribution guidelines:

See our contributor guide for general expectations for PRs.
Larger or significant changes should be discussed in an issue before creating a PR.
Contributions to our repos should follow the Dart style guide and use dart format.
Most changes should add an entry to the changelog and may need to rev the pubspec package version.
Changes to packages require corresponding tests.

Note that many Dart repos have a weekly cadence for reviewing PRs - please allow for some latency before initial review feedback.

TheComputerM · 2024-08-21T10:10:31Z

A demo of the web app: https://ffigenpad.surge.sh/

TheComputerM · 2024-08-21T10:12:17Z

@dcharkes @eyebrowsoffire I think we are ready of review.

Levi-Lesches · 2024-08-21T17:07:57Z

Hey, I've been subscribed to the PR for a while now, nice work! I noticed there's a complete copy of ffigen in the code, and I was wondering if maybe it can be imported instead so it stays up-to-date with the version on pub.dev?

TheComputerM · 2024-08-21T20:00:24Z

@Levi-Lesches thanks for your interest! there are still multiple 'issues' preventing me from doing that, I have listed them on this blog post, in short stuff like loading dylibs and ffi.Structs are not really patchable if you directly import from ffigen.

I am also experimenting with a way we can use patch changes to somehow upstream changes.

eyebrowsoffire

A lot of these files are copy-pasta'd from the ffigen library (with some modifications). Could you add a comment to the top of each of those that indicates which file it is derived from?

TheComputerM · 2024-08-23T17:56:21Z

A lot of these files are copy-pasta'd from the ffigen library (with some modifications). Could you add a comment to the top of each of those that indicates which file it is derived from?

@eyebrowsoffire I made sure to keep the project structure the same (ie ffigen/lib/src/code_generator/type.dart is just at ffigenpad/lib/src/code_generator/type.dart), is there really a need for a separate comment? (it may also mess with the git patch thingy)

eyebrowsoffire · 2024-08-23T18:01:58Z

A lot of these files are copy-pasta'd from the ffigen library (with some modifications). Could you add a comment to the top of each of those that indicates which file it is derived from?

@eyebrowsoffire I made sure to keep the project structure the same (ie ffigen/lib/src/code_generator/type.dart is just at ffigenpad/lib/src/code_generator/type.dart), is there really a need for a separate comment? (it may also mess with the git patch thingy)

How about a section of the readme that lists which subsets of the tree are derived from other sources and where to find them? I know you have kept the project structure the same, but even just reviewing this code right now it is not clear to me what is a new file and what is a copied/modified file.

dcharkes

Sorry for the delay!

dcharkes · 2024-08-29T06:53:55Z

.github/workflows/ffigenpad.yml

+
+      - run: dart pub get
+        working-directory: pkgs/ffigenpad
+


Run dart analyze and dart format on the code on the CI here.

pkgs/ffigenpad/CHANGELOG.md

pkgs/ffigenpad/analysis_options.yaml

dcharkes · 2024-08-29T07:06:46Z

pkgs/ffigenpad/lib/src/ffigen.dart

+    }
+  }
+
+  // TODO: look into better color formatting


File bugs on dart-lang/native and put the URL in TODO(<url>): <short decription>.

dcharkes · 2024-08-29T07:06:58Z

pkgs/ffigenpad/lib/src/header_parser/parser.dart

+  // Log all headers for user.
+  _logger.info('Input Headers: ${config.entryPoints}');
+
+  // TODO: for some reason it can't add to a List of pointers


dcharkes · 2024-08-29T07:26:13Z

pkgs/ffigenpad/third_party/libclang/README.md

+
+### include/
+
+Contains header files for libclang that are used by ffigen to generate dart:ffi bindings.


dart:ffi in backticks.

Is there a script to update include/clang-c/ from the clang build?

I don't think it's needed but I'll add it.

pkgs/ffigenpad/README.md

dcharkes · 2024-08-29T07:50:24Z

pkgs/ffigenpad/README.md

+cd web
+npm i # install dependencies
+npm run build
+# preview with: npm run preview


http://localhost:4173/ -> 404

TheComputerM added 30 commits July 25, 2024 12:07

[ffigenpad] init project

67b39c8

[ffigenpad] add libclang.wasm bindings

af7d2e4

[ffigenpad] fix ffi.Struct generation

4ce5c06

[ffigenpad] fix missing publish_to in pubspec

e862969

[ffigenpad] add remaining functions

9bf1b10

[ffigenpad] add website skeleton

a40ca94

[ffigenpad] functional libclang app

307e760

[ffigenpad] port some utils

46e3e23

[ffigenpad] add wrapper functions

79cb95e

[ffigenpad] add llvm build folder

5e6c243

[ffigenpad] format wrapper.c

bd0122b

[ffigenpad] more wrapper functions

46ceb47

[ffigenpad] add build script for libclang.wasm

ed74146

[ffigenpad] more wrapper functions

4069841

[ffigenpad] fix build scripts

fd84f55

[ffigenpad] generate enums as integers

b5adb4f

[ffigenpad] add clang_types stub

3f38614

[ffigenpad] add logging package

aa15b0d

[ffigenpad] moar functions

0de06c0

[ffigenpad] create a Clang class

3940d70

[ffigenpad] remove wasm function

22b5f83

[ffigenpad] is life just copy pasting

b8111ec

[ffigenpad] modify visitChildren wrapper

463985e

[ffigenpad] rename to calloc and add emscriptens function interop

1155d8e

[ffigenpad] test out visitChildren

35e5f0e

Merge branch 'main' into ffigenpad

bf85ca5

[ffigenpad] moar utils

3bed03c

[ffigenpad] add data.dart

6180832

[ffigenpad] add code generator

d0bbe2b

[ffigenpad] add config

75e6994

TheComputerM added 4 commits August 19, 2024 23:17

[ffigenpad] update README

f5b4f5e

[ffigenpad] update packages

8e266bc

Merge branch 'dart-lang:main' into ffigenpad

7c5fc5c

[ffigenpad] add deploy to github actios

945e1e0

github-actions bot added the type-infra A repository infrastructure change or enhancement label Aug 19, 2024

TheComputerM added 7 commits August 20, 2024 13:35

[ffigenpad] change paths

60d9799

[ffigenpad] add optimize build

8485fdc

[ffigenpad] remove build files

9ef321d

[ffigenpad] remove gitignore

4df8cbf

[ffigenpad] optimize builds

a5dac1f

[ffigenpad] add llvm-project license

6daf9a6

[ffigenpad] move park-ui components to third_party

c3d84d5

TheComputerM marked this pull request as ready for review August 21, 2024 10:09

TheComputerM added 2 commits August 23, 2024 21:46

[ffigenpad] tidy up

b39a9bf

[ffigenpad] correct command

06925e6

eyebrowsoffire suggested changes Aug 23, 2024

View reviewed changes

TheComputerM added 2 commits August 24, 2024 10:04

[ffigenpad] add details on project structure

01eacb6

add missing entry

4806041

dcharkes reviewed Aug 29, 2024

View reviewed changes

TheComputerM added 4 commits September 1, 2024 15:07

refactor

19d19ff

ffigenpad: refactor based on lint rules

f4b654f

Merge branch 'dart-lang:main' into ffigenpad

1755e12

[ffigenpad] cleanup

2312d63

TheComputerM marked this pull request as draft September 5, 2024 05:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FFIgenPad #1390

FFIgenPad #1390

TheComputerM commented Aug 3, 2024

TheComputerM commented Aug 21, 2024

TheComputerM commented Aug 21, 2024

Levi-Lesches commented Aug 21, 2024

TheComputerM commented Aug 21, 2024 •

edited

Loading

eyebrowsoffire left a comment

TheComputerM commented Aug 23, 2024

eyebrowsoffire commented Aug 23, 2024

dcharkes left a comment

dcharkes Aug 29, 2024

dcharkes Aug 29, 2024

dcharkes Aug 29, 2024

dcharkes Aug 29, 2024

TheComputerM Sep 4, 2024

dcharkes Aug 29, 2024


		### include/

		Contains header files for libclang that are used by ffigen to generate dart:ffi bindings.

FFIgenPad #1390

Are you sure you want to change the base?

FFIgenPad #1390

Conversation

TheComputerM commented Aug 3, 2024

TheComputerM commented Aug 21, 2024

TheComputerM commented Aug 21, 2024

Levi-Lesches commented Aug 21, 2024

TheComputerM commented Aug 21, 2024 • edited Loading

eyebrowsoffire left a comment

Choose a reason for hiding this comment

TheComputerM commented Aug 23, 2024

eyebrowsoffire commented Aug 23, 2024

dcharkes left a comment

Choose a reason for hiding this comment

dcharkes Aug 29, 2024

Choose a reason for hiding this comment

dcharkes Aug 29, 2024

Choose a reason for hiding this comment

dcharkes Aug 29, 2024

Choose a reason for hiding this comment

dcharkes Aug 29, 2024

Choose a reason for hiding this comment

TheComputerM Sep 4, 2024

Choose a reason for hiding this comment

dcharkes Aug 29, 2024

Choose a reason for hiding this comment

TheComputerM commented Aug 21, 2024 •

edited

Loading