Abi Generation

[coleFD]

@itegulov
This discussion is in response to the discussion here: near/near-sdk-abi#1

To reiterate, my team is preparing a grant application regarding some work we are doing in this area (abi generation, client library generation, and dev environments), so I want to make sure we aren't wasting time on our project. If that is the case, we would love guidance on what needs to be built. We are just wanting to add value to the NEAR ecosystem, regardless if it results in personal revenue or not.

So far most of our work has gone into generating an abi from a project root, where our plan for the abi schema is similar to the near team's approach (an array for functions, and a mapping for types that can be crossed referenced by types encountered in the functions field.

Our approach contains 3 data structures:

1. a high level structure that contains the parsed outputs directly relating to the desired data from the contract (functions, types, imports, etc).
2. A file system tree that uses stores the parsed output for each file as well as its hashes for caching
3. A dependency graph (rust analyzer's crate graph) in the case that crucial contract information needs to be parsed from external libraries

Our build steps include:

1. build crate graph
2. check for a cached FileTree or build new tree. If detected, it only checks for changes to files (by checking changes to file hashes and root hashes)
3. traverse FileTree and obtain parsed output for each file (does not parse external crates at this point)
4. build abi by traversing FileTree and referencing the parsed output structure (if it finds a crucial import to an external crate, this is the point where it searches for that file so that it can be parsed)

Our method utilizes the rust-analyzer's parser and dependency graph since this seems to now be the most trusted language server system for rust. Our method most likely has more storage and compute overhead than this approach, but our reasoning to do this was to prevent forcing the user to add macros around the project. Curious what you guys think.

Here is our Parse Structure:

Here is the FileTree Structure:

Our ideal solution would be to parse the contract's wasm but we could not personally find a solution with the current wasm output.

[coleFD]

Also, is it cool if I add an issue to this repo? I noticed the abi generator only detects "contract" directories but doesnt run if you run the command within a workspace root, which would be a nice future small but helpful feature

[willemneal]

@coleFD My team is also aiming to work on this issue. We went with using https://github.com/bnjjj/witgen since it's not just near specific. I would be interested to collaborate and bring your current solution into witgen. Currently we treat the crate as one syn file and have macro attributes to determine if an import is a wit interface and then combine all deps into one.

We also have a current solution for building contracts with https://github.com/raendev/raen. And again would love to work with you to improve the types generation aspect. We are finishing up wit-bindgen to JSON, at which point we can generate the ABI from the wit. Wit is more compact and our tool actually generates a smaller binary with the embedded types artifact (currently JSON, but soon to be WIT).

Here is our current admin panel to view a contract's methods if built with raen. https://raen.dev/admin

Again I think your solution would be great for the wasm ecosystem at large not just near.

[willemneal]

@coleFD We are about to send our grant proposal to the NF this week, so perhaps we should connect so that we can ensure that your grant complements ours so that you can get the funding and we can work together! If this interests you you can email me at

team at ahalabs.dev

[austinabell]

Moved this discussion to this repo, as this is where discussion around ABI related things will happen.

I think the strategy is super interesting, and I'm very curious to see what it looks like if/when it is open sourced.

but our reasoning to do this was to prevent forcing the user to add macros around the project.

Ours doesn't require that as of now either, but with a completely different strategy.

How do you think would be the best way we could collaborate on this?