Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: support compilation to wasip1 #344

Merged
merged 1 commit into from
Aug 29, 2023

Conversation

achille-roussel
Copy link
Contributor

Problem

I'm interested in getting ristretto to build to the new wasip1 target, but it fails due to ristretto's assumption that the program is being compiled to a unix system where files can be mapped to memory. At this time, the x/sys/unix package does not consider wasip1 to be a unix platform, and memory mapping isn't supported either when compiling to WebAssembly.

$ GOOS=wasip1 GOARCH=wasm gotip build
package github.com/dgraph-io/ristretto
	imports github.com/dgraph-io/ristretto/z
	imports golang.org/x/sys/unix: build constraints exclude all Go files in /go/pkg/mod/golang.org/x/sys@v0.0.0-20221010170243-090e33056c14/unix

Solution

The proposed solution in this PR is to provide stubs implementations of the memory mapping functions which return ENOSYS to indicate that the functionalities are not available when compiled to GOOS=wasip1.

If I'm reading the code correctly, this means that persisting data structures to a storage medium will not be supported, but the rest of ristretto should work as expected.

I believe this to be a useful trade-off since many dependents of ristretto only use the in-memory data structures (e.g. https://github.com/DataDog/datadog-agent/blob/main/pkg/obfuscate/cache.go, which is what I'm interested in getting to compile).

A more advanced solution could be to rework the z.Buffer internals so the data structure can work on platforms where mapping files to memory isn't supported; however, this represents a bigger undertaking, I would like to first unblock dependencies by allowing the program to compile.

Please let me know if you have any concerns or feedback on the change!

Signed-off-by: Achille Roussel <achille.roussel@gmail.com>
@CLAassistant
Copy link

CLAassistant commented Jul 23, 2023

CLA assistant check
All committers have signed the CLA.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@achille-roussel achille-roussel changed the title z: support compilation to wasip1 fix: support compilation to wasip1 Jul 23, 2023
@mangalaman93 mangalaman93 merged commit 3f6b44a into dgraph-io:main Aug 29, 2023
@mangalaman93
Copy link
Contributor

Thank you for the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants