Tinygo need serialization tools #447
Comments
|
Agree, having Google ProtoBuf is must have from the embeded world point of view... I didn't try to compile ProtoBuf with TinyGo so far. @aykevl, maybe mark with issue as |
|
From the list of things that work and what they are blocked on, it looks like to enable a lot of them we need to get base64 working first before moving on to solve gob, etc. The error for base64 regarding branching on a const expression might be addressed in dev, according to a comment on #437 two days ago, so this is a good place to look. If this fixed base64, it probably unblocks work on other packages. Gob and json might benefit from the recent updates to reflection, so it's certainly something worth checking on dev so that we can get going with more encoders. |
|
As of TinyGo v0.8.0 the |
Still, progress, thanks to everyone involved! |
|
The 0.9 release gave us base32, base64, and csv. We'll probably need additional reflection support before json, xml, and gob. I expect ProtoBuf will have other things it needs. Great progress! |
|
It's not necessary to use encoding/json to parse and encode JSON in Go. This library compiles on tinygo with -target wasm: https://github.com/buger/jsonparser This is possible because it import graph is really small - the only thing that imports 'reflect' is its fmt dependency: https://godoc.org/github.com/buger/jsonparser?import-graph I wasn't able to test jsonparser because I'd like to use it to handle a HTTP request response, but alas I don't see any examples for passing values from JS to exported WASM functions, even though I can call them in the JS callback. |
|
This is a good tip. While most people using JSON probably don't want the tradeoffs in jsonparser, I'd be willing to bet most users of tinygo would want those tradeoffs, at least if they are using tinygo for tiny machines. |
|
for golang/protobuf, apart from the json issue mention above, there are the following other issues: |
|
For anyone else trying to get protobuf to work... it can. Sort of. If you use gogoproto, you can have it generate marshal and unmarshal functions that work on the bytes, without reflection. Then, you "just" need to stub the proto package with something like this package proto
import "errors"
const Marshal = 0
const GoGoProtoPackageIsVersion3 = 0
type Message interface {
}
type InternalMessageInfo interface {
DiscardUnknown(interface{})
Merge(interface{}, interface{})
Marshal(interface{}, interface{}, interface{}) ([]byte, error)
}
func RegisterFile(string, []byte) {}
func RegisterType(interface{}, string) {}
func CompactTextString(interface{}) string {
return "nope"
}You can then call Marshal and Unmarshal directly. |
|
Just wanted to give a heads up. My tests have found that https://github.com/valyala/fastjson compiles successfully and runs with the TinyGo compiler targeting WASM/WASI. |
|
Is there any other way to compile those packages that are not supported by tinygo? Must find support instead? |
The issue is the TinyGo compiler doesn't support 100% of Go's So at the moment, the only work around is find JSON packages that don't need those unimplemented functions. |
|
I am a third-party package using the xml package, which is not easy to replace. |
|
Unfortunately, there isn't much that can be done right now until the necessary changes/fixes are made within the TinyGo compiler. |
|
The |
|
I made a proof of concept demo to use code generation to implement |
|
@taowen Yes please! |
|
@dgryski all features of |
|
It looks like |
|
Since #2314 was merged, |
Is there a chance to make it work on macos / M1? |
The error you've linked is for x86_64, not arm64. |
|
I haven't tried
Which is correct? Ping @mathetake @jptosso |
|
Attempting to use |
|
some experiments: https://github.com/cbrake/tinygo karmem at least compiles. Could not get any protobuf stuff to compile -- may need to port nanopb or something ... |
|
@cbrake VTProtobuf with the references to the reflection-based Google protobuf libraries removed - and one extra generator which creates .pb.go with just the structs & not the reflection stuff - would probably do the job. It's possible with just hacks, I think. |
|
Anyone found a drop in replacement for xml/ encoding ? |
Me too |
|
I can't believe Go (TinyGo) is about to miss the boat on WASM |
|
There is are a few non reflection based xml packages out there to replace encoding/xml. How did the tinygo Team get JSON done ? I ask because xml is holding me back |
|
I'm working on fixing reflection support which should enable both JSON and XML |
|
Tinygo+ encoding/xml is now working and the code is pushed
…On Sat, 4 Mar 2023 at 21:43 Gerard Webb ***@***.***> wrote:
There is are a few non reflection based xml packages out there to replace
encoding/xml.
How did the tinygo Team get JSON done ? I ask because xml is holding me
back
—
Reply to this email directly, view it on GitHub
<#447 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJN7TR7FFTSIVLCQVLHMNTW2PORJANCNFSM4H6XEOZQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
I tested XML and works really well ... i have a dependency on https://github.com/go-pdf/fpdf. encoding/gob is still in the works right ? Is there anything i can do to help.. |
|
The latest development branch should have |
|
@dgryski thanks for the prompt response. When I compiled it kick now tell me the line number of where it failed. I assume that I need to use sone flags or not compile removing debugb symbols ? |
and others
We really need serialization tools for tinygo, because tiny go can not use serialization tools just like coding/json encoding/gob in official golang.
The text was updated successfully, but these errors were encountered: