SDKs for various environments #54

hackerwins · 2020-05-23T04:00:22Z

Description:

We provide Go Client and JS SDK. It would be good to implement the SDK in the following order.

Why:

Users can use Yorkie in a variety of environments.

gedw99 · 2020-06-02T21:05:48Z

Dart Flutter would be cool.
FLutter runs on Mobile, Desktop and Web from just the dart code.

Is has full GRPC and Protobufs support too.

The other options is to compile the golang using gomobile and embed it inside flutter:
Here is an example of doing that with the Protobufs used as the API:
https://github.com/textileio/grpc-ipfs-lite/blob/master/Makefile

also a high level discussion about it.
https://www.freecodecamp.org/news/flutter-platform-channels-with-protobuf-e895e533dfb7/

Let me know if you want some help. Yorkie looks really good !

Also i noticed your using the envoy Proxy.
I use golang backend with flutter frontend and the ENvoy proxy to do the GRPC - Web stuff.
Works really well

Code: https://github.com/getcouragenow/packages/tree/master/maintemplate/server/deployments/helm/templates

Demo: https://maintemplate.ci.getcouragenow.org/#/chat/0

hackerwins · 2020-06-03T04:00:48Z

@gedw99 I didn't know about Dart Flutter, but thanks for letting me know. And thank you for your interest in Yorkie. Providing a Flutter SDK seems like a good idea.

But these days, I am working on the server, so it is difficult to me to directly develop the Flutter SDK for now. When someone is working on the Flutter SDK, I think we can talk and I can help the one to implement Yorkie's algorithm.

gedw99 · 2020-06-03T07:54:45Z

I can do the flutter layer.

Your architecture is very similar to our project, since you use envoy and grpc-web etc.

The team is in Telegram btw:
https://t.me/GetCourageNow_dev

I am really interested in the CRDT Server and offline editing, and think i can help with it since we all program in golang and flutter

gedw99 · 2020-06-11T21:05:55Z

I managed to get the Golang girlie client running inside flutter web and native ( mobiles and desktop ).

I had to do some very bad hacks though. I just wanted to see how it behaved.

I am trying out a client side dB called genii.
It’s Golang based and can cross compile to wasm. This is a doc store with sql abilities

gedw99 · 2020-06-11T21:08:31Z

Sorry I pressed return too early. The protobuf pub sub or looks really nice btw.

For the ja and react clients do you have a client side dB ?

joe-getcouragenow · 2020-06-18T08:08:09Z

So for Dart then everything in the following will need to be ported to Dart in order to use this in flutter ?
"github.com/yorkie-team/yorkie/pkg/document"
"github.com/yorkie-team/yorkie/pkg/document/key"
"github.com/yorkie-team/yorkie/pkg/document/time"

hackerwins · 2020-06-18T08:22:35Z

@joe-getcouragenow That's right. Under the yorkie/pkg/document package you mentioned, there is Document logic composed of CRDT. In addition, the logic for the Client is required.

I think it would be helpful to figure out the scope of developing SDK based on the JS SDK.
https://github.com/yorkie-team/yorkie-js-sdk/tree/master/src

joe-getcouragenow · 2020-06-18T19:40:52Z

@joe-getcouragenow That's right. Under the yorkie/pkg/document package you mentioned, there is Document logic composed of CRDT. In addition, the logic for the Client is required.

I think it would be helpful to figure out the scope of developing SDK based on the JS SDK.
https://github.com/yorkie-team/yorkie-js-sdk/tree/master/src

Not sure what your mean by the above.

hackerwins · 2020-06-19T08:50:50Z

I mean, it would be helpful to develop new SDK in other environments or languages with reference to the already developed JS SDK.

joe-getcouragenow · 2020-06-20T10:28:44Z

got it.

How about we WASM the golang code above ?
I am not joking...

from your JS layer its easy to use.
From flutter, and many others it easy to us.

Its a good candidate because its simple code, and you just need to inject the data you get over the wire into it and get the result out. Very good worker architecture.

If you dont like this we can also do it the traditional way...

joe-getcouragenow · 2020-06-20T10:31:12Z

code:
https://github.com/rodydavis/flutter_wasm_interop

hackerwins · 2020-06-22T01:04:21Z

@gedw99 There is no local DB on the JS SDK yet. Later, when we introduce local DB, we can store each property of the Document. I think it will be easy because we use Protobuf.

hackerwins · 2020-06-22T01:09:37Z

@joe-getcouragenow I don't know about WASM yet, but I think it's a good idea if it can be used in multiple environments.

gedw99 · 2020-06-22T08:58:33Z

So I think in order to support any client envoy proxy will be needed. You can easily run it in the docker compose.

This allows grpc to work with web ( grpc-web ) and native ( grpc ).

It will also make the system scale horizontally because envoy proxy runs as s sidecar to each server service. You can update the envoy config for all instances using the envoy grpc API.

This will mean that when you codegen from the grpc protobufs the generated code for all clients ( any language or web or native ) will just work.

https://github.com/envoyproxy/envoy/blob/master/examples/front-proxy/docker-compose.yaml

Also Auth and authz can be done with envoy too
https://medium.com/@ricklee_10931/envoy-external-authorization-a-simple-example-d50ef2ede631

The rest or your Architecture remain mostly untouched . In fact alot of the code on the clients and servers will not be needed due to the code generation.

hackerwins · 2020-06-22T09:14:39Z

@gedw99 Thank you for the explanation. I wonder if you are going to work on what you said.

joe-getcouragenow · 2020-06-22T10:34:02Z

@gedw99 Thank you for the explanation. I wonder if you are going to work on what you said.

Awesome....

I think the first thing is to move the docker compose that has envoy in it into the yorkie repo.
Its currently in yorkie-js-sdk which is the wrong place because envoy is used for everything and not just JS.
SO then then the server is ALWAS running envoy, and not just for the JS.

@hackerwins i think that pretty easy to do on your side first ?

Then i think automatic TLS is needed, so that its very easy to boot an Envoy with LTS.

"type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext"

Also the envoy config should be setup to run as an edge proxy, as its not being used as a Services mesh in the Yorkie context right ?
https://www.envoyproxy.io/docs/envoy/latest/configuration/best_practices/edge

Yu can see if has the TLS there also.

Once the Yorkie repo has this then its easy to add autocert integration for Lets Encyprt.

Later we can use Mongo DB to store the certs. This is needed for running more than one yorkie.

I am working on that now.

then i think the WASM of the types using tinygo is next. These types:
"github.com/yorkie-team/yorkie/pkg/document"
"github.com/yorkie-team/yorkie/pkg/document/key"
"github.com/yorkie-team/yorkie/pkg/document/time"

Because those types are so simple and have no reflection, tinygo can do it.

I am working on that now.

Finally...

At this point we can support all clients types and all languages.

everyone can handle grpc / protobuf
everyone can load WASM ( browser or non browser ) these days. See: https://github.com/wasmerio/wasmer#language-integrations

joe-getcouragenow · 2020-06-22T10:34:53Z

All:

let me know if you want to change anything Or if you like this approach

gedw99 · 2020-06-22T11:27:57Z

Sounds good to me 👍

Background on auto TLS :
https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/security/ssl

We need a provider. Probably autocert like what the golang team uses ?

joe-getcouragenow · 2020-07-01T08:34:13Z

@hackerwins

you

@hackerwins
I started to work on this, but as i got into it i realised that there are other things that need to be done first.
I will chat to you on Slack about these if you dont mind.

hackerwins · 2021-07-31T15:27:31Z

Lately, I've been interested in Rust. It would be nice to implement Rust SDK after the JS SDK.

gedw99 · 2021-08-02T07:14:41Z

@hackerwins you might be interested in the GIO golang gui.
I was using flutter and changed to using gio.

its 100% golang for Web, Desktop and Mobile. The underlying design is like Flutter in that its a canvas that the GPU ( webgl canvas in webs case ) renders to.

https://gioui.org/

The advantage with yorkie is of course that the backend and frontend gui is all in golang, so it can make things easier. Especially from an SDK point of view because you can share code on both side.

If you want more info let me know ..

https://github.com/planetdecred/godcr is a good example of a big app btw

hackerwins · 2022-07-28T07:14:28Z

We created yorkie-ios-sdk and yorkie-android-sdk repositories.

hackerwins added the enhancement 🌟 New feature or request label May 23, 2020

hackerwins mentioned this issue Jun 15, 2020

golang client #62

Closed

hackerwins added the help wanted 🦮 Extra attention is needed label Jul 23, 2020

hackerwins self-assigned this Oct 21, 2021

hackerwins assigned blurfx Oct 26, 2021

hackerwins added this to v0.2 Oct 29, 2021

hackerwins moved this to Todo in v0.2 Oct 29, 2021

hackerwins moved this from Todo to In Progress in v0.2 Oct 29, 2021

hackerwins removed this from v0.2 Dec 14, 2021

hackerwins added this to v0.3 Dec 14, 2021

hackerwins moved this to Todo in v0.3 Dec 14, 2021

hackerwins unassigned hackerwins and blurfx Jul 11, 2022

hackerwins closed this as completed Jul 28, 2022

Repository owner moved this from Todo to Done in v0.3 Jul 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SDKs for various environments #54

SDKs for various environments #54

hackerwins commented May 23, 2020 •

edited

Loading

gedw99 commented Jun 2, 2020 •

edited

Loading

hackerwins commented Jun 3, 2020

gedw99 commented Jun 3, 2020 •

edited

Loading

gedw99 commented Jun 11, 2020

gedw99 commented Jun 11, 2020

joe-getcouragenow commented Jun 18, 2020

hackerwins commented Jun 18, 2020

joe-getcouragenow commented Jun 18, 2020 •

edited

Loading

hackerwins commented Jun 19, 2020

joe-getcouragenow commented Jun 20, 2020

joe-getcouragenow commented Jun 20, 2020

hackerwins commented Jun 22, 2020

hackerwins commented Jun 22, 2020

gedw99 commented Jun 22, 2020

hackerwins commented Jun 22, 2020

joe-getcouragenow commented Jun 22, 2020

joe-getcouragenow commented Jun 22, 2020

gedw99 commented Jun 22, 2020

joe-getcouragenow commented Jul 1, 2020

hackerwins commented Jul 31, 2021

gedw99 commented Aug 2, 2021

hackerwins commented Jul 28, 2022

SDKs for various environments #54

SDKs for various environments #54

Comments

hackerwins commented May 23, 2020 • edited Loading

gedw99 commented Jun 2, 2020 • edited Loading

hackerwins commented Jun 3, 2020

gedw99 commented Jun 3, 2020 • edited Loading

gedw99 commented Jun 11, 2020

gedw99 commented Jun 11, 2020

joe-getcouragenow commented Jun 18, 2020

hackerwins commented Jun 18, 2020

joe-getcouragenow commented Jun 18, 2020 • edited Loading

hackerwins commented Jun 19, 2020

joe-getcouragenow commented Jun 20, 2020

joe-getcouragenow commented Jun 20, 2020

hackerwins commented Jun 22, 2020

hackerwins commented Jun 22, 2020

gedw99 commented Jun 22, 2020

hackerwins commented Jun 22, 2020

joe-getcouragenow commented Jun 22, 2020

joe-getcouragenow commented Jun 22, 2020

gedw99 commented Jun 22, 2020

joe-getcouragenow commented Jul 1, 2020

hackerwins commented Jul 31, 2021

gedw99 commented Aug 2, 2021

hackerwins commented Jul 28, 2022

hackerwins commented May 23, 2020 •

edited

Loading

gedw99 commented Jun 2, 2020 •

edited

Loading

gedw99 commented Jun 3, 2020 •

edited

Loading

joe-getcouragenow commented Jun 18, 2020 •

edited

Loading