-
Notifications
You must be signed in to change notification settings - Fork 9.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Embedded etcd — not possible for my use case? #4545
Comments
@peterbourgon I see a few separate things from this issue.
We plan to support this. But not high priority for now, check #4435.
The discovery mechanism should not even be part of etcd.Server. We will move it out soon.
We are not going to change these two for etcd server interface. And etcdserver is the glue code, not the actual application state machine. This is what etcd designed for.
It definitely can. You should be able to reuse most of code in etcd: v3 API: https://github.com/coreos/etcd/blob/master/etcdserver/v3demo_server.go (use your own server) storage API: https://github.com/coreos/etcd/blob/master/storage/consistent_watchable_store.go#L50 (use your own storage) So my suggestion is that you should build your own transportation layer, and maybe your own storage layer. And hook them up with your own Server implementation. You can share the same raft, server API and kv API implementation with etcd. |
@peterbourgon BTW, can you share what is your use case in more detail? |
Thank you for that detailed reply. Happy to provide more detail of my own. I have this gossip library called mesh, it has many nice features I won't get in to here and exposes a specific gossipy API. On top of that I've built meshconn, which implements net.PacketConn (think of it as UDP) on top of mesh. So far so good. Now I want to build some component that will take the meshconn as input and provide the etcd API as output. Right now it's raft-on-mesh prototype code. The important part is transport.go, which is probably not the right name for what it does. That's where I raft.StartNode and have a loop to drive it. I have basic implementation for the Raft requirements, but now I want to push entries into my state machine. Right now I have a no-op implementation but in reality I want to create some coreos/etcd/something object, which implements this state machine interface (or something like it) and provides the etcd API (or something like it) on the other side. Because I like diagrams, here's one:
When two boxes are touching I expect them to know about and talk to each other, but otherwise not. So no connection between state machine and storage, for example. Maybe you can adjust the drawing if I got some expectation wrong ;) but this is what I expect. I'm trying to figure out what to plug into the "State machine" box. And given what you say here...
...I guess it is not EtcdServer. What is it?
I don't totally understand. Here you define some interfaces that mirror the V3 API (got that much) but you implement them on EtcdServer, which I can't use. This, plus...
...leaves me a little confused. What do you mean by "use your own server"? I am trying to avoid reimplementing the core etcd business logic, a.k.a. the state machine. Is that possible? EDIT: I had a sort of "a-ha" moment with contrib/raftexample. It doesn't work like I expected, but I think I understand it now. I'm digging in and will report back with my findings. |
I still want to reply since I got some extra time tonight :) Hope it can offer you more information anyway.
EtcdServer struct is more like an engine right now. It acts as transporter, API layer, and a applier that pulls committed entries out of raft to execute them.
Most of the implementation is not on etcdserver. https://github.com/coreos/etcd/blob/master/etcdserver/v3demo_server.go#L248-L643.
The core business are in the storage pkg and the API part I mentioned above. The state machine (i would like to call it the applier since we have a pull model) part is trivial really, it just parses the raft entry and let the executor to execute them. The actual executor is the storage.KV. |
OK. Thanks to @xiang90 help I got the basic stuff working :) I will have more issues with more questions in the future, but for now I am very happy! Thank you. |
This is a spiritual successor to #4531. Thanks to your tips, I'm able to connect a raft.Node to my net.PacketConn transport. Apparently, it needs a state machine which satisfies this interface:
My goal here is to provide the etcd API (v3) to my users, but in-process e.g. as a Go interface. (I am willing to do a gRPC connect to localhost, if necessary.) So, I expect etcdserver.Server to satisfy both of these things. But there are many options in the etcdserver.ServerConfig which I don't think I have answers for:
The fact that all of these options are ambiguously or not defined for my use case, leads me to believe that I'm doing something wrong. Or maybe that etcd can not be used in the way that I intend. Can you please give your opinion? Have I made some false assumptions? Is there another object I should be using? Or maybe I'm going about this the wrong way?
Thank you very much for your time!
The text was updated successfully, but these errors were encountered: