How to launch a vineyard cluster ?

[Zarca]

Hi,
I'm puzzled about how the user can launch a vineyard cluster , Should i configure and start the etcd cluster firstly all
by myself ?
Is there any step or configurations i must pass to the libvineyard?

[sighingnow]

On kubernetes, I think you could launch a distributed vineyard cluster using helm, by following the instructions in README.

Without kubernetes, launching a vineyard cluster requires launch a etcd server first, then pass a --etcd_endpoint arguments to vineyardd. You could find all useful command line arguments by vineyardd --help.

Prebuilt vineyardd binary is available via Dockerhub libvineyard/vineyardd. For linux users, a daily built standalone binary is available at https://github.com/alibaba/libvineyard/releases/tag/latest.

[Zarca]

Hi, sighingnow
Actually, I'm using the vineyard without a kubernetes in deployment,
So, dose the etcd cluster play as a service registration for the vineyard's cluster,
and in the mean time, it also keep the metadata of the vineyard instance ?

[sighingnow]

In vineyard etcd cluster play as a service registration (aka. vineyard instance registartion), as well as global metadata synchronization.

Most of the objects in vineyard are "local objects", which means the object may be need to visible for clients on other nodes. But for global objects, e.g., a distributed tensor or a paritioned graph, we sync the metadata to etcd and other vineyard server could see such objects by watching the change stream from etcd.

[Zarca]

Sorry, I did not get it , what's the difference between the global object and the local object?
it seems the latest doc only support C++ API on this type of global objects...
I suppose that the user still can't get a remote object from remote vineyard instance without the communicator's support(you refer in issue #152 ) for data migration, even it is a global object.

[sighingnow]

1. Local vs. Global object

   • Local objects are object whose blobs are all available on the local machine.
   • Global objects are object whose blobs cannot (or not) be fit in to a single machine, e.g., a large graph may be partitioned to many workers. Conceptly these fragments construct a graph.

   When sharing a graph between applications (e.g., different graph computation engines), you just need to share the graph (whose metadata contains graph schema, partition info, etc.), you don't need to shared a list of fragments, and an extra map to describe the partition info. The graph is a standalone object, a global object.

2. Without explicit migrate you cannot obtain data (here means blobs) from a remote vineyard server, since there's no RPC data accessing in vineyard. However you could still inspect the metadata, which could helpful for application to know to organized the next steps better (e.g., set a proper parallel threads number).

[Zarca]

Okay, i see.
So the global object is just a set of distributed blobs across the cluster and be treated like a whole one conceptly,
This works when we can deploy parallel tasks without any data exchange among different nodes.
But in some cases, maybe the different nodes should exchange their own data (e.g. aggregation or group operations based on data's hash key ), will the communicator in the future release support it?

[sighingnow]

maybe the different nodes should exchange their own data (e.g. aggregation or group operations based on data's hash key )

We choose to leave such stuffs to application itself, or at least drivers. You know, users may have many different options to shuffle their data, e.g., HTTP, tcp, RDMA, MPI, or other customized logic. They all have pros as well as cons in different environments and vineyard cannot fit all.

The "aggregation or group operations based on data's hash key" won't be implmented in vineyard since vineyard itself is not a key-value storage. And, you konw, different users may want to hash on different part of the object with different hash functions, and then want to use different migration strategies after hashing. That is too application-specfic or problem-specific. Vineyard chooses to provide a pluggable design for users to implement and select such kind of drivers.

[sighingnow]

I think the tab "discussions" is a more suitable place for this thread :)

[sighingnow]

the client will be able to get the data's replication inside local vineyard instance.

Absolutely right.

it will also cause some vineyard instance got redundant data's replication.

The replication of data could be deleted by users with the client, since the migration itself is also an explicitly request.

BTW I'm wondering if you have some specific scenario that vineyard could fit (or partially fit). You could open a new thread of discussion about the problem, or if it isn't suitable for public discussion, you could drop an email to community@v6d.io and we would see if vineyard could help for your case.

We eagerly want to listen from the community for feedback! And you may find more discussion about vineyard at https://news.ycombinator.com/item?id=25832831

[Zarca]

Yep, i intended to enhance the apache arrow's plasma by implementing the functionality of remote data access , which is very similar to(though not exactly the same ) the libvineyard's communicator's data migration (in your future release). So, if the libvineyard will provide that , think i will not need to do it by myself now.

[Zarca]

BTW, will vineyard (or plan to) supply more interfaces for other popular language (e.g. Java ) in future release?

[sighingnow]

Yep, i intended to enhance the apache arrow's plasma by implementing the functionality of remote data access , which is very similar to(though not exactly the same ) the libvineyard's communicator's data migration (in your future release). So, if the libvineyard will provide that , think i will not need to do it by myself now.

Thanks for the information. The application (your program in this case) could invoke the migration API to reuse the routines provided by vineyard to fetch remote object to local.

BTW, will vineyard (or plan to) supply more interfaces for other popular language (e.g. Java ) in future release?

Yes, there's definitely a plan, and Java, Rust, Golang are in our todo list.

[Zarca]

Cool, I do really appreciate it and gonna looking forward to it's future releases