Feature: add state machine range and subscribe API

[lichuang]

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

Feature: add state machine range and subscribe API

Changelog

• New Feature

Related Issues

Fixes #issue #4607

Test Plan

Unit Tests

Stateless Tests

[vercel]

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/databend/databend/FWxLPa2NLSSmjcMZtaDhZctyyjSG
✅ Preview: https://databend-git-fork-lichuang-addsmrangesubscribeapi-databend.vercel.app

[Deployment for 281b5b2 canceled]

[mergify]

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

[drmingdrmer]

Why do we need range APIs for meta datas?

[lichuang]

Why do we need range APIs for meta datas?

in etcd watch API, see https://etcd.io/docs/v3.2/learning/api/#watch-api, they have the key range:

message WatchCreateRequest { bytes key = 1; bytes range_end = 2; // ... }

[drmingdrmer]

in etcd watch API, see https://etcd.io/docs/v3.2/learning/api/#watch-api, they have the key range:

message WatchCreateRequest { bytes key = 1; bytes range_end = 2; // ... }

On the doc page: Key, Range_End - The key range to watch..

If several given events happened as the following:

z:       z=3    
y:                     y=2
x: x=1         x=0
---+-----+-----+-------+------------------------------> time(revision)
   t1    t2    t3      t4

A watch request for key range [y,z], from t3, should only return event y=2 at t4.
In another word, to collect response data for watch, metasrv iterates all events in time order, then filters them by key range.
It does not need to iterate over the key range.

[lichuang]

in etcd watch API, see https://etcd.io/docs/v3.2/learning/api/#watch-api, they have the key range:
message WatchCreateRequest { bytes key = 1; bytes range_end = 2; // ... }

On the doc page: Key, Range_End - The key range to watch..

If several given events happened as the following:

z:       z=3    
y:                     y=2
x: x=1         x=0
---+-----+-----+-------+------------------------------> time(revision)
   t1    t2    t3      t4

A watch request for key range [y,z], from t3, should only return event y=2 at t4. In another word, to collect response data for watch, metasrv iterates all events in time order, then filters them by key range. It does not need to iterate over the key range.

I wonder if or not return the current event at the first time when a watcher has created, but in etcd doc and code, seems they do not return event when create watcher.ok, i delete the range API.

[drmingdrmer]

@lichuang
Is this ready to review or are you gonna cleanup the commits?

[drmingdrmer]

@lichuang
Is this ready to review or are you gonna cleanup the commits?

[lichuang]

Is this ready to review or are you gonna cleanup the commits?

ready

[drmingdrmer]

I wonder if or not return the current event at the first time when a watcher has created, but in etcd doc and code, seems they do not return event when create watcher.ok, i delete the range API.

The range APIs are still there.

BTW, another issue is that range-API is not snapshot-based. The result may be inconsistent.

[lichuang]

I wonder if or not return the current event at the first time when a watcher has created, but in etcd doc and code, seems they do not return event when create watcher.ok, i delete the range API.

The range APIs are still there.

BTW, another issue is that range-API is not snapshot-based. The result may be inconsistent.

sad, when rebase source it does not remove the range API...

[drmingdrmer]

LGTM