heartbeat to a ttl node

[YueHonghui]

Is there a interface to reset a ttl node timeout and not impact the modifyIndex, like a heartbeat but not notify watchers? I want to use this interface to implement a discovery service. The server start, set a node and ttl, before ttl timeout, it heartbeat to etcd to prevent ttl timeout. How can I do this?

[philips]

Create a directory with a TTL, put the key in that directory and heartbeat on the directory.

[philips]

@xiangli-cmu This is the right pattern, right? I need to add it to the new docs.

[YueHonghui]

I tested on v2.0-rc2 binary, errors occured.

set:curl -L http://10.5.20.79:4001/v2/keys/foo -XPUT -d value="1" -d ttl=50
rsp:{"action":"set","node":{"key":"/foo","value":"1","expiration":"2013-12-10T12:02:05.87680226+08:00","ttl":50,"modifiedIndex":3,"createdIndex":3}}
update1:curl -L http://10.5.20.79:4001/v2/keys/foo -XPUT -d ttl=50
rsp:{"action":"set","node":{"key":"/foo","prevValue":"1","dir":true,"expiration":"2013-12-10T12:02:16.909891696+08:00","ttl":50,"modifiedIndex":4,"createdIndex":4}}
update2:curl -L http://10.5.20.79:4001/v2/keys/foo -XPUT -d ttl=50
rsp:{"errorCode":102,"message":"Not A File","cause":"/foo","index":4}

The question is:

1. why the line update1 above change the node "foo" to a dir and increase the Index?
2. why the line update2 above throw an error?
   I just want to heartbeat to key foo and prevent it to expired, and has no change to modifyIndex.

[YueHonghui]

@philips Yes, that is what I want. And the heartbeat should not notify the watchers on that directory

[YueHonghui]

@philips
I have read #349 , and know why update1 change "foo" to a dir just now. And through http://news.gmane.org/gmane.comp.distributed.etcd , I notice that any modify(include same value assign) will increase the raft index. Now, how to implement this pattern?

There is a service named logger, and it have three servers(logger0, logger1, logger2) and many producers. Logger servers register at etcd through create dir with TTL under "v2/keys/logger/serverlist". For example, logger0 create dir "v2/keys/logger/serverlist/logger0" with TTL=30sec and put key "v2/keys/logger/serverlist/logger0/addr". Logger servers heartbeat to etcd to prevent expiration. At the same moment, all producers watch the key "v2/keys/logger/serverlist" for logger server create and delete.

Then, heartbeat should not notify the watchers on that directory, how to do that? What about to add filter to watch interface such as

curl -L "http://127.0.0.1:4001/v2/keys/logger/serverlist?wait=true&waitFilter=create.delete"

[philips]

@xiangli-cmu @benbjohnson Directories with TTLs are a little wonky right now:

The TTLs can't be refreshed:

$ curl -L http://127.0.0.1:4001/v2/keys/dir -XPUT -d ttl=30 -d dir=true
{"action":"set","node":{"key":"/dir","dir":true,"expiration":"2013-12-09T22:24:30.695515508-08:00","ttl":30,"modifiedIndex":16,"createdIndex":16}}%                                                                                             $ curl -L http://127.0.0.1:4001/v2/keys/dir -XPUT -d ttl=30 -d dir=true
{"errorCode":102,"message":"Not a file","cause":"/dir","index":16}

And a watch on a key in a directory that is expiring doesn't get an expire event:

$ curl -L http://127.0.0.1:4001/v2/keys/dir/file -XPUT -d value=asdf
{"action":"set","node":{"key":"/dir/file","value":"asdf","modifiedIndex":20,"createdIndex":20}}
$ curl -L http://127.0.0.1:4001/v2/keys/dir
{"action":"get","node":{"key":"/dir","dir":true,"expiration":"2013-12-09T22:25:53.221327498-08:00","ttl":1,"nodes":[{"key":"/dir/file","value":"asdf","modifiedIndex":20,"createdIndex":20}],"modifiedIndex":19,"createdIndex":19}}
$ curl -X GET http://127.0.0.1:4001/v2/keys/dir/file\?consistent\=true\&wait\=true
...hangs forever...

[xiang90]

@philips
For the first problem, PUT without a flag means to create or replace. And we do not allow to replace a directory.
To update the ttl, you need to do

$curl -L http://127.0.0.1:4001/v2/keys/dir?prevExist=true -XPUT -d ttl=100

[philips]

@xiangli-cmu Ah, duh. I will make sure to add that to the docs.

[xiang90]

@philips I am solving the second problem now. You can expect a pr very soon.
And let me know what you think about the first one.

[xiang90]

@YueHonghui To address your problem (do not notify watcher when a update happened)

1. Write a customized filter to filter out update ttl event by your client.
2. add waitTypes to wait operation to help filter unwanted notification at server side.
   What do you think @philips @benbjohnson

[benbjohnson]

@xiangli-cmu I'd rather not complicate the server side by adding something like waitTypes. I think that's a rabbit hole to try to manage all kinds of filters like that. Also, it seems like a simple enough thing to filter out client side.

[xiang90]

@benbjohnson Agree. I will close this issue.

[defunctzombie]

@philips why would one heartbeat on the directory and not a node within the directory? Lets say I have a "services" directory and I put ttl nodes in it. I would want to update the ttl on each individual node not the entire directory.

[philips]

@defunctzombie It is a bit hacky but you can have something like:

/service/a/config
/service/b/config

And then heartbeat the directories instead of the config.

It isn't ideal and I could see adding an API for this but I don't know what it should look like. Perhaps it is something like:

PUT /v2/keys/foobar?ttl=30&refresh=true

Then this request would not update the etcd-index of the key. Could you file a new bug to discuss this sort of API?

[defunctzombie]

Issue discussion continued in #1232