diff --git a/.gitignore b/.gitignore index aebb84829d7..a080789eff0 100644 --- a/.gitignore +++ b/.gitignore @@ -1,10 +1,6 @@ src/ pkg/ /etcd +/server/release_version.go /go-bindata -release_version.go /machine* -.vagrant/ -conf -info -log diff --git a/CHANGELOG b/CHANGELOG new file mode 100644 index 00000000000..351c41823f4 --- /dev/null +++ b/CHANGELOG @@ -0,0 +1,33 @@ +v0.2 +* Support directory creation and removal. +* Add Compare-and-Swap (CAS) support. +* Support recursive GETs. +* Support fully consistent GETs. +* Allow clients to watch specific paths. +* Allow clients to watch for key expiration. +* Unique key generation. +* Support hidden paths. +* Refactor low-level data store. +* Modularize store, server and API code. +* Integrate Gorilla Web Toolkit. +* Add tiered configuration (command line args, env variables, config file). +* Add peer protocol versioning. +* Add rolling upgrade support for future versions. +* Sync key expiration across cluster. +* Significantly improve test coverage. +* Improve migration testing. +* Configurable snapshot count. +* Reduce TCP connection count. +* Fix TCP connection leak. +* Bug Fixes: https://github.com/coreos/etcd/issues?milestone=1&state=closed + +Contributors: +* Xiang Li (@xiangli-cmu) +* Ben Johnson (@benbjohnson) +* Brandon Philips (@philips) +* Yifan (@yifan-gu) +* Rob Szumski +* Hongchao Deng (@fengjingchao) +* Kelsey Hightower (@kelseyhightower) +* Adrián (@adrianlzt) +* Antonio Terreno (@aterreno) diff --git a/Documentation/errorcode.md b/Documentation/errorcode.md new file mode 100644 index 00000000000..3a0443f8bc6 --- /dev/null +++ b/Documentation/errorcode.md @@ -0,0 +1,58 @@ +Error Code +====== + +This document describes the error code in **Etcd** project. + +It's categorized into four groups: + +- Command Related Error +- Post Form Related Error +- Raft Related Error +- Etcd Related Error + +Error code corresponding strerror +------ + + const ( + EcodeKeyNotFound = 100 + EcodeTestFailed = 101 + EcodeNotFile = 102 + EcodeNoMoreMachine = 103 + EcodeNotDir = 104 + EcodeNodeExist = 105 + EcodeKeyIsPreserved = 106 + + EcodeValueRequired = 200 + EcodePrevValueRequired = 201 + EcodeTTLNaN = 202 + EcodeIndexNaN = 203 + + EcodeRaftInternal = 300 + EcodeLeaderElect = 301 + + EcodeWatcherCleared = 400 + EcodeEventIndexCleared = 401 + ) + + // command related errors + errors[100] = "Key Not Found" + errors[101] = "Test Failed" //test and set + errors[102] = "Not A File" + errors[103] = "Reached the max number of machines in the cluster" + errors[104] = "Not A Directory" + errors[105] = "Already exists" // create + errors[106] = "The prefix of given key is a keyword in etcd" + + // Post form related errors + errors[200] = "Value is Required in POST form" + errors[201] = "PrevValue is Required in POST form" + errors[202] = "The given TTL in POST form is not a number" + errors[203] = "The given index in POST form is not a number" + + // raft related errors + errors[300] = "Raft Internal Error" + errors[301] = "During Leader Election" + + // etcd related errors + errors[400] = "watcher is cleared due to etcd recovery" + errors[401] = "The event in requested index is outdated and cleared" diff --git a/Documentation/etcd-file-system.md b/Documentation/etcd-file-system.md new file mode 100644 index 00000000000..092ecdee642 --- /dev/null +++ b/Documentation/etcd-file-system.md @@ -0,0 +1,101 @@ +#Etcd File System + +## Structure +[TODO] +![alt text](./img/etcd_fs_structure.jpg "etcd file system structure") + +## Node +In **Etcd**, the **Node** is the rudimentary element constructing the whole. +Currently **Etcd** file system is comprised in a Unix-like way of files and directories, and they are two kinds of nodes different in: + +- **File Node** has data associated with it. +- **Directory Node** has children nodes associated with it. + +Besides the file and directory difference, all nodes have common attributes and operations as follows: + +### Attributes: +- **Expiration Time** [optional] + + The node will be deleted when it expires. + +- **ACL** + + The path of access control list of the node. + +### Operation: +- **Get** (path, recursive, sorted) + + Get the content of the node + - If the node is a file, the data of the file will be returned. + - If the node is a directory, the child nodes of the directory will be returned. + - If recursive is true, it will recursively get the nodes of the directory. + - If sorted is true, the result will be sorted based on the path. + +- **Create** (path, value[optional], ttl [optional]) + + Create a file. Create operation will help to create intermediate directories with no expiration time. + - If the file already exists, create will fail. + - If the value is given, set will create a file. + - If the value is not given, set will crate a directory. + - If ttl is given, the node will be deleted when it expires. + +- **Update** (path, value[optional], ttl [optional]) + + Update the content of the node. + - If the value is given, the value of the key will be updated. + - If ttl is given, the expiration time of the node will be updated. + +- **Delete** (path, recursive) + + Delete the node of given path. + - If the node is a directory: + - If recursive is true, the operation will delete all nodes under the directory. + - If recursive is false, error will be returned. + +- **TestAndSet** (path, prevValue [prevIndex], value, ttl) + + Atomic *test and set* value to a file. If test succeeds, this operation will change the previous value of the file to the given value. + - If the prevValue is given, it will test against previous value of + the node. + - If the prevValue is empty, it will test if the node is not existing. + - If the prevValue is not empty, it will test if the prevValue is equal to the current value of the file. + - If the prevIndex is given, it will test if the create/last modified index of the node is equal to prevIndex. + +- **Renew** (path, ttl) + + Set the node's expiration time to (current time + ttl) + +## ACL + +### Theory +Etcd exports a Unix-like file system interface consisting of files and directories, collectively called nodes. +Each node has various meta-data, including three names of access control lists used to control reading, writing and changing (change ACL names for the node). + +We are storing the ACL names for nodes under a special *ACL* directory. +Each node has ACL name corresponding to one file within *ACL* dir. +Unless overridden, a node naturally inherits the ACL names of its parent directory on creation. + +For each ACL name, it has three children: *R (Reading)*, *W (Writing)*, *C (Changing)* + +Each permission is also a node. Under the node it contains the users who have this permission for the file refering to this ACL name. + +### Example +[TODO] +### Diagram +[TODO] + +### Interface + +Testing permissions: + +- (node *Node) get_perm() +- (node *Node) has_perm(perm string, user string) + +Setting/Changing permissions: + +- (node *Node) set_perm(perm string) +- (node *Node) change_ACLname(aclname string) + + +## User Group +[TODO] diff --git a/Documentation/external-documentation.md b/Documentation/external-documentation.md new file mode 100644 index 00000000000..a7d957aefdc --- /dev/null +++ b/Documentation/external-documentation.md @@ -0,0 +1,104 @@ +# Etcd Configuration + +Configuration options can be set in three places: + + 1. Command line flags + 2. Environment variables + 3. Configuration file + +Options set on the command line take precedence over all other sources. +Options set in environment variables take precedence over options set in +configuration files. + +## Command Line Flags + +### Required + +* `-n` - The node name. Defaults to `default-name`. + +### Optional + +* `-c` - The advertised public hostname:port for client communication. Defaults to `127.0.0.1:4001`. +* `-cl` - The listening hostname for client communication. Defaults to advertised ip. +* `-C` - A comma separated list of machines in the cluster (i.e `"203.0.113.101:7001,203.0.113.102:7001"`). +* `-CF` - The file path containing a comma separated list of machines in the cluster. +* `-clientCAFile` - The path of the client CAFile. Enables client cert authentication when present. +* `-clientCert` - The cert file of the client. +* `-clientKey` - The key file of the client. +* `-configfile` - The path of the etcd config file. Defaults to `/etc/etcd/etcd.conf`. +* `-cors` - A comma separated white list of origins for cross-origin resource sharing. +* `-cpuprofile` - The path to a file to output cpu profile data. Enables cpu profiling when present. +* `-d` - The directory to store log and snapshot. Defaults to the current working directory. +* `-m` - The max size of result buffer. Defaults to `1024`. +* `-maxsize` - The max size of the cluster. Defaults to `9`. +* `-r` - The max retry attempts when trying to join a cluster. Defaults to `3`. +* `-s` - The advertised public hostname:port for server communication. Defaults to `127.0.0.1:7001`. +* `-sl` - The listening hostname for server communication. Defaults to advertised ip. +* `-serverCAFile` - The path of the CAFile. Enables client/peer cert authentication when present. +* `-serverCert` - The cert file of the server. +* `-serverKey` - The key file of the server. +* `-snapshot` - Open or close snapshot. Defaults to `false`. +* `-v` - Enable verbose logging. Defaults to `false`. +* `-vv` - Enable very verbose logging. Defaults to `false`. +* `-version` - Print the version and exit. +* `-w` - The hostname:port of web interface. + +## Configuration File + +The etcd configuration file is written in [TOML](https://github.com/mojombo/toml) +and read from `/etc/etcd/etcd.conf` by default. + +```TOML +advertised_url = "127.0.0.1:4001" +ca_file = "" +cert_file = "" +cors = [] +cpu_profile_file = "" +datadir = "." +key_file = "" +listen_host = "127.0.0.1:4001" +machines = [] +machines_file = "" +max_cluster_size = 9 +max_result_buffer = 1024 +max_retry_attempts = 3 +name = "default-name" +snapshot = false +verbose = false +very_verbose = false +web_url = "" + +[peer] +advertised_url = "127.0.0.1:7001" +ca_file = "" +cert_file = "" +key_file = "" +listen_host = "127.0.0.1:7001" +``` + +## Environment Variables + + * `ETCD_ADVERTISED_URL` + * `ETCD_CA_FILE` + * `ETCD_CERT_FILE` + * `ETCD_CORS` + * `ETCD_CONFIG_FILE` + * `ETCD_CPU_PROFILE_FILE` + * `ETCD_DATADIR` + * `ETCD_KEY_FILE` + * `ETCD_LISTEN_HOST` + * `ETCD_MACHINES` + * `ETCD_MACHINES_FILE` + * `ETCD_MAX_RETRY_ATTEMPTS` + * `ETCD_MAX_CLUSTER_SIZE` + * `ETCD_MAX_RESULT_BUFFER` + * `ETCD_NAME` + * `ETCD_SNAPSHOT` + * `ETCD_VERBOSE` + * `ETCD_VERY_VERBOSE` + * `ETCD_WEB_URL` + * `ETCD_PEER_ADVERTISED_URL` + * `ETCD_PEER_CA_FILE` + * `ETCD_PEER_CERT_FILE` + * `ETCD_PEER_KEY_FILE` + * `ETCD_PEER_LISTEN_HOST` diff --git a/Documentation/img/etcd_fs_structure.jpg b/Documentation/img/etcd_fs_structure.jpg new file mode 100644 index 00000000000..51140caa048 Binary files /dev/null and b/Documentation/img/etcd_fs_structure.jpg differ diff --git a/README.md b/README.md index 183428ea99f..bd5f99eb40c 100644 --- a/README.md +++ b/README.md @@ -1,26 +1,26 @@ # etcd -README version 0.1.0 + +README version 0.2.0 [![Build Status](https://travis-ci.org/coreos/etcd.png)](https://travis-ci.org/coreos/etcd) -A highly-available key value store for shared configuration and service discovery. etcd is inspired by zookeeper and doozer, with a focus on: +A highly-available key value store for shared configuration and service discovery. +etcd is inspired by zookeeper and doozer, with a focus on: * Simple: curl'able user facing API (HTTP+JSON) * Secure: optional SSL client cert authentication * Fast: benchmarked 1000s of writes/s per instance * Reliable: Properly distributed using Raft -Etcd is written in Go and uses the [raft][raft] consensus algorithm to manage a highly-available replicated log. +Etcd is written in Go and uses the [Raft][raft] consensus algorithm to manage a highly-available replicated log. -See [etcdctl][etcdctl] for a simple command line client. Or feel free to just use curl, as in the examples below. +See [etcdctl][etcdctl] for a simple command line client. +Or feel free to just use curl, as in the examples below. [raft]: https://github.com/coreos/go-raft [etcdctl]: http://coreos.com/docs/etcdctl/ -## Contact -- Mailing list: http://coreos.com/lists/etcd-dev/ -- IRC: #coreos on irc.freenode.net ## Getting Started @@ -30,6 +30,7 @@ The latest release is available as a binary at [Github][github-release]. [github-release]: https://github.com/coreos/etcd/releases/ + ### Building You can build etcd from source: @@ -48,9 +49,11 @@ _NOTE_: you need go 1.1+. Please check your installation with go version ``` + ### Running a single node -These examples will use a single node cluster to show you the basics of the etcd REST API. Lets start etcd: +These examples will use a single node cluster to show you the basics of the etcd REST API. +Let's start etcd: ```sh ./etcd -d node0 -n node0 @@ -60,249 +63,328 @@ This will bring up an etcd node listening on port 4001 for client communication The `-d node0` argument tells etcd to write node configuration, logs and snapshots to the `./node0/` directory. The `-n node0` tells the rest of the cluster that this node is named node0. + + ## Usage ### Setting the value to a key -Let’s set the first key-value pair to the node. In this case the key is `/message` and the value is `Hello world`. +Let’s set the first key-value pair to the node. +In this case the key is `/message` and the value is `Hello world`. ```sh -curl -L http://127.0.0.1:4001/v1/keys/message -d value="Hello world" +curl -L http://127.0.0.1:4001/v2/keys/message -X PUT -d value="Hello world" ``` ```json -{"action":"SET","key":"/message","value":"Hello world","newKey":true,"index":3} +{"action":"set","key":"/message","value":"Hello world","modifiedIndex":2} ``` -This response contains five fields. We will introduce three more fields as we try more commands. +This response contains four fields. +We will introduce three more fields as we try more commands. -1. The action of the request; we set the value via a POST request, thus the action is `SET`. +1. The action of the request; we set the value via a `PUT` request, thus the action is `set`. -2. The key of the request; we set `/message` to `Hello world!`, so the key field is `/message`. -Notice we use a file system like structure to represent the key-value pairs. So each key starts with `/`. +2. The key of the request; we set `/message` to `Hello world`, so the key field is `/message`. +We use a file system like structure to represent the key-value pairs so each key starts with `/`. 3. The current value of the key; we set the value to`Hello world`. -4. If we set a new key; `/message` did not exist before, so this is a new key. +4. Modified Index is a unique, monotonically incrementing index created for each change to etcd. +Requests that change the index include `set`, `delete`, `update`, `create` and `compareAndSwap`. +Since the `get` and `watch` commands do not change state in the store, they do not change the index. +You may notice that in this example the index is `2` even though it is the first request you sent to the server. +This is because there are internal commands that also change the state like adding and syncing servers. -5. Index is the unique internal log index of the set request. Requests that change the log index include `SET`, `DELETE` and `TESTANDSET`. The `GET`, `LIST` and `WATCH` commands do not change state in the store and so they do not change the index. You may notice that in this example the index is 3, although it is the first request you sent to the server. This is because there are internal commands that also change the state like adding and syncing servers. ### Get the value of a key -Get the value that we just set in `/message` by issuing a GET: +We can get the value that we just set in `/message` by issuing a `GET` request: ```sh -curl -L http://127.0.0.1:4001/v1/keys/message +curl -L http://127.0.0.1:4001/v2/keys/message ``` ```json -{"action":"GET","key":"/message","value":"Hello world","index":3} +{"action":"get","key":"/message","value":"Hello world","modifiedIndex":2} ``` -### Change the value of a key -Change the value of `/message` from `Hello world` to `Hello etcd` with another POST to the key: + +### Changing the value of a key + +You can change the value of `/message` from `Hello world` to `Hello etcd` with another `PUT` request to the key: ```sh -curl -L http://127.0.0.1:4001/v1/keys/message -d value="Hello etcd" +curl -L http://127.0.0.1:4001/v1/keys/message -XPUT -d value="Hello etcd" ``` ```json -{"action":"SET","key":"/message","prevValue":"Hello world","value":"Hello etcd","index":4} +{"action":"set","key":"/message","prevValue":"Hello world","value":"Hello etcd","index":3} ``` -Notice that the `prevValue` is set to `Hello world`. +Notice that the `prevValue` is set to the previous value of the key - `Hello world`. +It is useful when you want to atomically set a value to a key and get its old value. + -### Delete a key +### Deleting a key -Remove the `/message` key with a DELETE: +You can remove the `/message` key with a `DELETE` request: ```sh -curl -L http://127.0.0.1:4001/v1/keys/message -X DELETE +curl -L http://127.0.0.1:4001/v2/keys/message -XDELETE ``` ```json -{"action":"DELETE","key":"/message","prevValue":"Hello etcd","index":5} +{"action":"delete","key":"/message","prevValue":"Hello etcd","modifiedIndex":4} ``` + ### Using key TTL -Keys in etcd can be set to expire after a specified number of seconds. That is done by setting a TTL (time to live) on the key when you POST: +Keys in etcd can be set to expire after a specified number of seconds. +You can do this by setting a TTL (time to live) on the key when send a `PUT` request: ```sh -curl -L http://127.0.0.1:4001/v1/keys/foo -d value=bar -d ttl=5 +curl -L http://127.0.0.1:4001/v2/keys/foo -XPUT -d value=bar -d ttl=5 ``` ```json -{"action":"SET","key":"/foo","value":"bar","newKey":true,"expiration":"2013-07-11T20:31:12.156146039-07:00","ttl":4,"index":6} +{"action":"set","key":"/foo","value":"bar","expiration":"2013-11-12T20:21:22.629352334-05:00","ttl":5,"modifiedIndex":5} ``` -Note the last two new fields in response: +Note the two new fields in response: + +1. The `expiration` is the time that this key will expire and be deleted. -1. The expiration is the time that this key will expire and be deleted. +2. The `ttl` is the time to live for the key, in seconds. -2. The ttl is the time to live of the key. +_NOTE_: Keys can only be expired by a cluster leader so if a node gets disconnected from the cluster, its keys will not expire until it rejoins. -Now you can try to get the key by sending: +Now you can try to get the key by sending a `GET` request: ```sh -curl -L http://127.0.0.1:4001/v1/keys/foo +curl -L http://127.0.0.1:4001/v2/keys/foo ``` If the TTL has expired, the key will be deleted, and you will be returned a 100. ```json -{"errorCode":100,"message":"Key Not Found","cause":"/foo"} +{"errorCode":100,"message":"Key Not Found","cause":"/foo","index":6} ``` -### Watching a prefix -We can watch a path prefix and get notifications if any key change under that prefix. +### Waiting for a change -In one terminal, we send a watch request: +We can watch for a change on a key and receive a notification by using long polling. +This also works for child keys by passing `recursive=true` in curl. + +In one terminal, we send a get request with `wait=true` : ```sh -curl -L http://127.0.0.1:4001/v1/watch/foo +curl -L http://127.0.0.1:4001/v2/keys/foo?wait=true ``` -Now, we are watching at the path prefix `/foo` and wait for any changes under this path. +Now we are waiting for any changes at path `/foo`. -In another terminal, we set a key `/foo/foo` to `barbar` to see what will happen: +In another terminal, we set a key `/foo` with value `bar`: ```sh -curl -L http://127.0.0.1:4001/v1/keys/foo/foo -d value=barbar +curl -L http://127.0.0.1:4001/v2/keys/foo -XPUT -d value=bar ``` The first terminal should get the notification and return with the same response as the set request. ```json -{"action":"SET","key":"/foo/foo","value":"barbar","newKey":true,"index":7} +{"action":"set","key":"/foo","value":"bar","modifiedIndex":7} ``` -However, the watch command can do more than this. Using the the index we can watch for commands that has happened in the past. This is useful for ensuring you don't miss events between watch commands. +However, the watch command can do more than this. +Using the the index we can watch for commands that has happened in the past. +This is useful for ensuring you don't miss events between watch commands. -Let's try to watch for the set command of index 6 again: +Let's try to watch for the set command of index 7 again: ```sh -curl -L http://127.0.0.1:4001/v1/watch/foo -d index=6 +curl -L http://127.0.0.1:4001/v2/keys/foo?wait=true\&waitIndex=7 ``` The watch command returns immediately with the same response as previous. -### Atomic Test and Set -Etcd can be used as a centralized coordination service in a cluster and `TestAndSet` is the most basic operation to build distributed lock service. This command will set the value only if the client provided `prevValue` is equal the current key value. +### Atomic Compare-and-Swap (CAS) + +Etcd can be used as a centralized coordination service in a cluster and `CompareAndSwap` is the most basic operation to build distributed lock service. + +This command will set the value of a key only if the client-provided conditions are equal to the current conditions. + +The current comparable conditions are: + +1. `prevValue` - checks the previous value of the key. -Here is a simple example. Let's create a key-value pair first: `foo=one`. +2. `prevIndex` - checks the previous index of the key. + +3. `prevExist` - checks existence of the key: if `prevExist` is true, it is a `update` request; if prevExist is `false`, it is a `create` request. + +Here is a simple example. +Let's create a key-value pair first: `foo=one`. ```sh -curl -L http://127.0.0.1:4001/v1/keys/foo -d value=one +curl -L http://127.0.0.1:4001/v1/keys/foo -XPUT -d value=one ``` -Let's try an invalid `TestAndSet` command. -We can give another parameter prevValue to set command to make it a TestAndSet command. +Let's try an invalid `CompareAndSwap` command first. +We can provide the `prevValue` parameter to the set command to make it a `CompareAndSwap` command. ```sh -curl -L http://127.0.0.1:4001/v1/keys/foo -d prevValue=two -d value=three +curl -L http://127.0.0.1:4001/v1/keys/foo?prevValue=two -XPUT -d value=three ``` -This will try to test if the previous of the key is two, it is change it to three. +This will try to compare the previous value of the key and the previous value we provided. If they are equal, the value of the key will change to three. ```json -{"errorCode":101,"message":"The given PrevValue is not equal to the value of the key","cause":"TestAndSet: one!=two"} +{"errorCode":101,"message":"Test Failed","cause":"[two != one] [0 != 8]","index":8} ``` -which means `testAndSet` failed. +which means `CompareAndSwap` failed. -Let us try a valid one. +Let's try a valid condition: ```sh -curl -L http://127.0.0.1:4001/v1/keys/foo -d prevValue=one -d value=two +curl -L http://127.0.0.1:4001/v2/keys/foo?prevValue=one -XPUT -d value=two ``` The response should be ```json -{"action":"SET","key":"/foo","prevValue":"one","value":"two","index":10} +{"action":"compareAndSwap","key":"/foo","prevValue":"one","value":"two","modifiedIndex":9} ``` -We successfully changed the value from “one” to “two”, since we give the correct previous value. +We successfully changed the value from “one” to “two” since we gave the correct previous value. -To set a key to a given value only if it does not exist, simply supply an empty prevValue parameter. + +### Listing a directory + +In etcd we can store two types of things: keys and directories. +Keys store a single string value. +Directories store a set of keys and/or other directories. + +In this example, let's first create some keys: + +We already have `/foo=two` so now we'll create another one called `/foo_dir/foo` with the value of `bar`: ```sh -curl -L http://127.0.0.1:4001/v1/keys/bar -d prevValue= -d value=four +curl -L http://127.0.0.1:4001/v2/keys/foo_dir/foo -XPUT -d value=bar ``` -Since the key "bar" does not exist, the response should be - ```json -{"action":"SET","key":"/bar","value":"four","newKey":true,"index":11} +{"action":"set","key":"/foo_dir/foo","value":"bar","modifiedIndex":10} ``` -However, using a empty prevValue with a key that does exist will fail. +Now we can list the keys under root `/`: ```sh -curl -L http://127.0.0.1:4001/v1/keys/bar -d prevValue= -d value=five +curl -L http://127.0.0.1:4001/v2/keys/ ``` -will result in +We should see the response as an array of items: ```json -{"errorCode":101,"message":"The given PrevValue is not equal to the value of the key","cause":"TestAndSet: four!="} +{"action":"get","key":"/","dir":true,"kvs":[{"key":"/foo","value":"two","modifiedIndex":9},{"key":"/foo_dir","dir":true,"modifiedIndex":10}],"modifiedIndex":0} ``` -### Listing a directory +Here we can see `/foo` is a key-value pair under `/` and `/foo_dir` is a directory. +We can also recursively get all the contents under a directory by adding `recursive=true`. + +```sh +curl -L http://127.0.0.1:4001/v2/keys/?recursive=true +``` + +```json +{"action":"get","key":"/","dir":true,"kvs":[{"key":"/foo","value":"two","modifiedIndex":9},{"key":"/foo_dir","dir":true,"kvs":[{"key":"/foo_dir/foo","value":"bar","modifiedIndex":10}],"modifiedIndex":10}],"modifiedIndex":0} +``` -Last we provide a simple List command to list all the keys under a prefix path. -Let us create some keys first. +### Deleting a directory -We already have `/foo/foo=barbar` +Now let's try to delete the directory `/foo_dir`. -We create another one `/foo/foo_dir/foo=barbarbar` +To delete a directory, we must add `recursive=true`. ```sh -curl -L http://127.0.0.1:4001/v1/keys/foo/foo_dir/bar -d value=barbarbar +curl -L http://127.0.0.1:4001/v2/keys/foo_dir?recursive=true -XDELETE +``` + +```json +{"action":"delete","key":"/foo_dir","dir":true,"modifiedIndex":11} ``` -Now list the keys under `/foo` + +### Creating a hidden node + +We can create a hidden key-value pair or directory by add a `_` prefix. +The hidden item will not be listed when sending a `GET` request for a directory. + +First we'll add a hidden key named `/_message`: ```sh -curl -L http://127.0.0.1:4001/v1/keys/foo/ +curl -L http://127.0.0.1:4001/v2/keys/_message -XPUT -d value="Hello hidden world" ``` -We should see the response as an array of items +```json +{"action":"set","key":"/_message","value":"Hello hidden world","modifiedIndex":12} +``` + + +Next we'll add a regular key named `/message`: + +```sh +curl -L http://127.0.0.1:4001/v2/keys/message -XPUT -d value="Hello world" +``` + +```json +{"action":"set","key":"/message","value":"Hello world","modifiedIndex":13} +``` + +Now let's try to get a listing of keys under the root directory, `/`: + +```sh +curl -L http://127.0.0.1:4001/v2/keys/ +``` ```json -[{"action":"GET","key":"/foo/foo","value":"barbar","index":10},{"action":"GET","key":"/foo/foo_dir","dir":true,"index":10}] +{"action":"get","key":"/","dir":true,"kvs":[{"key":"/foo","value":"two","modifiedIndex":9},{"key":"/message","value":"Hello world","modifiedIndex":13}],"modifiedIndex":0} ``` -which meas `foo=barbar` is a key-value pair under `/foo` and `foo_dir` is a directory. +Here we see the `/message` key but our hidden `/_message` key is not returned. + ## Advanced Usage ### Transport security with HTTPS -Etcd supports SSL/TLS and client cert authentication for clients to server, as well as server to server communication +Etcd supports SSL/TLS and client cert authentication for clients to server, as well as server to server communication. -First, you need to have a CA cert `clientCA.crt` and signed key pair `client.crt`, `client.key`. This site has a good reference for how to generate self-signed key pairs: +First, you need to have a CA cert `clientCA.crt` and signed key pair `client.crt`, `client.key`. +This site has a good reference for how to generate self-signed key pairs: http://www.g-loaded.eu/2005/11/10/be-your-own-ca/ For testing you can use the certificates in the `fixtures/ca` directory. -Next, lets configure etcd to use this keypair: +Let's configure etcd to use this keypair: ```sh ./etcd -n node0 -d node0 -clientCert=./fixtures/ca/server.crt -clientKey=./fixtures/ca/server.key.insecure -f ``` -`-f` forces new node configuration if existing configuration is found (WARNING: data loss!) -`-clientCert` and `-clientKey` are the key and cert for transport layer security between client and server +There are a few new options we're using: -You can now test the configuration using https: +* `-f` - forces a new node configuration, even if an existing configuration is found. (WARNING: data loss!) +* `-clientCert` and `-clientKey` specify the location of the cert and key files to be used for for transport layer security between the client and server. + +You can now test the configuration using HTTPS: ```sh -curl --cacert fixtures/ca/ca.crt https://127.0.0.1:4001/v1/keys/foo -d value=bar -v +curl --cacert ./fixtures/ca/server-chain.pem https://127.0.0.1:4001/v2/keys/foo -XPUT -d value=bar -v ``` You should be able to see the handshake succeed. @@ -313,15 +395,17 @@ SSLv3, TLS handshake, Finished (20): ... ``` -And also the response from the etcd server. +And also the response from the etcd server: ```json -{"action":"SET","key":"/foo","value":"bar","newKey":true,"index":3} +{"action":"set","key":"/foo","prevValue":"bar","value":"bar","modifiedIndex":3} ``` + ### Authentication with HTTPS client certificates -We can also do authentication using CA certs. The clients will provide their cert to the server and the server will check whether the cert is signed by the CA and decide whether to serve the request. +We can also do authentication using CA certs. +The clients will provide their cert to the server and the server will check whether the cert is signed by the CA and decide whether to serve the request. ```sh ./etcd -n node0 -d node0 -clientCAFile=./fixtures/ca/ca.crt -clientCert=./fixtures/ca/server.crt -clientKey=./fixtures/ca/server.key.insecure -f @@ -332,7 +416,7 @@ We can also do authentication using CA certs. The clients will provide their cer Try the same request to this server: ```sh -curl --cacert fixtures/ca/ca.crt https://127.0.0.1:4001/v1/keys/foo -d value=bar -v +curl --cacert ./fixtures/ca/server-chain.pem https://127.0.0.1:4001/v2/keys/foo -XPUT -d value=bar -v ``` The request should be rejected by the server. @@ -346,10 +430,11 @@ routines:SSL3_READ_BYTES:sslv3 alert bad certificate We need to give the CA signed cert to the server. ```sh -curl -L https://127.0.0.1:4001/v1/keys/foo -d value=bar -v --key myclient.key --cert myclient.crt -cacert clientCA.crt +curl --key ./fixtures/ca/server2.key.insecure --cert ./fixtures/ca/server2.crt --cacert ./fixtures/ca/server-chain.pem -L https://127.0.0.1:4001/v1/keys/foo -XPUT -d value=bar -v ``` -You should able to see +You should able to see: + ``` ... SSLv3, TLS handshake, CERT verify (15): @@ -360,16 +445,18 @@ TLS handshake, Finished (20) And also the response from the server: ```json -{"action":"SET","key":"/foo","value":"bar","newKey":true,"index":3} +{"action":"set","key":"/foo","prevValue":"bar","value":"bar","modifiedIndex":3} ``` + ## Clustering ### Example cluster of three machines -Let's explore the use of etcd clustering. We use go-raft as the underlying distributed protocol which provides consistency and persistence of the data across all of the etcd instances. +Let's explore the use of etcd clustering. +We use Raft as the underlying distributed protocol which provides consistency and persistence of the data across all of the etcd instances. -We'll start by creating 3 new etcd instances. +Let start by creating 3 new etcd instances. We use -s to specify server port and -c to specify client port and -d to specify the directory to store the log and info of the node in the cluster @@ -377,17 +464,17 @@ We use -s to specify server port and -c to specify client port and -d to specify ./etcd -s 127.0.0.1:7001 -c 127.0.0.1:4001 -d nodes/node1 -n node1 ``` -**Note:** If you want to run etcd on external IP address and still have access locally you need to add `-cl 0.0.0.0` so that it will listen on both external and localhost addresses. +**Note:** If you want to run etcd on an external IP address and still have access locally, you'll need to add `-cl 0.0.0.0` so that it will listen on both external and localhost addresses. A similar argument `-sl` is used to setup the listening address for the server port. -Let's join two more nodes to this cluster using the -C argument: +Let's join two more nodes to this cluster using the `-C` argument: ```sh ./etcd -s 127.0.0.1:7002 -c 127.0.0.1:4002 -C 127.0.0.1:7001 -d nodes/node2 -n node2 ./etcd -s 127.0.0.1:7003 -c 127.0.0.1:4003 -C 127.0.0.1:7001 -d nodes/node3 -n node3 ``` -Get the machines in the cluster: +We can retrieve a list of machines in the cluster using the HTTP API: ```sh curl -L http://127.0.0.1:4001/v1/machines @@ -399,24 +486,23 @@ We should see there are three nodes in the cluster http://127.0.0.1:4001, http://127.0.0.1:4002, http://127.0.0.1:4003 ``` -The machine list is also available via this API: +The machine list is also available via the main key API: ```sh curl -L http://127.0.0.1:4001/v1/keys/_etcd/machines ``` ```json -[{"action":"GET","key":"/_etcd/machines/node1","value":"raft=http://127.0.0.1:7001&etcd=http://127.0.0.1:4001","index":4},{"action":"GET","key":"/_etcd/machines/node2","value":"raft=http://127.0.0.1:7002&etcd=http://127.0.0.1:4002","index":4},{"action":"GET","key":"/_etcd/machines/node3","value":"raft=http://127.0.0.1:7003&etcd=http://127.0.0.1:4003","index":4}] +[{"action":"get","key":"/_etcd/machines/node1","value":"raft=http://127.0.0.1:7001\u0026etcd=http://127.0.0.1:4001","index":1},{"action":"get","key":"/_etcd/machines/node2","value":"raft=http://127.0.0.1:7002\u0026etcd=http://127.0.0.1:4002","index":1},{"action":"get","key":"/_etcd/machines/node3","value":"raft=http://127.0.0.1:7003\u0026etcd=http://127.0.0.1:4003","index":1}] ``` -The key of the machine is based on the ```commit index``` when it was added. The value of the machine is ```hostname```, ```raft port``` and ```client port```. - -Also try to get the current leader in the cluster +We can also get the current leader in the cluster: ``` -curl -L http://127.0.0.1:4001/v1/leader +curl -L http://127.0.0.1:4001/v2/leader ``` -The first server we set up should be the leader, if it has not died during these commands. + +The first server we set up should still be the leader unless it has died during these commands. ``` http://127.0.0.1:7001 @@ -425,25 +511,26 @@ http://127.0.0.1:7001 Now we can do normal SET and GET operations on keys as we explored earlier. ```sh -curl -L http://127.0.0.1:4001/v1/keys/foo -d value=bar +curl -L http://127.0.0.1:4001/v2/keys/foo -XPUT -d value=bar ``` ```json -{"action":"SET","key":"/foo","value":"bar","newKey":true,"index":5} +{"action":"set","key":"/foo","value":"bar","modifiedIndex":4} ``` + ### Killing Nodes in the Cluster -Let's kill the leader of the cluster and get the value from the other machine: +Now if we kill the leader of the cluster, we can get the value from one of the other two machines: ```sh curl -L http://127.0.0.1:4002/v1/keys/foo ``` -A new leader should have been elected. +We can also see that a new leader has been elected: ``` -curl -L http://127.0.0.1:4001/v1/leader +curl -L http://127.0.0.1:4002/v1/leader ``` ``` @@ -456,17 +543,11 @@ or http://127.0.0.1:7003 ``` -You should be able to see this: - -```json -{"action":"GET","key":"/foo","value":"bar","index":5} -``` - -It succeeded! ### Testing Persistence -OK. Next let us kill all the nodes to test persistence. Restart all the nodes using the same command as before. +Next we'll kill all the nodes to test persistence. +Type `CTRL-C` on each terminal and then rerun the same command you used to start each node. Your request for the `foo` key will return the correct value: @@ -475,19 +556,24 @@ curl -L http://127.0.0.1:4002/v1/keys/foo ``` ```json -{"action":"GET","key":"/foo","value":"bar","index":5} +{"action":"get","key":"/foo","value":"bar","index":4} ``` + ### Using HTTPS between servers -In the previous example we showed how to use SSL client certs for client to server communication. Etcd can also do internal server to server communication using SSL client certs. To do this just change the ```-client*``` flags to ```-server*```. +In the previous example we showed how to use SSL client certs for client-to-server communication. +Etcd can also do internal server-to-server communication using SSL client certs. +To do this just change the `-client*` flags to `-server*`. + +If you are using SSL for server-to-server communication, you must use it on all instances of etcd. -If you are using SSL for server to server communication, you must use it on all instances of etcd. ## Contributing See [CONTRIBUTING](https://github.com/coreos/etcd/blob/master/CONTRIBUTING.md) for details on submitting patches and contacting developers via IRC and mailing lists. + ## Libraries and Tools **Tools** @@ -503,7 +589,6 @@ See [CONTRIBUTING](https://github.com/coreos/etcd/blob/master/CONTRIBUTING.md) f - [justinsb/jetcd](https://github.com/justinsb/jetcd) - [diwakergupta/jetcd](https://github.com/diwakergupta/jetcd) - **Python libraries** - [transitorykris/etcd-py](https://github.com/transitorykris/etcd-py) @@ -528,10 +613,6 @@ See [CONTRIBUTING](https://github.com/coreos/etcd/blob/master/CONTRIBUTING.md) f - [aterreno/etcd-clojure](https://github.com/aterreno/etcd-clojure) -**Erlang libraries** - -- [marshall-lee/etcd.erl](https://github.com/marshall-lee/etcd.erl) - **Chef Integration** - [coderanger/etcd-chef](https://github.com/coderanger/etcd-chef) @@ -552,45 +633,49 @@ See [CONTRIBUTING](https://github.com/coreos/etcd/blob/master/CONTRIBUTING.md) f - [mattn/etcdenv](https://github.com/mattn/etcdenv) - "env" shebang with etcd integration - [kelseyhightower/confd](https://github.com/kelseyhightower/confd) - Manage local app config files using templates and data from etcd + ## FAQ ### What size cluster should I use? Every command the client sends to the master is broadcast to all of the followers. -But, the command is not committed until the majority of the cluster machines receive that command. +The command is not committed until the majority of the cluster machines receive that command. -Because of this majority voting property the ideal cluster should be kept small to keep speed up and be made up of an odd number of machines. +Because of this majority voting property, the ideal cluster should be kept small to keep speed up and be made up of an odd number of machines. -Odd numbers are good because if you have 8 machines the majority will be 5 and if you have 9 machines the majority will be 5. +Odd numbers are good because if you have 8 machines the majority will be 5 and if you have 9 machines the majority will still be 5. The result is that an 8 machine cluster can tolerate 3 machine failures and a 9 machine cluster can tolerate 4 nodes failures. And in the best case when all 9 machines are responding the cluster will perform at the speed of the fastest 5 nodes. + ### Why SSLv3 alert handshake failure when using SSL client auth? -The `TLS` pacakge of `golang` checks the key usage of certificate public key before using it. To use the certificate public key to do client auth, we need to add `clientAuth` to `Extended Key Usage` when creating the certificate public key. + +The `crypto/tls` package of `golang` checks the key usage of the certificate public key before using it. +To use the certificate public key to do client auth, we need to add `clientAuth` to `Extended Key Usage` when creating the certificate public key. Here is how to do it: Add the following section to your openssl.cnf: ``` -[ ssl_client ] +[ ssl_client ] ... extendedKeyUsage = clientAuth ... ``` -When creating the cert be sure to reference it in the -extensions flag: +When creating the cert be sure to reference it in the `-extensions` flag: ``` openssl ca -config openssl.cnf -policy policy_anything -extensions ssl_client -out certs/node.crt -infiles node.csr ``` + ## Project Details ### Versioning etcd uses [semantic versioning][semver]. -When we release v1.0.0 of etcd we will promise not to break the "v1" REST API. New minor versions may add additional features to the API however. You can get the version of etcd by issuing a request to /version: @@ -599,10 +684,11 @@ You can get the version of etcd by issuing a request to /version: curl -L http://127.0.0.1:4001/version ``` -During the v0 series of releases we may break the API as we fix bugs and get feedback. +During the pre-v1.0.0 series of releases we may break the API as we fix bugs and get feedback. [semver]: http://semver.org/ + ### License etcd is under the Apache 2.0 license. See the [LICENSE][license] file for details. diff --git a/Vagrantfile b/Vagrantfile deleted file mode 100644 index ec3de3765d6..00000000000 --- a/Vagrantfile +++ /dev/null @@ -1,31 +0,0 @@ -# This Vagrantfile is targeted at developers. It can be used to build and run etcd in an isolated VM. - -$provision = <\n\n\n
\n