Fpetrilli/improve docs

docs/iteration-io.md

@@ -1,4 +1,4 @@
-###Iteration and Array IO Specification
+### Iteration and Array IO Specification
 
 This DSL feature specifies that a command should be run for a given number of 
 iterations.  These iterations can be bounded by an integer or correspond to the
@@ -92,4 +92,4 @@ Ex3 -- using default %s index to reference node this is being run on
    line: 0
    should_be_equal_to: "0"
 
-```
+```

docs/selection.md

@@ -1,4 +1,4 @@
-###Selection Specification
+### Selection Specification
 
 The DSL selection feature chooses nodes on which to run commands.
 In addition to the on_node/end_node option each step can alternatively take

docs/selectors.md

@@ -0,0 +1,16 @@
+# Selectors
+
+## Label Selectors in Kubernetes for test run selection
+
+Sometimes, we want to run a particular test only against certain subsets of pods in the Kubernetes cluster (as we may be running multiple simultaneous tests together).
+
+For this purpose, one can make use of the `selector` attribute under a test definition. This follows the kubernetes format of `key=value`, where some of the most common are the `run` key, which indicates a grouping of pods that are part of a particular rungroup.
+
+For more information on the way Kubernetes uses labels and selectors, I delegate to [the Kubernetes documentation](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/).
+
+In this way, one can shape their tests against only go-ipfs or js-ipfs implementations, only ipfs-cluster deployments, only low-bandwidth pods, etc.
+
+```
+config:
+  selector: run=go-ipfs-stress
+```

docs/test_config.md

@@ -0,0 +1,24 @@
+# Test config
+
+Sometimes, we want to scale up or down our tests to compare against the size of a given architecture.
+
+To this end, we can use config.yml to define certain constants we wish to make use of. For an example of how to use this in a real test, see `config-writer.sh`, which creates the config.yml file from the user's input to automatically scale the tests, both by scaling the number of nodes the tests run on as well as scaling the number of pins we make during a test.
+
+In general, the config is available for substituting variables across tests.
+
+To use the config, the user can do one of two things:
+
+- A config file will be automatically found and used if named `config.yml` and in the same directory as the tests
+- Specify the path to the config using the flag `--config`.
+
+```
+params:
+  N: 5
+```
+
+Then, in the test definition:
+
+```
+config:
+  nodes: {{N}}
+```

docs/timeouts.md

@@ -0,0 +1,22 @@
+# Timeouts
+
+## Timeout for a step
+
+When running a command using IPFS, the result of a failure to resolve is often to hang a long time rather than an outright failure as IPFS searches for a way to connect you to the data you requested.
+
+Sometimes, we expect this to occur, and want to see that the test fails to resolve the data without waiting for its full timeout.
+
+To this end, one can make use of the `timeout` feature, and the `timeouts` parameter under the expected section (see validation.md). If a timeout is met, the timeouts value increments, and we can check against that
+to confirm our suspicions that the test has failed.
+
+The format of the parameter is the delay after which we consider the command to have timed out, expressed in **whole seconds**.
+
+```
+ - name: Cat file on node 2
+    on_node: 2
+    inputs:
+      - FILE
+      - HASH
+    cmd: ipfs cat $HASH
+    timeout: 2 
+```

docs/validation.md

@@ -0,0 +1,78 @@
+# Validation & Assertions
+
+## Inputs
+
+Within the confines of a 'step', adding the inputs parameter allows the user to specify the same name within the `cmd` tag as a standard `bash` variable (`$VARNAME`), 
+which creates the possibility to chain inputs from one step to the next.
+
+The format is simply to specify a name.
+
+```
+- name: Cat added file
+    on_node: 1
+    inputs:
+      - HASH
+    cmd: ipfs cat $HASH
+```
+
+## Outputs
+
+With in the confines of a 'step', adding the 'outputs' parameter allows the user to send the output from the command on a given line to a given name.
+This can then be verified later to validate the output of a particular command.
+
+The format is for a given `line`, save the output to the named variable `save_to`.
+
+For documentation of the `append_to` feature, see `iteration-io.md`.
+
+```
+steps:
+  - name: Add file
+    on_node: 1
+    cmd: head -c {{FILE_SIZE}} /dev/urandom | base64 > /tmp/file.txt && cat /tmp/file.txt && ipfs add -q /tmp/file.txt
+    outputs: 
+    - line: 0
+      save_to: FILE
+    - line: 1
+      save_to: HASH
+```
+
+## Assertions
+
+Once we're ready to validate the output of a particular command (often after chaining data from one node to another), we can make use of the assertions statement
+to create a check which `kubernetes-ipfs` will validate.
+
+Assertions as of the current version only have the parameter `should_be_equal_to`, which states that the given line should be equal to the provided output.
+
+If it is not, `kubernetes-ipfs` adds 1 to the 'failures' count. If it is, `kubernetes-ipfs` adds 1 to the 'successes' count.
+
+The format is as follows, using one of the 'inputs' passed to the step.
+
+```
+ name: Cat added file
+    on_node: {{    ON_NODE}}
+    inputs:
+      - FILE
+      - HASH
+    cmd: ipfs cat $HASH
+    assertions:
+    - line: 0
+      should_be_equal_to: FILE
+```
+
+## Expects
+
+Once the successes and failures have been talied up across runs for a particular test, the observed value is compared against the `expected` block.
+If the observed value matches, `kubernetes-ipfs` will report that expectations were met, and returns a 0 exit code. If they don't match, it will return
+a non-zero exit code, and state that the expectations were not met. This allows users to short-circut the tests (at the full-test level) and immediately fail 
+in the event of a test failure and to check the results of running a test from external software.
+
+Within the test definition:
+
+```
+ expected:
+       successes: 10
+       failures: 0
+       timeouts: 0
+```
+
+In this case, if the test fails once, we will return a non-zero exit code and fail the test.