Track clean-up (elastic#25)

* Updated following tests * Corrected config and updated following review
henningandersen · Aug 30, 2018 · 2965e86 · 2965e86
1 parent 574d168
commit 2965e86
Show file tree

Hide file tree

Showing 13 changed files with 115 additions and 128 deletions.
diff --git a/README.md b/README.md
@@ -25,20 +25,56 @@ Another option is to download the repository and point to it using the `--track-
 
 This is the default challenge, which performs bulk indexing at maximum throughput against a single index for a period of 20 minutes.
 
+The table below shows the track parameters that can be adjusted along with default values:
+
+| Parameter | Explanation | Type | Default Value |
+| --------- | ----------- | ---- | ------------- |
+| `number_of_replicas` | Number of index replicas | `int` | `0` |
+| `shard_count` | Number of primary shards | `int` | `2` |
+| `bulk_indexing_clients` | Number of bulk indexing clients/connections | `int` | `8` |
+
 ### 2) bulk-size-evaluation
 
 This challenge performs bulk-indexing against a single index with varying bulk request sizes, ranging from 125 events/request to 50000 events/request.
 
+The table below shows the track parameters that can be adjusted along with default values:
+
+| Parameter | Explanation | Type | Default Value |
+| --------- | ----------- | ---- | ------------- |
+| `number_of_replicas` | Number of index replicas | `int` | `0` |
+| `shard_count` | Number of primary shards | `int` | `2` |
+| `bulk_indexing_clients` | Number of bulk indexing clients/connections | `int` | `16` |
+
 ### 3) shard-sizing
 
 This challenge indexes 2 million events into an index consisting of a single shard 25 times. After each group of 2 million events has been inserted, 4 different Kibana dashboard configurations are benchmarked against the index. At this time no indexing takes place. There are two different dashboards being simulated, aggregating across 50% and 90% of the data in the shard.
 
 This challenge shows how shard sizing can be performed and how the nature of queries used can impact the results.
 
+The table below shows the track parameters that can be adjusted along with default values:
+
+| Parameter | Explanation | Type | Default Value |
+| --------- | ----------- | ---- | ------------- |
+| `number_of_replicas` | Number of index replicas | `int` | `0` |
+| `shard_count` | Number of primary shards | `int` | `2` |
+| `bulk_indexing_clients` | Number of bulk indexing clients/connections | `int` | `16` |
+| `shard_sizing_iterations` | Number of indexing querying iterations to run | `int` | `25` |
+| `shard_sizing_queries` | Number of queries of each type to run for each iteration | `int` | `20` |
+
 ### 4) elasticlogs-1bn-load
 
 This challenge indexes 1 billion events into a number of indices of 2 primary shards each, and results in around 200GB of indices being generated on disk. This can vary depending on the environment. It can be used give an idea of how max indexing performance behaves over an extended period of time.
 
+The table below shows the track parameters that can be adjusted along with default values:
+
+| Parameter | Explanation | Type | Default Value |
+| --------- | ----------- | ---- | ------------- |
+| `number_of_replicas` | Number of index replicas | `int` | `0` |
+| `shard_count` | Number of primary shards | `int` | `2` |
+| `bulk_indexing_clients` | Number of bulk indexing clients/connections | `int` | `20` |
+| `disk_type` | Type of disk used. If disk_type is not `ssd`, a single merge scheduler thread will be specified in the index template | `string` | `ssd` |
+| `translog_sync` | If value is not `request`, translog will be configured to use `async` mode | `string` | `request` |
+
 ### 5) elasticlogs-querying
 
 This challenge runs mixed Kibana queries against the index created in the **elasticlogs-1bn-load** track. No concurrent indexing is performed.
@@ -49,6 +85,19 @@ This challenge assumes that the *elasticlogs-1bn-load* track has been executed a
 
 In this challenge rate-limited indexing at varying levels is combined with a fixed level of querying. If metrics from the run are stored in Elasticsearch, it is possible analyse these in Kibana in order to identify how indexing rate affects query latency and vice versa.
 
+The table below shows the track parameters that can be adjusted along with default values:
+
+| Parameter | Explanation | Type | Default Value |
+| --------- | ----------- | ---- | ------------- |
+| `number_of_replicas` | Number of index replicas | `int` | `0` |
+| `shard_count` | Number of primary shards | `int` | `2` |
+| `bulk_indexing_clients` | Number of bulk indexing clients/connections | `int` | `24` |
+| `disk_type` | Type of disk used. If disk_type is not `ssd`, a single merge scheduler thread will be specified in the index template | `string` | `ssd` |
+| `translog_sync` | If value is not `request`, translog will be configured to use `async` mode | `string` | `request` |
+| `rate_limit_duration_secs` | Duration in seconds for each rate limited benchmark rate_limit_step | `int` | `1200` |
+| `rate_limit_step` | Number of requests per second to use as a rate_limit_step. `2` indicates rate limiting will increase in steps of 2k EPS | `int` | `2` |
+| `rate_limit_max` | Maximum number of requests per second to use for rate-limiting. `32` indicates a top target indexing rate of 32k EPS | `int` | `32` |
+
 ### 7) elasticlogs-continuous-index-and-query
 
 This challenge is suitable for long term execution and runs in two phases. Both phases (`p1`, `p2`) index documents containing auto-generated event, however, `p1` indexes events at the max possible speed, whereas `p2` throttles indexing to a specified rate and in parallel executes four queries simulating Kibana dashboards and queries. The created index gets rolled over after the configured max size and the maximum amount of rolled over indices are also configurable.

diff --git a/eventdata/challenges/document_id_benchmark.json b/eventdata/challenges/document_id_benchmark.json
@@ -58,9 +58,7 @@
             "index.number_of_replicas": 0,
             "index.number_of_shards": 1
           },
-          "mappings": 
-              {% include "mappings.json" %}
-          ,
+          "mappings": "mappings.json",
           "aliases": {}
         },
         "index_template_name": "elasticlogs-auto"
@@ -93,9 +91,7 @@
             "index.number_of_replicas": 0,
             "index.number_of_shards": 1
           },
-          "mappings": 
-              {% include "mappings.json" %}
-          ,
+          "mappings": "mappings.json",
           "aliases": {}
         },
         "index_template_name": "elasticlogs-uuid"
@@ -129,9 +125,7 @@
             "index.number_of_replicas": 0,
             "index.number_of_shards": 1
           },
-          "mappings": 
-              {% include "mappings.json" %}
-          ,
+          "mappings": "mappings.json",
           "aliases": {}
         },
         "index_template_name": "elasticlogs-epoch-no_delay"
@@ -165,9 +159,7 @@
             "index.number_of_replicas": 0,
             "index.number_of_shards": 1
           },
-          "mappings": 
-              {% include "mappings.json" %}
-          ,
+          "mappings": "mappings.json",
           "aliases": {}
         },
         "index_template_name": "elasticlogs-epoch-10pct_60s"
@@ -203,9 +195,7 @@
             "index.number_of_replicas": 0,
             "index.number_of_shards": 1
           },
-          "mappings": 
-              {% include "mappings.json" %}
-          ,
+          "mappings": "mappings.json",
           "aliases": {}
         },
         "index_template_name": "elasticlogs-epoch-10pct_300s"

diff --git a/eventdata/challenges/large-shard-sizing.json b/eventdata/challenges/large-shard-sizing.json
@@ -32,9 +32,7 @@
             "index.number_of_replicas": 0,
             "index.number_of_shards": 1
           },
-          "mappings": 
-            {% include "mappings.json" %}
-          ,
+          "mappings": "mappings.json",
           "aliases": {}
         },
         "index_template_name": "elasticlogs-auto"
@@ -188,9 +186,7 @@
             "index.number_of_replicas": 0,
             "index.number_of_shards": 1
           },
-          "mappings": 
-            {% include "mappings.json" %}
-          ,
+          "mappings": "mappings.json",
           "aliases": {}
         },
         "index_template_name": "elasticlogs-{{ id_type }}"

diff --git a/eventdata/challenges/refresh-interval.json b/eventdata/challenges/refresh-interval.json
@@ -36,9 +36,7 @@
             "index.number_of_replicas": {{ p_number_of_replicas }},
             "index.number_of_shards": {{ p_shard_count }}
           },
-          "mappings": 
-            {% include "mappings.json" %}
-            ,
+          "mappings": "mappings.json",
           "aliases": {}
         },
         "index_template_name": "elasticlogs"
@@ -82,9 +80,7 @@
             "index.number_of_replicas": {{ p_number_of_replicas }},
             "index.number_of_shards": {{ p_shard_count }}
           },
-          "mappings": 
-            {% include "mappings.json" %}
-            ,
+          "mappings": "mappings.json",
           "aliases": {}
         },
         "index_template_name": "elasticlogs"
@@ -128,9 +124,7 @@
             "index.number_of_replicas": {{ p_number_of_replicas }},
             "index.number_of_shards": {{ p_shard_count }}
           },
-          "mappings": 
-            {% include "mappings.json" %}
-            ,
+          "mappings": "mappings.json",
           "aliases": {}
         },
         "index_template_name": "elasticlogs"
@@ -174,9 +168,7 @@
             "index.number_of_replicas": {{ p_number_of_replicas }},
             "index.number_of_shards": {{ p_shard_count }}
           },
-          "mappings": 
-            {% include "mappings.json" %}
-            ,
+          "mappings": "mappings.json",
           "aliases": {}
         },
         "index_template_name": "elasticlogs"
@@ -220,9 +212,7 @@
             "index.number_of_replicas": {{ p_number_of_replicas }},
             "index.number_of_shards": {{ p_shard_count }}
           },
-          "mappings": 
-            {% include "mappings.json" %}
-            ,
+          "mappings": "mappings.json",
           "aliases": {}
         },
         "index_template_name": "elasticlogs"

diff --git a/eventdata/challenges/test.json b/eventdata/challenges/test.json
diff --git a/eventdata/challenges/test2.json b/eventdata/challenges/test2.json
diff --git a/eventdata/operations/indexing.json b/eventdata/operations/indexing.json
@@ -106,9 +106,7 @@ rolledover_indices_suffix_separator: used by the `delete_rolledover_index_patter
       "index.number_of_replicas": {{ number_of_replicas | default(0) }},
       "index.number_of_shards": {{ shard_count | default(2) }}
     },
-    "mappings":
-      {% include "mappings.json" %}
-    ,
+    "mappings": "mappings.json",
     "aliases": {}
   },
   "index_template_name": "elasticlogs_q_write"
@@ -127,14 +125,12 @@ rolledover_indices_suffix_separator: used by the `delete_rolledover_index_patter
       "index.translog.durability": "async",
       {% endif %}
       {% if (p_disk_type != 'ssd') %}
-      "index.merge.scheduler.max_thread_count": 1,
+      "index.merge.scheduler.max_thread_count": 1,  
       {% endif %}
       "index.number_of_replicas": {{ number_of_replicas | default(0) }},
       "index.number_of_shards": {{ shard_count | default(2) }}
     },
-    "mappings":
-        {% include "mappings.json" %}
-    ,
+    "mappings": "mappings.json",
     "aliases": {}
   },
   "index_template_name": "elasticlogs_i_write"
@@ -157,9 +153,7 @@ rolledover_indices_suffix_separator: used by the `delete_rolledover_index_patter
       "index.number_of_replicas": {{ number_of_replicas | default(0) }},
       "index.number_of_shards": {{ shard_count | default(1) }}
     },
-    "mappings":
-        {% include "mappings.json" %}
-    ,
+    "mappings": "mappings.json",
     "aliases": {}
   },
   "index_template_name": "elasticlogs"