Make exists_set_param macro available to tracks (#1012)

Frequently it's useful to specify a setting based on the presence of a track parameter. To avoid repetition this commit makes a Jinja2 macro called `exists_set_param()` available for all tracks (if they import the rally helpers).
elastic · Jun 10, 2020 · 8d328de · 8d328de
1 parent 19d4df7
commit 8d328de
Show file tree

Hide file tree

Showing 2 changed files with 66 additions and 4 deletions.
diff --git a/docs/adding_tracks.rst b/docs/adding_tracks.rst
@@ -560,6 +560,8 @@ To reuse operation definitions across challenges, you can define them in a separ
 
 Note how we reference to the operations by their name (e.g. ``create``, ``bulk-index``, ``force-merge`` or ``query-match-all``).
 
+.. _track_collect_helper:
+
 You can also use Rally's collect helper to simplify including multiple challenges::
 
     {% import "rally.helpers" as rally %}
@@ -634,7 +636,7 @@ The changes are:
 
     Rally's log file contains the fully rendered track after it has loaded it successfully.
 
-You can even use `Jinja2 variables <http://jinja.pocoo.org/docs/2.9/templates/#assignments>`_ but then you need to import the Rally helpers a bit differently. You also need to declare all variables before the ``import`` statement::
+You can even use `Jinja2 variables <http://jinja.pocoo.org/docs/dev/templates/#assignments>`_ but then you need to import the Rally helpers a bit differently. You also need to declare all variables before the ``import`` statement::
 
         {% set clients = 16 %}
         {% import "rally.helpers" as rally with context %}
@@ -723,6 +725,52 @@ You can find an example in the ``http_logs`` track::
 
 The data set that is used in the ``http_logs`` track starts on 26-04-1998 but we want to ignore the first few days for this query, so we start on 15-05-1998. The expression ``{{'15-05-1998' | days_ago(now)}}`` yields the difference in days between now and the fixed start date and allows us to benchmark time range queries relative to now with a predetermined data set.
 
+* ``rally.collect(parts)``: a `macro <https://jinja.pocoo.org/docs/dev/templates/#macros>`_ that you can use to join track fragments. See the :ref:`example above<track_collect_helper>`.
+* ``rally.exists_set_param(setting_name, value, default_value=None, comma=True)``: a `macro <https://jinja.pocoo.org/docs/dev/templates/#macros>`_ that you can use to set the value of a track parameter without having to check if it exists.
+
+.. important::
+    To use macros you must declare ``{% import "rally.helpers" as rally with context %}`` at the top of your track; see :ref:`the docs <track_collect_helper>` for more details and the `geonames track <https://github.com/elastic/rally-tracks/blob/b2f86df5f0c18461fdb64dd9ee1fe16bd3653b9d/geonames/track.json#L1>`_ for an example.
+
+Example:
+
+Suppose you need an operation that specifies the Elasticsearch transient setting ``indices.recovery.max_bytes_per_sec`` if and only if it has been provided as a track parameter.
+
+Your operation could look like::
+
+    {
+      "operation": {
+        "operation-type": "raw-request",
+        "method": "PUT",
+        "path": "/_cluster/settings",
+        "body": {
+          "transient": {
+            "cluster.routing.allocation.node_initial_primaries_recoveries": 8
+            {{ rally.exists_set_param("indices.recovery.max_bytes_per_sec", es_snapshot_restore_recovery_max_bytes_per_sec) }}
+          }
+        }
+      }
+    }
+
+Note the lack of a comma after the first setting ``cluster.routing.allocation.node_initial_primaries_recoveries``. This is intentional since the helper will insert it if the parameter exists (this behavior can be changed using ``comma=False``).
+
+Assuming we pass ``--track-params="es_snapshot_restore_recovery_max_bytes_per_sec:-1"`` the helper will end up rendering the operation as::
+
+    {
+      "operation": {
+        "operation-type": "raw-request",
+        "method": "PUT",
+        "path": "/_cluster/settings",
+        "body": {
+          "transient": {
+            "cluster.routing.allocation.node_initial_primaries_recoveries": 8,"indices.recovery.max_bytes_per_sec": -1
+          }
+        }
+      }
+    }
+
+
+The parameter ``default_value`` controls the value to use for the setting if it is undefined. If the setting is undefined and there is no default value, nothing will be added.
+
 .. _adding_tracks_custom_param_sources:
 
 Custom parameter sources

diff --git a/esrally/track/loader.py b/esrally/track/loader.py
@@ -607,20 +607,34 @@ def default_internal_template_vars(glob_helper=lambda f: [], clock=time.Clock):
 
 
 def render_template(template_source, template_vars=None, template_internal_vars=None, loader=None):
-    macros = """
+    macros = [
+        """
         {% macro collect(parts) -%}
             {% set comma = joiner() %}
             {% for part in glob(parts) %}
                 {{ comma() }}
                 {% include part %}
             {% endfor %}
         {%- endmacro %}
-    """
+        """,
+        """
+        {% macro exists_set_param(setting_name, value, default_value=None, comma=True) -%}
+            {% if value is defined or default_value is not none %}
+                {% if comma %} , {% endif %}
+                {% if default_value is not none %}
+                  "{{ setting_name }}": {{ value | default(default_value) | tojson }}
+                {% else %}
+                  "{{ setting_name }}": {{ value | tojson }}
+                {% endif %}
+              {% endif %}
+        {%- endmacro %}
+        """
+    ]
 
     # place helpers dict loader first to prevent users from overriding our macros.
     env = jinja2.Environment(
         loader=jinja2.ChoiceLoader([
-            jinja2.DictLoader({"rally.helpers": macros}),
+            jinja2.DictLoader({"rally.helpers": "".join(macros)}),
             jinja2.BaseLoader(),
             loader
         ])