AttributeError: 'module' object has no attribute 'JSONDecodeError'

[cjx-great]

when I try to list tracks, some errors occur when paesing indices.body, please help me figure out what went wrong.

Rally version (get with esrally --version):

esrally 1.0.0

Invoked command:

esrally list tracks --track-repository=billions-rally-tracks

Configuration file (located in ~/track.json)):

{% set index_count = 1 %}
{% import "rally.helpers" as rally with context %}
{
  "version": 2,
  "description": "index",
  "indices": [
    {% set comma = joiner() %}
    {% for item in range(index_count) %}
    {{ comma() }}
    {
      "name": "billions-jssz.benchmark.scene1-{{item}}",
      "body": "settings/mapping.json",
      "types": [ "logs" ]
    }
    {% endfor %}
  ],
  "corpora": [
    {{ rally.collect(parts="corpora/corpora.json") }}
  ],
  "challenges": [
    {{ rally.collect(parts="challenges/index.json") }}
  ]
}

Configuration file (located in ~/settings/mapping.json)):

{
  "settings": {
    "index.number_of_shards": {{number_of_shards | default(1)}},
    "index.number_of_replicas": {{number_of_replicas | default(0)}}
  },
  "mappings": {
    "type": {
      "_source": {
        "enabled": {{ source_enabled | default(true) }}
      }
    }
  }
}

logs:

2018-09-04 06:39:57,173 -not-actor-/PID:85039 esrally.track.loader INFO Loading template [definition for index billions-jssz.benchmark.scene1-0 in settings/mapping.json].
2018-09-04 06:39:57,175 -not-actor-/PID:85039 esrally.rally ERROR A fatal error occurred while running subcommand [list].
Traceback (most recent call last):
  File "/data/rally/lib/python3.4/site-packages/esrally/track/loader.py", line 753, in _load_template
    return json.loads(rendered)
  File "/usr/lib/python3.4/json/__init__.py", line 318, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.4/json/decoder.py", line 343, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.4/json/decoder.py", line 361, in raw_decode
    raise ValueError(errmsg("Expecting value", s, err.value)) from None
ValueError: Expecting value: line 9 column 20 (char 158)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/rally/lib/python3.4/site-packages/esrally/rally.py", line 452, in dispatch_sub_command
    dispatch_list(cfg)
  File "/data/rally/lib/python3.4/site-packages/esrally/rally.py", line 351, in dispatch_list
    track.list_tracks(cfg)
  File "/data/rally/lib/python3.4/site-packages/esrally/track/loader.py", line 41, in list_tracks
    available_tracks = tracks(cfg)
  File "/data/rally/lib/python3.4/site-packages/esrally/track/loader.py", line 37, in tracks
    for track_name in repo.track_names]
  File "/data/rally/lib/python3.4/site-packages/esrally/track/loader.py", line 37, in <listcomp>
    for track_name in repo.track_names]
  File "/data/rally/lib/python3.4/site-packages/esrally/track/loader.py", line 636, in read
    return self.read_track(track_name, track_spec, mapping_dir)
  File "/data/rally/lib/python3.4/site-packages/esrally/track/loader.py", line 697, in __call__
    for idx in self._r(track_specification, "indices", mandatory=False, default_value=[])]
  File "/data/rally/lib/python3.4/site-packages/esrally/track/loader.py", line 697, in <listcomp>
    for idx in self._r(track_specification, "indices", mandatory=False, default_value=[])]
  File "/data/rally/lib/python3.4/site-packages/esrally/track/loader.py", line 732, in _create_index
    body = self._load_template(f.read(), "definition for index {} in {}".format(index_name, body_file))
  File "/data/rally/lib/python3.4/site-packages/esrally/track/loader.py", line 754, in _load_template
    except (json.JSONDecodeError, jinja2.exceptions.TemplateError) as e:
AttributeError: 'module' object has no attribute 'JSONDecodeError'

[danielmitterdorfer]

Thanks for your report @cjx-great. You can see the original problem here:

ValueError: Expecting value: line 9 column 20 (char 158)

Unfortunately, this is not very easy to see because apparently there is a bug with Rally in Python 3.4 (which you are using). If you'd use Python 3.5 or Python 3.6 you'll see this on the command line:

[ERROR] Cannot list. Could not load file template for 'definition for index billions-jssz.benchmark.scene1-0 in settings/mappings.json'
	Expecting value: line 9 column 20 (char 158)

When we look in line 9 of the filesettings/mappings.json we see:

"enabled": {{ source_enabled | default(true) }}

After this has been processed by the template engine, this is still not valid JSON. You can fix this either by converting the result to JSON explicitly:

"enabled": {{ source_enabled | default(true) | tojson }}

or just have the template engine render it as a string instead of a Python boolean:

"enabled": {{ source_enabled | default('true') }}

We'll fix the problem w.r.t. AttributeError: 'module' object has no attribute 'JSONDecodeError' with the next version of Rally so it will show the correct error message then. Meanwhile I recommend to upgrade to Python 3.5 or Python 3.6 as a workaround.

[danielmitterdorfer]

The fix is now on master and will be released with Rally 1.0.1.