Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

loading data as yaml with date or datetime fails with json exception #3

Closed
epDHowwD opened this issue Dec 6, 2022 · 4 comments · May be fixed by #4
Closed

loading data as yaml with date or datetime fails with json exception #3

epDHowwD opened this issue Dec 6, 2022 · 4 comments · May be fixed by #4

Comments

@epDHowwD
Copy link

epDHowwD commented Dec 6, 2022

some data file:

example:
    - someDate: 2022-02-02

Using the cli interface:

$ jinja --version
jinja, version 1.0.7

Error is:

$ jinja render -t foobar -d data.yaml
Traceback (most recent call last):
  File "/usr/local/bin/jinja", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/jinja2_tools/cli.py", line 53, in render
    data = Data(data, verbose).get_data()
  File "/usr/local/lib/python3.10/site-packages/jinja2_tools/objects.py", line 69, in get_data
    print_verbose({'title': '[Data]', 'content': json.dumps(
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 201, in encode
    chunks = list(chunks)
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 325, in _iterencode_list
    yield from chunks
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type date is not JSON serializable

Looking into objects.py there is a json.dumps invocation on the loaded yaml which seems not be able to cope with date and timestamps.

@Avielyo10
Copy link
Owner

Avielyo10 commented Sep 14, 2023

@epDHowwD Hi 👋
Apologies for the delay in addressing your concern. JSON has specific guidelines regarding the allowable types, which include Number, String, Boolean, Array, Object, Whitespace, and Null.

Try:

example:
    - someDate: "2022-02-02"

@epDHowwD
Copy link
Author

No I cannot change the database so the tool on top works. The yaml data comes various data sources and we use your cmdline tool (and I cannot underline its usefulness in this scenario enough) to extract information from it to display it in our documentation. We basically generate documentation from the actual configuration files. These config files we cannot change just so they work in the docu display well.

What we did, and what I propose you to fix differently, is to patch the tool in the way to remove the verbosity logging as json of the data being read in the function jinja2_tools.objects.Data.get_data:

#!/usr/bin/env python3

# -*- coding: utf-8 -*-
import re
import sys
import jinja2_tools

# fixes https://github.com/Avielyo10/jinja2-tools/issues/3

from jinja2_tools.exceptions import InvalidDataType
from jinja2_tools.util import input_handler, load_yaml
from jinja2_tools.cli import main

def fixed_get_data(self):
    try:
        ih_content = input_handler(self.data)
    except InvalidDataType as err:
        print(err.message, file=sys.stderr)
        sys.exit(128)
    else:
        self.data = load_yaml(ih_content)
        # print_verbose({'title': '[Data]', 'content': json.dumps(
        #     self.data, indent=2), 'verbose': self.verbose})
        return self.data

jinja2_tools.objects.Data.get_data = fixed_get_data

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

For example you can replace the json.dumps function with some yaml dump or pprint. That would cover more data types. At least as much as yaml supports generally.

@Avielyo10
Copy link
Owner

@epDHowwD got you, I'll create a fix for all the dumps!
Thanks!

@Avielyo10
Copy link
Owner

Avielyo10 commented Mar 6, 2024

Done!
Thank you for using this tool!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants