Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON Decoding error with Python2 + Anaconda: Decimal('0') is not JSON serializable #1155

Closed
drewbanin opened this issue Nov 27, 2018 · 0 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@drewbanin
Copy link
Contributor

Issue

Issue description

The dbt test command can fail with an error like:

Decimal('0') is not JSON serializable

dbt explicitly implements a Decimal-aware JSON serializer, but it appears that there is more than one type of Decimal at play here. The agate library will optionally include the cdecimal version of Decimal if it is available (code here).

dbt does a check here that looks like:

        if isinstance(obj, Decimal):
            return float(obj)
        return super(JSONEncoder, self).default(obj)

here, Decimal is decimal.Decimal, but it could be that obj is of type cdecimal.Decimal. This leads to a confusing error.

Instead, dbt should check if the obj is of type decimal.Decimal or cdecimal.Decimal.

I think the Anaconda part of this is crucial, as I'm unsure how else someone might have cdecimal installed in their system. You can install it manually with pip install m3-cdecimal.

Results

"/Users/xxx/anaconda2/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/Users/xxx/anaconda2/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/Users/xxx/anaconda2/lib/python2.7/site-packages/dbt/utils.py", line 481, in default
    return super(JSONEncoder, self).default(obj)
  File "/Users/xxx/anaconda2/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: Decimal('0') is not JSON serializable

System information

The output of dbt --version:

0.12.1

The operating system you're running on: OSX

The python version you're using (probably the output of python --version)
2.7.15 installed via Anaconda.

@drewbanin drewbanin added the bug Something isn't working label Nov 27, 2018
@drewbanin drewbanin added this to the Wilt Chamberlain milestone Nov 28, 2018
@beckjake beckjake self-assigned this Dec 3, 2018
beckjake added a commit that referenced this issue Dec 3, 2018
…coding

handle cdecimal.Decimal during serialization [#1155]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants