Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError: Object of type datetime is not JSON serializable #21

Closed
dosisod opened this issue Mar 21, 2023 · 1 comment · Fixed by #22
Closed

TypeError: Object of type datetime is not JSON serializable #21

dosisod opened this issue Mar 21, 2023 · 1 comment · Fixed by #22
Labels
bug Something isn't working

Comments

@dosisod
Copy link
Contributor

dosisod commented Mar 21, 2023

The following snippet:

import asyncio
from datetime import datetime, timezone

from githubkit.github import GitHub

github = GitHub()

async def main():
    await github.rest.checks.async_create(
        "user",
        "repo",
        name="name",
        head_sha="0000000000000000000000000000000000000000",
        started_at=datetime.now(timezone.utc)  # problematic line here
    )

asyncio.run(main())

Fails with the following exception(s):

Traceback (most recent call last):
  File "[redacted]/.venv/lib/python3.10/site-packages/githubkit/core.py", line 265, in _arequest
    return await client.request(
  File "[redacted]/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1520, in request
    request = self.build_request(
  File "[redacted]/.venv/lib/python3.10/site-packages/httpx/_client.py", line 360, in build_request
    return Request(
  File "[redacted]/.venv/lib/python3.10/site-packages/httpx/_models.py", line 339, in __init__
    headers, stream = encode_request(
  File "[redacted]/.venv/lib/python3.10/site-packages/httpx/_content.py", line 215, in encode_request
    return encode_json(json)
  File "[redacted]/.venv/lib/python3.10/site-packages/httpx/_content.py", line 178, in encode_json
    body = json_dumps(json).encode("utf-8")
  File "/usr/lib/python3.10/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type datetime is not JSON serializable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "[redacted]/main.py", line 34, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "[redacted]/main.py", line 20, in main
    x = await github.rest.checks.async_create(
  File "[redacted]/.venv/lib/python3.10/site-packages/githubkit/rest/checks.py", line 276, in async_create
    return await self._github.arequest(
  File "[redacted]/.venv/lib/python3.10/site-packages/githubkit/core.py", line 346, in arequest
    raw_resp = await self._arequest(
  File "[redacted]/.venv/lib/python3.10/site-packages/githubkit/core.py", line 279, in _arequest
    raise RequestError(repr(e)) from e
githubkit.exception.RequestError: TypeError('Object of type datetime is not JSON serializable')

When you look at the code for the async_check function:

json = json.dict(by_alias=True) if isinstance(json, BaseModel) else json

You will see that the .dict() method is called, which will return a Python dict, not a JSON-safe dict (which is what the json variable implies). This "json" object gets propogated all the way to httpx, and when it tries to JSON encode it, it fails.

Some ideas for how to fix this:

  1. Pydantic does have a .json() function, though it returns a string and not a dict. We could replace every call to json.dict(...) with loads(json.json(...)), where loads is the json.loads method from the stdlib, though there are a lot of API calls we would have to do this for, and it would introduce some overhead.
  2. Write a function that walks a dict object that stringifies datetime objects using .isoformat(). By adding said function right here (and to all the other API helper functions):

    githubkit/githubkit/core.py

    Lines 346 to 356 in bc1eb96

    raw_resp = await self._arequest(
    method,
    url,
    params=params,
    content=content,
    data=data,
    files=files,
    json=json,
    headers=headers,
    cookies=cookies,
    )

    We could stringify all datetime objects without changing every API function. There might be other object types other than datetimes which we might also want to support, datetimes just happened to be the ones that where giving me trouble.

I really like this library, but this bug is keeping me from using it in production 😢. I wouldn't mind opening a PR for this, it sounds easy enough, assuming we go with option 2.

@yanyongyu yanyongyu added the bug Something isn't working label Mar 21, 2023
@yanyongyu
Copy link
Owner

yanyongyu commented Mar 21, 2023

It seems httpx does not provide a way to custom json encoder. We should do this type transform before using httpx.

https://github.com/encode/httpx/blob/f1157dbc4102ac8e227a0a0bb12a877f592eff58/httpx/_content.py#L176-L181

We can simply add a wrapper to json dict using pydantic's default encoder pydantic_encoder.

https://github.com/pydantic/pydantic/blob/e12352c8208348ccdfd090b047b62b06641efd64/pydantic/json.py#L72-L90

This wrapper may be added to both _request and _arequest function to take effect. PR welcome 👍🏻

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants