Excluding attributes on an attrs model from unstructuring breaks auto unstructuring

[mitchej123]

• cattrs version: 1.8.0
• Python version: 3.9.2
• Operating System: OSX - Big Sur 11.5.2

Description

I have a field on an attrs model that I want excluded when unstructuring. When registering an unstructure hook using make_dict_unstructure_fn and field=override(Skip=True) (as read in the documentation) the other unstructure hooks aren't being triggered (UUID, nested models, etc). The same issue happens if the example using register_unstructure_hook_factory is used instead.

What I Did

Sample program to demonstrate the issue. Running json.dumps on the unstructured Breaks will break complaining that Object of type UUID is not JSON serializable (or if the ID is commented out then Object of type Nested), whereas it works properly on Works

from __future__ import annotations

import json
from typing import Optional
from uuid import UUID, uuid4

import attr
from cattr import GenConverter
from cattr.gen import make_dict_unstructure_fn, override


@attr.s(auto_attribs=True, slots=True, frozen=True)
class Nested:
    nested_string: Optional[str] = attr.ib(factory=lambda: str(uuid4()))


@attr.s(auto_attribs=True, slots=True, frozen=True)
class Works(object):
    id: UUID = attr.ib(factory=uuid4)
    nested: Nested = attr.ib(factory=Nested)


@attr.s(auto_attribs=True, slots=True, frozen=True)
class Breaks(object):
    id: UUID = attr.ib(factory=uuid4)
    exclude_me: str = attr.ib(factory=lambda: str(uuid4()))
    nested: Nested = attr.ib(factory=Nested)


def _gen_conv() -> GenConverter:
    _conv = GenConverter()
    _conv.register_unstructure_hook(frozenset, lambda v: list(v))
    _conv.register_unstructure_hook(UUID, lambda v: str(v))

    # This handles skipping `exclude_me` properly, but other nested attr classes don't get properly unstructured
    _conv.register_unstructure_hook(Breaks, make_dict_unstructure_fn(Breaks, _conv, exclude_me=override(omit=True)))

    return _conv


def unstructure_breaks(breaks):
    doesnt_break = conv.unstructure(breaks)
    doesnt_break.pop("exclude_me", None)
    return doesnt_break


if __name__ == '__main__':
    conv = _gen_conv()
    print(json.dumps(conv.unstructure(Works())))
    print(json.dumps(conv.unstructure(Breaks()))) # Breaks because it can't serialize UUID/Nested

❯ python cattr_poc.py
{"id": "dbab1954-5338-40f1-a9f8-9ba8ec5a280d", "nested": {"nested_string": "e0317c43-c06e-44c8-b6df-fd992649a57c"}}
Traceback (most recent call last):
  File "/Users/jason/dev/cattr_poc.py", line 50, in <module>
    print(json.dumps(unstructure_breaks(Breaks())))
  File "/Users/jason/.pyenv/versions/3.9.2/lib/python3.9/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/Users/jason/.pyenv/versions/3.9.2/lib/python3.9/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/Users/jason/.pyenv/versions/3.9.2/lib/python3.9/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/Users/jason/.pyenv/versions/3.9.2/lib/python3.9/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type UUID is not JSON serializable

What I would expect

I'm looking for a good way to exclude one or more attributes from unstructuring an attrs model without needing to wrap them in a function like this:

def unstructure_breaks(breaks):
    doesnt_break = conv.unstructure(breaks)
    doesnt_break.pop("exclude_me", None)
    return doesnt_break

[Tinche]

Hello,

I'm on my honeymoon atm so my access to a computer is limited. Does it work if you remove the __future__ import?

[mitchej123]

I'm on my honeymoon atm so my access to a computer is limited.

No worries!

Does it work if you remove the __future__ import?

Yes, if I remove the __future__ import it works. You mentioning that made me think of another similar issue that I wasn't able to reproduce in a minimal fashion (but was able to work around), and it looks like the same underlying issue and workaround! The attr.field() is returning {str} UUID instead of <Class UUID>, so the dispatch isn't able to find the <Class UUID> converter.

If I leave the __future__ import in, and change the _gen_conv function as follows it works as expected

def _gen_conv() -> GenConverter:
    _conv = GenConverter()
    _conv.register_unstructure_hook(frozenset, lambda v: list(v))
    _conv.register_unstructure_hook(UUID, lambda v: str(v))

    attr.resolve_types(Breaks) # Force resolution of the types so it doesn't fall back to string types
    _conv.register_unstructure_hook(Breaks, make_dict_unstructure_fn(Breaks, _conv, exclude_me=override(omit=True)))

    return _conv

Evidence that it's only finding a str type

Setting a breakpoint here, and looking at attr.fields(obj.__class__)

    def unstructure(self, obj: Any, unstructure_as=None) -> Any:
        return self._unstructure_func.dispatch(
            obj.__class__ if unstructure_as is None else unstructure_as
        )(obj)

[Tinche]

Yeah, I will add annotation resolution to the gen functions in the next release. In the meantime, you can work around the issue by calling it yourself, as mentioned :)

[Tinche]

This should work now on master, the gen functions will call resolve_types for you.

[mitchej123]

Awesome thanks, will give it a try.

[anthrotype]

@Tinche Any chances that we can have a new release soon that includes this patch? Thanks in advace

[Tinche]

Yeah, there will be a release this week!

[praateekmahajan]

Just saw this issue, seems similar to something I reported on attrs python-attrs/attrs#901

@Tinche any thoughts on that?

[Tinche]

This should now work with the latest release, forgot to close.