Skip to content

Commit

Permalink
Make validation notes pickable.
Browse files Browse the repository at this point in the history
Certain test runners pickle the exceptions raised to accumulate them from
multiple processes.

For exceptions with `AttributeValidationNote` and/or `IterableValidationNote`
attached, because we have overridden `__new__` on these classes, pickle cannot
correctly unpickle them with the required arguments. For this use case,
exceptions being pickable is necessary to accumulate errors with the
`multiprocessing` setup.

This implements `__getnewargs__` to fix this problem.

Certain test runners that pickle the exceptions raised to accumulate
them from multiple processes. For exceptions with `AttributeValidationNote`
and/or `IterableValidationNote`, because we have overriden __new__ on
these classes, pickle is not able to correctly unpickle them with the required
arguments. This implements __getnewargs__ to fix this problem.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
  • Loading branch information
PIG208 committed Aug 5, 2023
1 parent d1b106a commit 14e961b
Show file tree
Hide file tree
Showing 3 changed files with 32 additions and 1 deletion.
2 changes: 2 additions & 0 deletions HISTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@
([#398](https://github.com/python-attrs/cattrs/issues/398) [#399](https://github.com/python-attrs/cattrs/pull/399))
- Broaden loads' type definition for the preconf orjson converter.
([#400](https://github.com/python-attrs/cattrs/pull/400))
- `AttributeValidationNote` and `IterableValidationNote` are now picklable.
([#408](https://github.com/python-attrs/cattrs/pull/408))

## 23.1.2 (2023-06-02)

Expand Down
6 changes: 6 additions & 0 deletions src/cattrs/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ def __new__(
instance.type = type
return instance

def __getnewargs__(self) -> Tuple[str, Union[int, str], Any]:
return (str(self), self.index, self.type)


class IterableValidationError(BaseValidationError):
"""Raised when structuring an iterable."""
Expand Down Expand Up @@ -76,6 +79,9 @@ def __new__(cls, string: str, name: str, type: Any) -> "AttributeValidationNote"
instance.type = type
return instance

def __getnewargs__(self) -> Tuple[str, str, Any]:
return (str(self), self.name, self.type)


class ClassValidationError(BaseValidationError):
"""Raised when validating a class if any attributes are invalid."""
Expand Down
25 changes: 24 additions & 1 deletion tests/test_validation.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,19 @@
from typing import Dict, FrozenSet, List, Set, Tuple

import pytest
import pickle
from attrs import define, field
from attrs.validators import in_
from hypothesis import given

from cattrs import Converter
from cattrs._compat import Counter
from cattrs.errors import ClassValidationError, IterableValidationError
from cattrs.errors import (
AttributeValidationNote,
ClassValidationError,
IterableValidationError,
IterableValidationNote,
)


def test_class_validation():
Expand Down Expand Up @@ -178,3 +184,20 @@ def test_hetero_tuple_validation():
assert exc.value.exceptions[0].__notes__ == [
"Structuring typing.Tuple[int, int, int] @ index 2"
]


def test_notes_pickling():
"""Validation notes should be picklable"""
note = pickle.loads( # noqa: S301
pickle.dumps(IterableValidationNote("foo", "key", str))
)
assert note == "foo"
assert note.index == "key"
assert note.type is str

note = pickle.loads( # noqa: S301
pickle.dumps(AttributeValidationNote("foo", "name", int))
)
assert note == "foo"
assert note.name == "name"
assert note.type is int

0 comments on commit 14e961b

Please sign in to comment.