[Artifacts] Support deserializing additional fields in BaseResource for forward compatibility within a major version

[MichelleArk]

Housekeeping

• I am a maintainer of dbt-core

Short description

A subclass of a BaseResource should be able to make forward-compatible changes (adding a field or deleting a field with a default value) within and a major version and have previous code successfully deserialize the payload.

This requires the previous code to ignore additional fields during deserialization which we do not currently have in place, but can be introduced in __pre_deserialize__ on BaseResource.

Acceptance criteria

• any subclass of a BaseResource can be extended with a new field and old writers can successfully deserialize it
• any subclass of a BaseResource can delete a field with a default and old writers can successfully deserialize it (should already work)

Suggested Tests

1. Test that a (mocked) new minor version of a BaseResource (serialized with a value for a new optional field) can be deserialized successfully. e.g. something like PreviousBaseResource.from_dict(CurrentBaseResource(...).to_dict())
2. Test that a (mocked) new minor version of a BaseResource (serialized without a field that was previously optional) can be deserialized successfully.

Impact to Other Teams

N/A

Will backports be required?

No

Context

Useful docs on forward/backward schema compatability: https://developer.confluent.io/patterns/event-stream/schema-compatibility/#:~:text=The%20types%20of%20backwards%20compatible,readers%20use%20the%20default%20value

• Consider using a flag (e.g ignore_additional_fields) in to_dict/from_dict to inform whether or not we should actually ignore additional fields. Could mashumaro could support this option natively?

[Fatal1ty]

Hey @MichelleArk

I just want to say that mashumaro supports forward compatibility out of the box, so you don’t need any specific code for that:

• from_dict ignores any additional keys not related with existing dataclass fields
• from_dict will use the default value if there is no key for the field in the input dict

Maybe I'm missing something and you currently have something extra that breaks forward compatibility on your side?