[python-fastapi] Fix additionalProperties support & support pydantic v2

[mgoltzsche]

• Don't let generated model code extend the type specified within the additionalProperties schema (or the array item schema) but the pydantic BaseModel.
• Support additionalProperties the pydantic v2 way within __dict__: Previously, the generated additional_properties field showed up within the response of the generated API as opposed marshaling the model so that its fields are added to/embedded into the root object. Apparently that is because pydantic v2 does not honour the generated to_dict methods (which would have mapped the object to the correct representation) but, instead, supports additional properties natively by specifying extra=allow within the model_config. Correspondingly, the following changes have been applied:
  • To allow additional fields, specify extra=allow within the model_config.
  • Don't generate the additional_properties field but use __dict__ - users can use pydantic's built-in model.model_extra instead.
  • Don't generate the {to|from}_{dict|json} methods since pydantic is taking care of the model mapping based on the declared fields and model_config - users can use pydantic's model.model_dump[_json] instead.
  • Let the generated {to|from}_{dict|json} methods delegate to Pydantic's model_dump[_json] methods.
  • Let the generated API endpoints enable the exclude_unset response marshalling option in order to omit fields from the response that weren't explicitly set by the code. This is so that non-required fields don't show up with null values within the response (which would be invalid according to the OpenAPI spec, unless those fields are explicitly marked as nullable within the OpenAPI schema).
• Support oneOf and anyOf schemas the Pydantic v2 way:
  • Generate (discriminated) Unions and leave it to Pydantic to validate and figure out the type.
  • Generate model constructors that forcefully set the discriminator field in order to ensure it is included in the marshalled representation (otherwise it would be ignored when not explicitly set due to the exclude_unset marshalling option being enabled).

Closes #19311
Relates to #17703 (might also close that one)

cc @cbornet @tomplus @krjakbrjak @fa0311 @multani

PR checklist

• Read the contribution guidelines.
• Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
• Run the following to build the project and update samples:

  ./mvnw clean package 
  ./bin/generate-samples.sh ./bin/configs/*.yaml
  ./bin/utils/export_docs_generators.sh
  

  (For Windows users, please run the script in Git BASH)
  Commit all changed files.
  This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
  These must match the expectations made by your contribution.
  You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
  IMPORTANT: Do NOT purge/delete any folders/files (e.g. tests) when regenerating the samples as manually written tests may be removed.
• File the PR against the correct branch: master (upcoming 7.6.0 minor release - breaking changes with fallbacks), 8.0.x (breaking changes without fallbacks)
• If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

[denosaurtrain]

Thanks for the PR! I'll need something like this PR to move off python-pydantic-v1.

I have a question about this bit:

Don't generate the {to|from}_{dict|json} methods since pydantic is taking care of the model mapping based on the declared fields and model_config - users can use pydantic's model.model_dump[_json] instead.

Why remove {to|from}_{dict|json} (and to_str) rather than replace the body of those methods with the right pydantic calls? I imagine this could be a substantial breaking change for some folks.

[mgoltzsche]

Why remove {to|from}_{dict|json} (and to_str) rather than replace the body of those methods with the right pydantic calls? I imagine this could be a substantial breaking change for some folks.

The idea is to let the generated code fully leverage the fastapi/pydantic platform idioms, which include letting the route methods return typed objects that pydantic then maps and validates based on the type annotation.
I didn't want to bloat the generated code unnecessarily and I was thinking the python-fastapi generator is young enough so that I could still make such a breaking change. However, to make it backward-compatible, I'll adjust the PR to let it generate those methods again but letting them delegate to pydantic's methods...

[mgoltzsche]

I updated the PR correspondingly but unfortunately this is still a breaking change since

• nullable field semantics are not honoured anymore - would need to be generated into the type annotation like this maybe or applied via the exclude_unset marshalling option.
• Discriminator logic for oneOf schemas is not generated - would need to be generated into the type annotation like this.

I can look into making the adjustments some other evening.

[denosaurtrain]

I like it! I anticipate using this as a client to replace python-pydantic-v1. Hopefully a maintainer can chime in/approve when you are ready!

[mgoltzsche]

I also made the generator generate oneOf and anyOf schemas as (discriminated) Unions the Pydantic v2 way and enabled the exclude_unset response option for generated API routes/endpoints (PR description updated correspondingly). The PR is ready for review now.

[denosaurtrain]

Epic! I hope a maintainer can review it for you soon.