Update OpenAPI Generator to v5.3.0

[palnabarun]

What is the feature and why do you need it:

OpenAPI Generator has got some improvements that we should pull in. Doing that will resolve issues like #859, #1131.

Risks

There have a lot!! of changes since v4.3.0. We would need to go through most of those changes through the release notes and see if anything can break the client.

Describe the solution you'd like to see:

Update OpenAPI Generator to v5.3.0 in https://github.com/kubernetes-client/gen/blob/a3aef4de7a1d5dab72021aa282fffd8bc8a022ca/openapi/python.sh#L48

[itaru2622]

@palnabarun It also needs kubernetes-client/gen#208, when I tested with openapi-generator v5.3.0.

actually I would like you to support openapi-generator v5.3.1, for #1579, which is expected to be released in near future.

[itaru2622]

When I tested with openapi-generator v5.3.0, I found that
A) openapi's code generation was succeeded when gen#208(+gen#209) involved.
B) scripts/release.sh is failed in patching process by scripts/*

I found there are lots of patches(B) against generated codes(A).
some of those(B) may be acceptable to openapi-generator when PR submitted, but others may not.

so, one of the solutions is: fork openapi-generator and apply patches to forked repository.
then, it is possible to get originally generated codes and also keep up with upstream changes by rebasing.

To change code generation, It just needs to modify https://github.com/OpenAPITools/openapi-generator/tree/master/modules/openapi-generator/src/main/resources/python/*

those files are mustache based, but most of part is usual python code.
for example, one line of #1579 is: https://github.com/OpenAPITools/openapi-generator/blob/3a667784acab58d5107daf557cd1ba7bb679e0a1/modules/openapi-generator/src/main/resources/python/rest.mustache#L69

openapi-generator also provides the way to use proprietary template with their official logic by 'generate -t' option as below.
in this case,

• it needs just partial folder (openapi-generator/modules/openapi-generator/src/main/resources/python/*).
• it needs some method to keep up with upstream(openapi) changes in their template folder.

dir_myTemplate=/my-openapi-generator/modules/openapi-generator/src/main/resources/python
docker run -v ${dir_myTemplate}:/myTemplate openapi-generator:v5.3.0 generate -t /myTemplate ....

[roycaihw]

/help

@itaru2622 Are you willing to drive the upgrade?

cc @roycaihw

[k8s-ci-robot]

@roycaihw:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

@itaru2622 Are you willing to drive the upgrade?

cc @roycaihw

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[itaru2622]

@roycaihw You made me so suprised :-)
Unfortunately I don't have much time now, so it is hard to drive this issue solving. but I may be able to help you.

It is better to note that openapi-generator's bump speed is quite fast as described below.

https://github.com/OpenAPITools/openapi-generator/milestones

[idsvandermolen]

One of the interesting things I think was introduced in OpenAPI Generator 5 was improved validation of types, for example the enum type of protocol in ContainerPort (https://github.com/kubernetes-client/python/blob/master/kubernetes/client/models/v1_container_port.py#L169) currently isn't validated to have a correct value. When generated with version 5 afaict it will be validated to contain one of the correct enumeration values (SCTP,TCP,UDP).
If kubernetes-client/python get's this kind of validation, it will catch issues early (shift left) and when used to generate manifests, there's no more need for kubeval or kubeconform for validation.

[itaru2622]

@idsvandermolen yes, there are some improvement in openapi-generator.

unfortunately, to get benefit of those improvements on validation, it needs to update openapi defenition which is managed by server side at https://github.com/kubernetes/kubernetes/blob/master/api/openapi-spec/swagger.json

it may take to be adjusted.

[idsvandermolen]

unfortunately, to get benefit of those improvements on validation, it needs to update openapi defenition which is managed by server side at https://github.com/kubernetes/kubernetes/blob/master/api/openapi-spec/swagger.json

it may take to be adjusted.

true, but I noticed for example the upstream swagger.json already has defined enum type for container port protocol field. Afaict this is not validated in current python kubernetes client, which is based on OpenAPI generator 4.x.
But it should be "automatically" validated when it uses OpenAPI generator 5.x. I did a simplistic generation of python code with latest 5.3.1-snapshot and validation logic for enums is added to the python model.
So that would already be a nice benefit.

(I did see the "same" protocol field elsewhere in the upstream swagger.json is still a simple string type, so indeed that should be fixed upstream)

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[roycaihw]

/remove-lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[rosscdh]

So it sounds like we just have to manually patch the python k client in the mean time?
#1028
#740