Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #20044] generate pydantic models from handwritten schema #20475

Conversation

maxi297
Copy link
Contributor

@maxi297 maxi297 commented Dec 14, 2022

What

This PR addressed this ticket. It will allow for automatic model generation based on the handwritten component manifest.

How

The generations follows the same life-cycle as the generation of the Airbyte protocol files i.e.:

Manually
Run ./gradlew generateComponentManifestClassFiles

Automatically (as part of GitHub "Connectors Base" check)
When the check runs:

  • Step "Format" will generate the code and format it
  • Step "Ensure no file change from regenerating connector definitions/specs" will ensure that there are no diffs between the newly generated files and the one currently in the repo

Note that airbyte-cdk/python/airbyte_cdk/sources/declarative/models/__init__.py has been excluded from flake8 because it generated error F401 and F403.

Recommended reading order

airbyte-cdk/python/airbyte_cdk/sources/declarative/models/low_code_component_schema.py has been autogenerated and the validation on the generation is part of issue #19759

🚨 User Impact 🚨

None yet. Eventually, we will use the generated model in the factory that generates our Python objects but this is for a later issue

brianjlai and others added 30 commits December 6, 2022 23:42
- ApiKeyAuthenticator
- BasicHttpAuthenticator
- BearerAuthenticator
- DeclarativeOauth2Authenticator
- NoAuth
…rm validation against the handwritten schema
build.gradle Show resolved Hide resolved
Copy link
Contributor

@brianjlai brianjlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! I just had a few follow up questions related to your comments.

Also, can you update the airbyte-cdk readme with how to run the model generation command locally as well since it will start being part of our workflow

build.gradle Show resolved Hide resolved
.pre-commit-config.yaml Show resolved Hide resolved
@maxi297 maxi297 temporarily deployed to more-secrets December 15, 2022 15:43 — with GitHub Actions Inactive
@maxi297 maxi297 temporarily deployed to more-secrets December 15, 2022 15:43 — with GitHub Actions Inactive
@maxi297 maxi297 force-pushed the issue-20044_generate-pydantic-models-from-handwritten-schema branch from ddf2d94 to 4471266 Compare December 15, 2022 15:52
@maxi297 maxi297 temporarily deployed to more-secrets December 15, 2022 15:53 — with GitHub Actions Inactive
@maxi297 maxi297 temporarily deployed to more-secrets December 15, 2022 15:54 — with GitHub Actions Inactive
@maxi297
Copy link
Contributor Author

maxi297 commented Dec 19, 2022

My bad for triggering the source acceptance test on this PR. It should have been on 20599

Base automatically changed from low_code_handwritten_component_schema to master December 19, 2022 20:42
@maxi297 maxi297 temporarily deployed to more-secrets December 20, 2022 15:37 — with GitHub Actions Inactive
@maxi297 maxi297 temporarily deployed to more-secrets December 20, 2022 15:37 — with GitHub Actions Inactive
@maxi297 maxi297 temporarily deployed to more-secrets December 20, 2022 20:58 — with GitHub Actions Inactive
@maxi297 maxi297 temporarily deployed to more-secrets December 20, 2022 20:59 — with GitHub Actions Inactive
@maxi297 maxi297 temporarily deployed to more-secrets December 20, 2022 21:41 — with GitHub Actions Inactive
@maxi297 maxi297 temporarily deployed to more-secrets December 20, 2022 21:41 — with GitHub Actions Inactive
@maxi297
Copy link
Contributor Author

maxi297 commented Dec 20, 2022

/approve-and-merge reason="This is tooling improvement that has no impact on the airbyte-cdk itself but will help our development team to iterate through the low-code project"

@octavia-approvington
Copy link
Contributor

What are we doing again?
merge and squash

@octavia-approvington octavia-approvington merged commit 1fdaa1d into master Dec 20, 2022
@octavia-approvington octavia-approvington deleted the issue-20044_generate-pydantic-models-from-handwritten-schema branch December 20, 2022 22:53
@maxi297
Copy link
Contributor Author

maxi297 commented Dec 20, 2022

Squash and merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CDK Connector Development Kit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants