-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove NormalizationRunnerFactory #20245
Conversation
…DBT and normalizationImage fields. Added to the GenerateInputActivityImpl and TemporalClient classes code parts for read destination_definition.yaml and get suportDBT and normalizationImage fields. Added logging and comparing normalization images from NormalizationRunnerFactory and destination_definition.yaml
…korotkov/remove_factory � Conflicts: � airbyte-commons-worker/src/main/java/io/airbyte/workers/normalization/NormalizationRunnerFactory.java
…korotkov/remove_factory
…remove_factory � Conflicts: � airbyte-commons-worker/src/main/java/io/airbyte/workers/normalization/NormalizationRunnerFactory.java � airbyte-commons-worker/src/test/java/io/airbyte/workers/normalization/NormalizationRunnerFactoryTest.java � airbyte-container-orchestrator/src/main/java/io/airbyte/container_orchestrator/DbtJobOrchestrator.java � airbyte-container-orchestrator/src/main/java/io/airbyte/container_orchestrator/NormalizationJobOrchestrator.java � airbyte-integrations/bases/standard-destination-test/src/main/java/io/airbyte/integrations/standardtest/destination/DestinationAcceptanceTest.java � airbyte-workers/src/main/java/io/airbyte/workers/temporal/sync/DbtTransformationActivityImpl.java � airbyte-workers/src/main/java/io/airbyte/workers/temporal/sync/NormalizationActivityImpl.java
…ctor_definition table
@gosusnp, I listened to your comments and moved |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the update, looks good to me with a couple of small details with the extraction some descriptions became obsolete, also our coding style is to use explicit import rather than wilcards.
...-config/config-models/src/main/resources/types/NormalizationDestinationDefinitionConfig.yaml
Outdated
Show resolved
Hide resolved
.../config-persistence/src/main/java/io/airbyte/config/persistence/ActorDefinitionMigrator.java
Outdated
Show resolved
Hide resolved
airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/ConfigWriter.java
Show resolved
Hide resolved
airbyte-config/config-persistence/src/main/java/io/airbyte/config/persistence/DbConverter.java
Outdated
Show resolved
Hide resolved
This reverts commit eecfafd.
What
Removed NormalizationRunnerFactory.class and updated all places where it using. Updated DefaultNormalizationRunner.class and changed
NormalizationRunnerFactory#create
tonew DefaultNormalizationRunner
. Updated DestinationAcceptanceTest and added methods for read inaormation about normalization and DBT from definition file. Updated check_images_exist.sh to get information fromdestination_definition.yaml
file instead of NormalizationRunnerFactory classHow
Describe the solution
Recommended reading order
DefaultNormalizationRunner.java
DestinationAcceptanceTest.java
check_images_exist.sh
🚨 User Impact 🚨
Removed NormalizationRunnerFactory.class. Thereby we separate the normalization from the platform.
Pre-merge Checklist
Expand the relevant checklist and delete the others.
New Connector
Community member or Airbyter
airbyte_secret
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.README.md
bootstrap.md
. See description and examplesdocs/integrations/<source or destination>/<name>.md
including changelog. See changelog exampledocs/integrations/README.md
airbyte-integrations/builds.md
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing/publish
command described hereUpdating a connector
Community member or Airbyter
airbyte_secret
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.README.md
bootstrap.md
. See description and examplesdocs/integrations/<source or destination>/<name>.md
including changelog. See changelog exampleAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing/publish
command described hereConnector Generator
-scaffold
in their name) have been updated with the latest scaffold by running./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates
then checking in your changesTests
Unit
Put your unit tests output here.
Integration
Put your integration tests output here.
Acceptance
Put your acceptance tests output here.