Skip to content

Commit

Permalink
Generate seed connector specs on build (airbytehq#7501)
Browse files Browse the repository at this point in the history
* add specs module with logic to fetch specs on build

* format + build and add gradle dependency for new script

* check seed file for existing specs + refactor

* add tests + a bit more refactoring

* run gw format

* update yaml config persistence to merge specs into definitions

* add comment

* delete secrets migration to be consistent with master

* add dep

* add tests for GcsBucketSpecFetcher

* get rid of static block + format

* DRY up parse call

* add GCS details to comment

* formatting + fix test

* update comment

* do not format seed specs files

* change signature of run to allow cloud to reuse this script

* run gw format

* revert commits that change signature of run

* fix comment typo

Co-authored-by: Davin Chia <davinchia@gmail.com>

* rename enum to be distinct from the enum in cloud

* add missing dependencies between modules

* add readme for seed connector spec generator

* reword

* reference readme in comment

* ignore 'spec' field in newFields logic

Co-authored-by: Davin Chia <davinchia@gmail.com>
  • Loading branch information
2 people authored and schlattk committed Jan 4, 2022
1 parent fadecd9 commit 1a64a74
Show file tree
Hide file tree
Showing 23 changed files with 9,185 additions and 69 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@
public enum SeedType {

STANDARD_SOURCE_DEFINITION("/seed/source_definitions.yaml", "sourceDefinitionId"),
STANDARD_DESTINATION_DEFINITION("/seed/destination_definitions.yaml", "destinationDefinitionId");
STANDARD_DESTINATION_DEFINITION("/seed/destination_definitions.yaml", "destinationDefinitionId"),
SOURCE_SPEC("/seed/source_specs.yaml", "dockerImage"),
DESTINATION_SPEC("/seed/destination_specs.yaml", "dockerImage");

final String resourcePath;
// ID field name
Expand Down
Loading

0 comments on commit 1a64a74

Please sign in to comment.