db migrations to support new webhook operations #17671
Conversation
| } | ||
|
|
||
| private void addWebhookOperationConfigColumn(final DSLContext ctx) { | ||
| ctx.alterTable("operation").addColumnIfNotExists(DSL.field("operator_webhook", |
There was a problem hiding this comment.
@mfsiega-airbyte one bit I'm not sure I understand - can you remind me again why we need to add this to the operation table? Is it so we have a way to indicate that the operation is of a specific webhook type?
There was a problem hiding this comment.
Basically: different connections might want different configs (and that happens by creating different entries in the operation table).
There are a few reasons:
- For one, we need to say "use the webhook for this connection (and not that connection)". So we need to create an operation, and then the connection has a list of operations to execute, so we can indicate that we want to use the operation with a particular connection.
- We might also want to e.g., change the parameters between connections.
There was a problem hiding this comment.
That sounds like something better suited for the connection_operation table. Maybe worth a quick sync tmrw?
There was a problem hiding this comment.
The connection_operation table today just holds:
- id
- connection_id
- operation_id
- created_at/updated_at
essentially just hooking up connections and operations. (I'm not 100% sure why the connection_operation table exists, instead of just having an operation_ids column on the connection table, but I assume there's a good reason).
Whereas the operation table is where e.g., the existing dbt transformation config lives (which is mostly how I decided to put it there).
Regardless, +1 to syncing tomorrow - maybe good to run through the whole thing e2e for some early feedback as well.
There was a problem hiding this comment.
spoke offline: this is column is needed so webhook configs are stored on the operation table.
…vation * master: (22 commits) Update full-refresh-append.md (#17784) Update full-refresh-overwrite.md (#17783) Update incremental-append.md (#17785) Update incremental-deduped-history.md (#17786) Update cdc.md (#17787) 🪟 🔧 Ignore classnames during jest snapshot comparison (#17773) feat: replace openjdk with amazoncorretto:17.0.4 on connectors for seсurity compliance (#17511) Start testing buildpulse. (#17712) Add missing types to the registry (#17763) jobs db descriptions (#16543) config db data catalog (#16427) Update lowcode docs (#17752) db migrations to support new webhook operations (#17671) Bump Airbyte version from 0.40.13 to 0.40.14 (#17762) September Release Notes (#17754) Revert "Use java-datadog-tracer-base image (#17625)" (#17759) Add connection migrations for schema changes (#17651) Connection Form Refactor - Part Two (#16748) Improve E2E testing around the Connection Form (#17577) Bump strict encrypt version (#17747) ...
What
Db updates to support new webhook operations.
How
Spec: https://docs.google.com/document/d/129NhHfCRq5jo3eSFtL_P6dYaLCFrFiU1QniXapp-Uqk