You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
a user of the connector builder UI needs the ability to specify stream schemas in the UI such that when they click "export connector" the schemas they specified in the UI are the same ones that actually run in the connector itself outside of the connector builder.
Implementation options
Bundle into manifest
The most obvious way to achieve this is to bundle schema definitions into the manifest itself. This way the user maintains only one file they download from the UI, which they copy-paste into their connector directory to build the connector.
The pro of this approach is that it is the most portable/lightweight
The con is that it makes the YAML file very large/maybe unreadable
Implementation steps:
Create StaticSchemaLoader class which takes a schema mapping object as parameter
Add StaticSchemaLoader to the class type registry
Update one of the connectors to use the new schema definition:
Continue using schemas/ folders pattern
Instead of bundling, we could also change the behavior of the "export YAML" button to download instead a zip folder which contains the YAML manifest in addition to a schemas/ folder, containing one .json schema file per stream. This would mirror exactly the way we typically specify schemas today.
The pro of this approach is that it keeps the YAML file clean
The con of this approach is that it complicates how a user consumes a downloaded connector
Acceptance Criteria
Follow implementation steps for "bundle" option 1 above
At least 1 existing connector defines its stream schemas inline
The output of the discover method must be unchanged for connectors we migrated
Out of scope:
Schema auto-detection
Open questions
do we want to rename SchemaLoader to Schema since we're not always loading it from a file?
The text was updated successfully, but these errors were encountered:
Tell us about the problem you're trying to solve
a user of the connector builder UI needs the ability to specify stream schemas in the UI such that when they click "export connector" the schemas they specified in the UI are the same ones that actually run in the connector itself outside of the connector builder.
Implementation options
Bundle into manifest
The most obvious way to achieve this is to bundle schema definitions into the manifest itself. This way the user maintains only one file they download from the UI, which they copy-paste into their connector directory to build the connector.
The pro of this approach is that it is the most portable/lightweight
The con is that it makes the YAML file very large/maybe unreadable
Implementation steps:
StaticSchemaLoader
class which takes a schema mapping object as parameterStaticSchemaLoader
to the class type registryContinue using
schemas/
folders patternInstead of bundling, we could also change the behavior of the "export YAML" button to download instead a zip folder which contains the YAML manifest in addition to a
schemas/
folder, containing one.json
schema file per stream. This would mirror exactly the way we typically specify schemas today.The pro of this approach is that it keeps the YAML file clean
The con of this approach is that it complicates how a user consumes a downloaded connector
Acceptance Criteria
Out of scope:
Open questions
SchemaLoader
toSchema
since we're not always loading it from a file?The text was updated successfully, but these errors were encountered: