CDK: Create a generic method for transforming records fields in accordance with streams schemas #5224
Labels
area/connectors
Connector related issues
CDK
Connector Development Kit
connectors/source/facebook-marketing
connectors/sources-api
lang/python
type/enhancement
New feature or request
Milestone
Tell us about the problem you're trying to solve
Right now we have few issues where fields types in records don't match types from streams schemas (#4292, #4841, #4827).
Describe the solution you’d like
It would be nice to have a generic solution, a function or a class (like
Transformer
class in singer), which will go through each field in records (through nested fields too) and transform types to the ones in streams schemas.Describe the alternative you’ve considered or used
Right now if we want to change type of fields in records we are using
transform()
function (like in source-github). In this case we need to manually specify fields and desired types. It works good for root keys, but for nested keys it's not so convenient to use such method.Additional context
First edition of this task will be implemented in #4827 issue.
Are you willing to submit a PR?
Yes
The text was updated successfully, but these errors were encountered: