2021-09-13 07:12:45 INFO () WorkerRun(call):62 - Executing worker wrapper. Airbyte version: 0.29.17-alpha 2021-09-13 07:12:46 INFO () TemporalAttemptExecution(get):114 - Executing worker wrapper. Airbyte version: 0.29.17-alpha 2021-09-13 07:12:46 WARN () Databases(createPostgresDatabaseWithRetry):57 - Waiting for database to become available... 2021-09-13 07:12:46 INFO () JobsDatabaseInstance(lambda$static$2):45 - Testing if jobs database is ready... 2021-09-13 07:12:47 INFO () Databases(createPostgresDatabaseWithRetry):74 - Database available! 2021-09-13 07:12:47 WARN () JsonMetaSchema(newValidator):338 - Unknown keyword example - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2021-09-13 07:12:47 INFO () DefaultReplicationWorker(run):102 - start sync worker. job id: 1 attempt id: 0 2021-09-13 07:12:47 INFO () DefaultReplicationWorker(run):111 - configured sync modes: {null.issues=incremental - append_dedup} 2021-09-13 07:12:47 INFO () DefaultAirbyteDestination(start):78 - Running destination... 2021-09-13 07:12:47 INFO () LineGobbler(voidCall):85 - Checking if airbyte/destination-postgres:0.3.11 exists... 2021-09-13 07:12:47 INFO () LineGobbler(voidCall):85 - airbyte/destination-postgres:0.3.11 was found locally. 2021-09-13 07:12:47 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/1/0 --network host --log-driver none airbyte/destination-postgres:0.3.11 write --config destination_config.json --catalog destination_catalog.json 2021-09-13 07:12:47 INFO () LineGobbler(voidCall):85 - Checking if airbyte/source-jira:0.2.10 exists... 2021-09-13 07:12:47 INFO () LineGobbler(voidCall):85 - airbyte/source-jira:0.2.10 was found locally. 2021-09-13 07:12:47 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/1/0 --network host --log-driver none airbyte/source-jira:0.2.10 read --config source_config.json --catalog source_catalog.json 2021-09-13 07:12:47 INFO () DefaultReplicationWorker(lambda$getDestinationOutputRunnable$3):246 - Destination output thread started. 2021-09-13 07:12:47 INFO () DefaultReplicationWorker(run):139 - Waiting for source thread to join. 2021-09-13 07:12:47 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):210 - Replication thread started. 2021-09-13 07:12:49 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:49 INFO i.a.i.d.p.PostgresDestination(main):85 - {} - starting destination: class io.airbyte.integrations.destination.postgres.PostgresDestination 2021-09-13 07:12:50 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Starting syncing SourceJira 2021-09-13 07:12:50 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Syncing stream: issues 2021-09-13 07:12:50 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:50 INFO i.a.i.b.IntegrationRunner(run):96 - {} - Running integration: io.airbyte.integrations.base.ssh.SshWrappedDestination 2021-09-13 07:12:50 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:50 INFO i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2021-09-13 07:12:50 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:50 INFO i.a.i.b.IntegrationRunner(run):100 - {} - Command: WRITE 2021-09-13 07:12:50 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:50 INFO i.a.i.b.IntegrationRunner(run):101 - {} - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2021-09-13 07:12:50 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:50 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2021-09-13 07:12:50 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:50 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2021-09-13 07:12:50 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:50 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2021-09-13 07:12:50 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:50 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword multiline - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2021-09-13 07:12:51 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:51 INFO i.a.i.b.s.SshTunnel(getInstance):190 - {} - Starting connection with method: NO_TUNNEL 2021-09-13 07:12:51 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:51 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$toWriteConfig$0):116 - {} - Write config: WriteConfig{streamName=issues, namespace=null, outputSchemaName=public, tmpTableName=_airbyte_tmp_vuy_issues, outputTableName=_airbyte_raw_issues, syncMode=append_dedup} 2021-09-13 07:12:51 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:51 INFO i.a.i.d.b.BufferedStreamConsumer(startTracked):143 - {} - class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started. 2021-09-13 07:12:51 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:51 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):139 - {} - Preparing tmp tables in destination started for 1 streams 2021-09-13 07:12:51 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:51 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):143 - {} - Preparing tmp table in destination started for stream issues. schema: public, tmp table name: _airbyte_tmp_vuy_issues 2021-09-13 07:12:51 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:12:51 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onStartFunction$1):149 - {} - Preparing tables in destination completed. 2021-09-13 07:13:00 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 1000 2021-09-13 07:13:14 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 2000 2021-09-13 07:13:26 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 3000 2021-09-13 07:13:38 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 4000 2021-09-13 07:13:50 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 5000 2021-09-13 07:14:03 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 6000 2021-09-13 07:14:16 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 7000 2021-09-13 07:14:28 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 8000 2021-09-13 07:14:39 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 9000 2021-09-13 07:14:50 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 10000 2021-09-13 07:15:01 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 11000 2021-09-13 07:15:12 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 12000 2021-09-13 07:15:23 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 13000 2021-09-13 07:15:33 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 14000 2021-09-13 07:15:45 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 15000 2021-09-13 07:15:57 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 16000 2021-09-13 07:16:09 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 17000 2021-09-13 07:16:21 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 18000 2021-09-13 07:16:35 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 19000 2021-09-13 07:16:46 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 20000 2021-09-13 07:16:58 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 21000 2021-09-13 07:17:10 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 22000 2021-09-13 07:17:22 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 23000 2021-09-13 07:17:34 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 24000 2021-09-13 07:17:43 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 25000 2021-09-13 07:17:54 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 26000 2021-09-13 07:18:05 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 27000 2021-09-13 07:18:15 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 28000 2021-09-13 07:18:25 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 29000 2021-09-13 07:18:38 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 30000 2021-09-13 07:18:50 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 31000 2021-09-13 07:19:01 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 32000 2021-09-13 07:19:12 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 33000 2021-09-13 07:19:23 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):223 - Records read: 34000 2021-09-13 07:19:33 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Setting state of issues stream to {'updated': '2021-09-13T10:16:22.302000+03:00'} 2021-09-13 07:19:33 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Read 34972 records from issues stream 2021-09-13 07:19:33 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Finished syncing SourceJira 2021-09-13 07:19:33 INFO () DefaultReplicationWorker(run):141 - Source thread complete. 2021-09-13 07:19:33 INFO () DefaultReplicationWorker(run):142 - Waiting for destination thread to join. 2021-09-13 07:19:33 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:19:33 INFO i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):80 - {} - Airbyte message consumer: succeeded. 2021-09-13 07:19:33 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:19:33 INFO i.a.i.d.b.BufferedStreamConsumer(close):212 - {} - executing on success close procedure. 2021-09-13 07:19:34 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:19:34 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):176 - {} - Finalizing tables in destination started for 1 streams 2021-09-13 07:19:34 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:19:34 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):181 - {} - Finalizing stream issues. schema public, tmp table _airbyte_tmp_vuy_issues, final table _airbyte_raw_issues 2021-09-13 07:19:34 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:19:34 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):194 - {} - Executing finalization of tables. 2021-09-13 07:19:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:19:36 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):196 - {} - Finalizing tables in destination completed. 2021-09-13 07:19:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:19:36 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):199 - {} - Cleaning tmp tables in destination started for 1 streams 2021-09-13 07:19:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:19:36 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):203 - {} - Cleaning tmp table in destination started for stream issues. schema public, tmp table name: _airbyte_tmp_vuy_issues 2021-09-13 07:19:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:19:36 INFO i.a.i.d.j.JdbcBufferedConsumerFactory(lambda$onCloseFunction$3):208 - {} - Cleaning tmp tables in destination completed. 2021-09-13 07:19:36 INFO () DefaultReplicationWorker(lambda$getDestinationOutputRunnable$3):251 - state in DefaultReplicationWorker from Destination: io.airbyte.protocol.models.AirbyteMessage@3d2c8e4c[type=STATE,log=,spec=,connectionStatus=,catalog=,record=,state=io.airbyte.protocol.models.AirbyteStateMessage@5f108dcf[data={"issues":{"updated":"2021-09-13T10:16:22.302000+03:00"}},additionalProperties={}],additionalProperties={}] 2021-09-13 07:19:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:19:36 INFO i.a.i.b.IntegrationRunner(run):153 - {} - Completed integration: io.airbyte.integrations.base.ssh.SshWrappedDestination 2021-09-13 07:19:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-13 07:19:36 INFO i.a.i.d.p.PostgresDestination(main):87 - {} - completed destination: class io.airbyte.integrations.destination.postgres.PostgresDestination 2021-09-13 07:19:36 INFO () DefaultReplicationWorker(run):144 - Destination thread complete. 2021-09-13 07:19:36 INFO () DefaultReplicationWorker(run):172 - sync summary: io.airbyte.config.ReplicationAttemptSummary@19edf97a[status=completed,recordsSynced=34972,bytesSynced=65440411,startTime=1631517167233,endTime=1631517576643] 2021-09-13 07:19:36 INFO () DefaultReplicationWorker(run):179 - Source output at least one state message 2021-09-13 07:19:36 INFO () DefaultReplicationWorker(run):185 - State capture: Updated state to: Optional[io.airbyte.config.State@23dd6533[state={"issues":{"updated":"2021-09-13T10:16:22.302000+03:00"}}]] 2021-09-13 07:19:36 INFO () TemporalAttemptExecution(get):135 - Stopping cancellation check scheduling... 2021-09-13 07:19:36 INFO () SyncWorkflow$ReplicationActivityImpl(replicate):184 - sync summary: io.airbyte.config.StandardSyncOutput@6dc0884c[standardSyncSummary=io.airbyte.config.StandardSyncSummary@448e1f0f[status=completed,recordsSynced=34972,bytesSynced=65440411,startTime=1631517167233,endTime=1631517576643],state=io.airbyte.config.State@23dd6533[state={"issues":{"updated":"2021-09-13T10:16:22.302000+03:00"}}],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@70dc8cd2[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@6984429c[stream=io.airbyte.protocol.models.AirbyteStream@ef3508[name=issues,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":"string","readOnly":true,"description":"The ID of the issue."},"key":{"type":"string","readOnly":true,"description":"The key of the issue."},"self":{"type":"string","format":"uri","readOnly":true,"description":"The URL of the issue details."},"names":{"type":"object","readOnly":true,"description":"The ID and name of each field present on the issue.","additionalProperties":{"type":"string","readOnly":true}},"expand":{"xml":{"attribute":true},"type":"string","readOnly":true,"description":"Expand options that include additional issue details in the response."},"fields":{"type":"object","properties":{"created":{"type":["string","null"],"format":"date-time","description":"This field is not shown in schema / swagger, but exists in records and we use it as cursor fiekd. Updated may be absent. Added to solve the #4341"},"updated":{"type":["string","null"],"format":"date-time","description":"This field is not shown in schema / swagger, but exists in records and we use it as cursor fiekd. Updated may be absent. Added to solve the #4341"}},"additionalProperties":{}},"schema":{"type":"object","readOnly":true,"description":"The schema describing each field present on the issue."},"editmeta":{"readOnly":true,"description":"The metadata for the fields on the issue that can be amended."},"changelog":{"readOnly":true,"description":"Details of changelogs associated with the issue."},"operations":{"readOnly":true,"description":"The operations that can be performed on the issue."},"properties":{"type":"object","readOnly":true,"description":"Details of the issue properties identified in the request.","additionalProperties":{"readOnly":true}},"transitions":{"type":"array","readOnly":true,"description":"The transitions that can be performed on the issue."},"renderedFields":{"type":"object","readOnly":true,"description":"The rendered value of each field present on the issue.","additionalProperties":{"readOnly":true}},"fieldsToInclude":{"type":"object"},"versionedRepresentations":{"type":"object","readOnly":true,"description":"The versions of each field on the issue.","additionalProperties":{"type":"object","readOnly":true,"additionalProperties":{"readOnly":true}}}},"additionalProperties":false},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[updated],sourceDefinedPrimaryKey=[[id]],namespace=,additionalProperties={}],syncMode=incremental,cursorField=[updated],destinationSyncMode=append_dedup,primaryKey=[[id]],additionalProperties={}]],additionalProperties={}]] 2021-09-13 07:19:36 INFO () TemporalAttemptExecution(get):114 - Executing worker wrapper. Airbyte version: 0.29.17-alpha 2021-09-13 07:19:36 WARN () Databases(createPostgresDatabaseWithRetry):57 - Waiting for database to become available... 2021-09-13 07:19:36 INFO () JobsDatabaseInstance(lambda$static$2):45 - Testing if jobs database is ready... 2021-09-13 07:19:36 INFO () Databases(createPostgresDatabaseWithRetry):74 - Database available! 2021-09-13 07:19:36 INFO () DefaultNormalizationWorker(run):63 - Running normalization. 2021-09-13 07:19:37 INFO () LineGobbler(voidCall):85 - Checking if airbyte/normalization:0.1.44 exists... 2021-09-13 07:19:37 INFO () LineGobbler(voidCall):85 - airbyte/normalization:0.1.44 was found locally. 2021-09-13 07:19:37 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/1/0/normalize --network host --log-driver none airbyte/normalization:0.1.44 run --integration-type postgres --config destination_config.json --catalog destination_catalog.json 2021-09-13 07:19:37 INFO () LineGobbler(voidCall):85 - Running: transform-config --config destination_config.json --integration-type postgres --out /data/1/0/normalize 2021-09-13 07:19:38 INFO () LineGobbler(voidCall):85 - Namespace(config='destination_config.json', integration_type=, out='/data/1/0/normalize') 2021-09-13 07:19:38 INFO () LineGobbler(voidCall):85 - transform_postgres 2021-09-13 07:19:38 INFO () LineGobbler(voidCall):85 - Running: transform-catalog --integration-type postgres --profile-config-dir /data/1/0/normalize --catalog destination_catalog.json --out /data/1/0/normalize/models/generated/ --json-column _airbyte_data 2021-09-13 07:19:38 INFO () LineGobbler(voidCall):85 - Processing destination_catalog.json... 2021-09-13 07:19:38 INFO () LineGobbler(voidCall):85 - Generating airbyte_ctes/public/issues_ab1.sql from issues 2021-09-13 07:19:38 INFO () LineGobbler(voidCall):85 - WARN: Unknown type for column editmeta at issues 2021-09-13 07:19:38 INFO () LineGobbler(voidCall):85 - WARN: Unknown type for column changelog at issues 2021-09-13 07:19:38 INFO () LineGobbler(voidCall):85 - WARN: Unknown type for column operations at issues 2021-09-13 07:19:38 INFO () LineGobbler(voidCall):85 - Generating airbyte_ctes/public/issues_ab2.sql from issues 2021-09-13 07:19:38 INFO () LineGobbler(voidCall):85 - Generating airbyte_ctes/public/issues_ab3.sql from issues 2021-09-13 07:19:38 INFO () LineGobbler(voidCall):85 - Generating airbyte_ctes/public/issues_ab4.sql from issues 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - Traceback (most recent call last): 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - File "/usr/local/bin/transform-catalog", line 8, in 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - sys.exit(main()) 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - File "/usr/local/lib/python3.8/site-packages/normalization/transform_catalog/transform.py", line 102, in main 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - TransformCatalog().run(args) 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - File "/usr/local/lib/python3.8/site-packages/normalization/transform_catalog/transform.py", line 55, in run 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - self.process_catalog() 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - File "/usr/local/lib/python3.8/site-packages/normalization/transform_catalog/transform.py", line 82, in process_catalog 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - processor.process(catalog_file=catalog_file, json_column_name=json_col, default_schema=schema) 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - File "/usr/local/lib/python3.8/site-packages/normalization/transform_catalog/catalog_processor.py", line 94, in process 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - nested_processors = stream_processor.process() 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - File "/usr/local/lib/python3.8/site-packages/normalization/transform_catalog/stream_processor.py", line 244, in process 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - self.generate_scd_type_2_model(from_table, column_names) + where_clause, 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - File "/usr/local/lib/python3.8/site-packages/normalization/transform_catalog/stream_processor.py", line 587, in generate_scd_type_2_model 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - cursor_field=self.get_cursor_field(column_names), 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - File "/usr/local/lib/python3.8/site-packages/normalization/transform_catalog/stream_processor.py", line 602, in get_cursor_field 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - cursor = column_names[self.cursor_field[0]][0] 2021-09-13 07:19:38 ERROR () LineGobbler(voidCall):85 - KeyError: 'updated' 2021-09-13 07:19:38 INFO () TemporalAttemptExecution(lambda$getWorkerThread$1):168 - Completing future exceptionally... io.airbyte.workers.WorkerException: Normalization Failed. at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:77) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:38) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$1(TemporalAttemptExecution.java:165) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at java.lang.Thread.run(Thread.java:832) [?:?] Caused by: io.airbyte.workers.WorkerException: Normalization Failed. at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:74) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] ... 3 more Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:151) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:62) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:38) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$1(TemporalAttemptExecution.java:165) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at java.lang.Thread.run(Thread.java:832) [?:?] 2021-09-13 07:19:38 INFO () TemporalAttemptExecution(get):135 - Stopping cancellation check scheduling... 2021-09-13 07:19:38 WARN () POJOActivityTaskHandler$POJOActivityImplementation(execute):243 - Activity failure. ActivityId=0e222a55-e205-3761-bfbf-4f945cefdfeb, activityType=Normalize, attempt=1 java.util.concurrent.ExecutionException: io.airbyte.workers.WorkerException: Normalization Failed. at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) ~[?:?] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2063) ~[?:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:133) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.temporal.SyncWorkflow$NormalizationActivityImpl.normalize(SyncWorkflow.java:291) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:564) ~[?:?] at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:277) ~[temporal-sdk-1.0.4.jar:?] at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:216) ~[temporal-sdk-1.0.4.jar:?] at io.temporal.internal.sync.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:181) ~[temporal-sdk-1.0.4.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:192) ~[temporal-sdk-1.0.4.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:154) ~[temporal-sdk-1.0.4.jar:?] at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:73) ~[temporal-sdk-1.0.4.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?] at java.lang.Thread.run(Thread.java:832) [?:?] Caused by: io.airbyte.workers.WorkerException: Normalization Failed. at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:77) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:38) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$1(TemporalAttemptExecution.java:165) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] ... 1 more Caused by: io.airbyte.workers.WorkerException: Normalization Failed. at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:74) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:38) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$1(TemporalAttemptExecution.java:165) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] ... 1 more Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:151) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:62) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:38) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$1(TemporalAttemptExecution.java:165) ~[io.airbyte-airbyte-workers-0.29.17-alpha.jar:?] at java.lang.Thread.run(Thread.java:832) [?:?]