Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka Destination connector Issue in UI #5789

Closed
nkjotiyana opened this issue Sep 1, 2021 · 0 comments · Fixed by #6040
Closed

Kafka Destination connector Issue in UI #5789

nkjotiyana opened this issue Sep 1, 2021 · 0 comments · Fixed by #6040

Comments

@nkjotiyana
Copy link
Contributor

Enviroment

  • Airbyte version: 0.29.8-alpha
  • OS Version / Instance: macOS
  • Deployment: Docker
  • Source Connector and version: NA
  • Destination Connector and version: 2.8.0

Screenshot 2021-08-27 at 5 50 41 PM

  • Severity: Medium
  • Step where error happened: while creating Kafka Destination in UI

Current Behavior

When Kafka destination is getting created from UI, It giving error like below

Verification error(s) occurred for CHECK. Errors: [$.buffer_memory: is missing but it is required, $.max_block_ms: is missing but it is required, $.linger_ms: is missing but it is required, $.socket_connection_setup_timeout_max_ms: is missing but it is required, $.socket_connection_setup_timeout_ms: is missing but it is required]

when I update fields value getting below error :

Verification error(s) occurred for CHECK. Errors: [$.buffer_memory: string found, number expected, $.max_block_ms: string found, number expected, $.socket_connection_setup_timeout_max_ms:

Its not able to create kafka destination due to above error.
It seems like a bug, as fields value are coming as string instead of integer and thats why its giving error.

Expected Behavior

It should accept default value of fields
It should send fields value as integer not as string.
Kafka destination should be created properly.

Logs

If applicable, please upload the logs from the failing operation.
For sync jobs, you can download the full logs from the UI by going to the sync attempt page and
clicking the download logs button at the top right of the logs display window.

LOG

2021-09-01 16:09:53 INFO () TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.29.8-alpha
2021-09-01 16:09:53 INFO () LineGobbler(voidCall):85 - Checking if airbyte/destination-kafka:0.1.1 exists...
2021-09-01 16:09:53 INFO () LineGobbler(voidCall):85 - airbyte/destination-kafka:0.1.1 was found locally.
2021-09-01 16:09:53 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/5c735263-7ff1-4aa9-8f2d-819056e01958/0 --network host --log-driver none airbyte/destination-kafka:0.1.1 check --config source_config.json
2021-09-01 16:09:55 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-01 16:09:55 INFO i.a.i.d.k.KafkaDestination(main):104 - {} - Starting destination: class io.airbyte.integrations.destination.kafka.KafkaDestination
2021-09-01 16:09:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-01 16:09:56 INFO i.a.i.b.IntegrationRunner(run):96 - {} - Running integration: io.airbyte.integrations.destination.kafka.KafkaDestination
2021-09-01 16:09:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-01 16:09:56 INFO i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {check=null, config=source_config.json}
2021-09-01 16:09:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-01 16:09:56 INFO i.a.i.b.IntegrationRunner(run):100 - {} - Command: CHECK
2021-09-01 16:09:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-01 16:09:56 INFO i.a.i.b.IntegrationRunner(run):101 - {} - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'}
2021-09-01 16:09:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-01 16:09:56 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-09-01 16:09:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-01 16:09:56 WARN c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-09-01 16:09:56 INFO () TemporalAttemptExecution(get):133 - Stopping cancellation check scheduling...

Steps to Reproduce

  1. Deploy airbyte on Mac machine on docker
  2. create Kafka destination from UI.
  3. Pu all required parameters , error will occur.

Are you willing to submit a PR?

NO

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants