-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🎉 New Destination: Pulsar #7315
🎉 New Destination: Pulsar #7315
Conversation
Awesome @mmolimar I requested the team to review your contribution |
@mmolimar seems like you have an issue with acceptance tests hanging |
Let me check this and I'll get back to you. Thanks @VitaliiMaltsev ! |
...on-pulsar/src/main/java/io/airbyte/integrations/destination/pulsar/PulsarRecordConsumer.java
Outdated
Show resolved
Hide resolved
...on-pulsar/src/main/java/io/airbyte/integrations/destination/pulsar/PulsarRecordConsumer.java
Show resolved
Hide resolved
...gration/java/io/airbyte/integrations/destination/pulsar/PulsarDestinationAcceptanceTest.java
Outdated
Show resolved
Hide resolved
...gration/java/io/airbyte/integrations/destination/pulsar/PulsarDestinationAcceptanceTest.java
Outdated
Show resolved
Hide resolved
...pulsar/src/main/java/io/airbyte/integrations/destination/pulsar/PulsarDestinationConfig.java
Outdated
Show resolved
Hide resolved
...gration/java/io/airbyte/integrations/destination/pulsar/PulsarDestinationAcceptanceTest.java
Show resolved
Hide resolved
final PulsarRecordConsumer recordConsumer = new PulsarRecordConsumer(config, CATALOG, mock(Consumer.class), NAMING_RESOLVER); | ||
|
||
final Map<AirbyteStreamNameNamespacePair, Producer<JsonNode>> producerMap = recordConsumer.buildProducerMap(); | ||
assertEquals(1, producerMap.size()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please add also unit test to build ProducerMap with more then 1 stream
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
still needed unit test for producerMap.size() > 1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done!
...ulsar/src/test/java/io/airbyte/integrations/destination/pulsar/PulsarRecordConsumerTest.java
Show resolved
Hide resolved
Hey @VitaliiMaltsev I did a refactor and added some configurations. Let me know what you think. |
I think everything should be fine now @VitaliiMaltsev |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…yte into marcos/test-pr-7315
9dbe0ac
to
8e7071c
Compare
* Adding Pulsar destination * Add enable_chunking property * Add dependency for DNS resolve for Mac * Updating build.gradle * Adding definition * Refactor and fix tests * Adding new config properties * Reformat code * Close resources when checking connection * Updating doc * Adding more test to build the producer map * Rename brokers property * Avoid emitting null states to the output collector * fix config file * run seed source/destination Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
What
Destination for Apache Pulsar.
Related with this issue. Part of the hacktoberfest
Pre-merge Checklist
Expand the relevant checklist and delete the others.
New Connector
Community member or Airbyter
airbyte_secret
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.README.md
bootstrap.md
. See description and examplesdocs/SUMMARY.md
docs/integrations/<source or destination>/<name>.md
including changelog. See changelog exampledocs/integrations/README.md
airbyte-integrations/builds.md
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing./publish
command described hereUpdating a connector
Community member or Airbyter
airbyte_secret
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.README.md
bootstrap.md
. See description and examplesdocs/integrations/<source or destination>/<name>.md
including changelog. See changelog exampleAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing./publish
command described hereConnector Generator
-scaffold
in their name) have been updated with the latest scaffold by running./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates
then checking in your changes