Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Flaky tests high-level tracking #3481

Open
3 of 14 tasks
graytaylor0 opened this issue Oct 11, 2023 · 16 comments · Fixed by #3471
Open
3 of 14 tasks

[BUG] Flaky tests high-level tracking #3481

graytaylor0 opened this issue Oct 11, 2023 · 16 comments · Fixed by #3471
Labels
bug Something isn't working maintenance Issues to help maintain the project, such as improving builds, testing, etc.

Comments

@graytaylor0
Copy link
Member

graytaylor0 commented Oct 11, 2023 •
edited by oeyh
Loading

Describe the bug
Running the data-prepper build and github actions frequently has failing flaky tests, and often requires retrying the tests multiple times before it succeeds. This issue is meant to allow easy tracking of these flaky tests, by adding an item to the checklist when a flaky test is seen.

Flaky tests
@graytaylor0 graytaylor0 added bug Something isn't working untriaged labels Oct 11, 2023
@graytaylor0 graytaylor0 added maintenance Issues to help maintain the project, such as improving builds, testing, etc. and removed untriaged labels Oct 11, 2023
@graytaylor0 graytaylor0 mentioned this issue Oct 11, 2023
Merged
3 tasks
@dlvenable dlvenable mentioned this issue Nov 4, 2023
Closed
2 tasks
@dlvenable
Copy link
Member 

> Task :data-prepper-core:integrationTest

Router_ThreeRoutesIT > sending_alpha_and_beta_events_sends_to_sinks_that_take_both(String) > [3] ConditionalRoutingIT_alpha_beta_gamma FAILED
    java.lang.AssertionError at Router_ThreeRoutesIT.java:89

Router_ThreeRoutesIT > sending_alpha_and_beta_events_sends_to_both_sinks() FAILED
    java.lang.AssertionError at Router_ThreeRoutesIT.java:112

@dlvenable
Copy link
Member 

AwsCloudMapPeerListProviderTest > WithSeveralFailedAttempts > listener_gets_list_after_several_failed_attempts() FAILED
    java.lang.AssertionError at AwsCloudMapPeerListProviderTest.java:345

@dlvenable
Copy link
Member 

PipelinesWithAcksIT > two_pipelines_with_multiple_records() FAILED
    java.lang.NullPointerException at PipelinesWithAcksIT.java:105

@dlvenable
Copy link
Member 

OTelTraceRawProcessorTest > traceGroupCacheMaxSize_provides_an_upper_bound(long, int) > [1] 1, 4 FAILED
    java.lang.AssertionError at OTelTraceRawProcessorTest.java:270

@dlvenable dlvenable mentioned this issue Nov 10, 2023
Merged
4 tasks
@dlvenable dlvenable changed the title [BUG] High Level Tracking of Flaky Data Prepper Tests [BUG] Flaky tests high-level tracking Nov 10, 2023
@dlvenable dlvenable mentioned this issue Nov 10, 2023
Merged
4 tasks
@dlvenable
Copy link
Member 

OTelTraceRawProcessorTest > testExportRequestFlushByMixedMultiThread() FAILED
    org.opentest4j.AssertionFailedError at OTelTraceRawProcessorTest.java:193

@dlvenable
Copy link
Member

I saw this error again in a recent build:

Router_ThreeRoutesIT > sending_alpha_and_beta_events_sends_to_sinks_that_take_both(String) > [1] ConditionalRoutingIT_all FAILED
    java.lang.AssertionError at Router_ThreeRoutesIT.java:89

Router_ThreeRoutesIT > sending_alpha_and_beta_events_sends_to_sinks_that_take_both(String) > [3] ConditionalRoutingIT_alpha_beta_gamma FAILED
    java.lang.AssertionError at Router_ThreeRoutesIT.java:89

@dlvenable
Copy link
Member

I'm seeing lots of failures in the kafka-plugins project in the Java 21 build.


> Task :data-prepper-plugins:kafka-plugins:test

KafkaBufferTest > test_kafkaBuffer_producerThrows() FAILED
    java.lang.IllegalAccessError at KafkaBufferTest.java:155

KafkaBufferTest > test_kafkaBuffer_isEmpty_BufferNotEmpty() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KafkaBufferTest > testShutdown_Successful() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KafkaBufferTest > test_kafkaBuffer_isEmpty_TopicNotEmpty() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KafkaBufferTest > test_kafkaBuffer_getDrainTimeout() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KafkaBufferTest > test_kafkaBuffer_doCheckpoint() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KafkaBufferTest > isWrittenOffHeapOnly_returns_true() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KafkaBufferTest > test_kafkaBuffer_isEmpty_True() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KafkaBufferTest > test_kafkaBuffer_basicFunctionality() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KafkaBufferTest > test_kafkaBuffer_postProcess() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KafkaBufferTest > test_kafkaBuffer_doWriteAll() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KafkaBufferTest > testShutdown_Timeout() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KafkaBufferTest > testShutdown_InterruptedException() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KafkaBufferTest > test_kafkaBuffer_isEmpty_MultipleTopics_AllNotEmpty() FAILED
    java.lang.NullPointerException at KafkaBufferTest.java:155

KeyFactoryTest > getKeySupplier_returns_using_first_InnerKeyFactory_that_supports_the_TopicConfig() FAILED
    java.lang.NullPointerException at KeyFactoryTest.java:54



> Task :data-prepper-plugins:kafka-plugins:test

EncryptionContextTest > fromEncryptionKey_includes_correct_Key() FAILED
    java.lang.AssertionError at EncryptionContextTest.java:55

EncryptionContextTest > encryption_and_decryption_are_symmetric() FAILED
    java.lang.NullPointerException at EncryptionContextTest.java:36

MessageFormatSerializationFactoryTest > getSerializer_returns_expected_deserializer(MessageFormat, Class) > [1] BYTES, class org.apache.kafka.common.serialization.ByteArraySerializer FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:72

MessageFormatSerializationFactoryTest > getSerializer_returns_expected_deserializer(MessageFormat, Class) > [2] PLAINTEXT, class org.apache.kafka.common.serialization.StringSerializer FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:72

MessageFormatSerializationFactoryTest > getSerializer_returns_expected_deserializer(MessageFormat, Class) > [3] JSON, class org.apache.kafka.connect.json.JsonSerializer FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:72

MessageFormatSerializationFactoryTest > getSerializer_returns_expected_deserializer(MessageFormat, Class) > [4] AVRO, class io.confluent.kafka.serializers.KafkaAvroSerializer FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:72

MessageFormatSerializationFactoryTest > getDeserializer_returns_same_instance_for_multiple_calls(MessageFormat) > [1] PLAINTEXT FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:59

MessageFormatSerializationFactoryTest > getDeserializer_returns_same_instance_for_multiple_calls(MessageFormat) > [2] JSON FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:59

MessageFormatSerializationFactoryTest > getDeserializer_returns_same_instance_for_multiple_calls(MessageFormat) > [3] AVRO FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:59

MessageFormatSerializationFactoryTest > getDeserializer_returns_same_instance_for_multiple_calls(MessageFormat) > [4] BYTES FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:59

MessageFormatSerializationFactoryTest > getSerializer_returns_same_instance_for_multiple_calls(MessageFormat) > [1] PLAINTEXT FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:87

MessageFormatSerializationFactoryTest > getSerializer_returns_same_instance_for_multiple_calls(MessageFormat) > [2] JSON FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:87

MessageFormatSerializationFactoryTest > getSerializer_returns_same_instance_for_multiple_calls(MessageFormat) > [3] AVRO FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:87

MessageFormatSerializationFactoryTest > getSerializer_returns_same_instance_for_multiple_calls(MessageFormat) > [4] BYTES FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:87

MessageFormatSerializationFactoryTest > getDeserializer_returns_deserializer_instance(MessageFormat) > [1] PLAINTEXT FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:51

MessageFormatSerializationFactoryTest > getDeserializer_returns_deserializer_instance(MessageFormat) > [2] JSON FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:51

MessageFormatSerializationFactoryTest > getDeserializer_returns_deserializer_instance(MessageFormat) > [3] AVRO FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:51

MessageFormatSerializationFactoryTest > getDeserializer_returns_deserializer_instance(MessageFormat) > [4] BYTES FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:51

MessageFormatSerializationFactoryTest > getDeserializer_returns_expected_deserializer(MessageFormat, Class) > [1] BYTES, class org.apache.kafka.common.serialization.ByteArrayDeserializer FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:44

MessageFormatSerializationFactoryTest > getDeserializer_returns_expected_deserializer(MessageFormat, Class) > [2] PLAINTEXT, class org.apache.kafka.common.serialization.StringDeserializer FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:44

MessageFormatSerializationFactoryTest > getDeserializer_returns_expected_deserializer(MessageFormat, Class) > [3] JSON, class org.apache.kafka.connect.json.JsonDeserializer FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:44

MessageFormatSerializationFactoryTest > getDeserializer_returns_expected_deserializer(MessageFormat, Class) > [4] AVRO, class io.confluent.kafka.serializers.KafkaAvroDeserializer FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:44

MessageFormatSerializationFactoryTest > getSerializer_returns_deserializer_instance(MessageFormat) > [1] PLAINTEXT FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:79

MessageFormatSerializationFactoryTest > getSerializer_returns_deserializer_instance(MessageFormat) > [2] JSON FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:79

MessageFormatSerializationFactoryTest > getSerializer_returns_deserializer_instance(MessageFormat) > [3] AVRO FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:79

MessageFormatSerializationFactoryTest > getSerializer_returns_deserializer_instance(MessageFormat) > [4] BYTES FAILED
    java.lang.NullPointerException at MessageFormatSerializationFactoryTest.java:79

AuthConfigTest > testSaslAuthConfigWithPlainText() FAILED
    java.lang.ClassCastException at AuthConfigTest.java:56

AuthConfigTest > testSaslAuthConfigWithMskIam() FAILED
    java.lang.ClassCastException at AuthConfigTest.java:56

AuthConfigTest > testSaslAuthConfigWithOAuth() FAILED
    java.lang.ClassCastException at AuthConfigTest.java:56

AuthConfigTest > testPlainTextAuthConfig() FAILED
    java.lang.ClassCastException at AuthConfigTest.java:56

CommonTopicConfigTest > testConfigValues_from_yaml() FAILED
    java.lang.ClassCastException at CommonTopicConfigTest.java:41

CommonTopicConfigTest > test_topicsConfig_not_null() FAILED
    java.lang.ClassCastException at CommonTopicConfigTest.java:41

CommonTopicConfigTest > testConfigValues_default() FAILED
    java.lang.ClassCastException at CommonTopicConfigTest.java:41

CommonTopicConfigTest > testConfigValues_from_yaml_not_null() FAILED
    java.lang.ClassCastException at CommonTopicConfigTest.java:41

KmsConfigTest > toCredentialsOptions_returns_options_with_correct_values() FAILED
    java.lang.IllegalArgumentException at KmsConfigTest.java:33
        Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException at KmsConfigTest.java:33

MskBrokerConnectionTypeTest > fromTypeValue(MskBrokerConnectionType) > [1] null FAILED
    java.lang.NullPointerException at MskBrokerConnectionTypeTest.java:19

MskBrokerConnectionTypeTest > fromTypeValue(MskBrokerConnectionType) > [2] null FAILED
    java.lang.NullPointerException at MskBrokerConnectionTypeTest.java:19

MskBrokerConnectionTypeTest > fromTypeValue(MskBrokerConnectionType) > [3] null FAILED
    java.lang.NullPointerException at MskBrokerConnectionTypeTest.java:19

OAuthConfigTest > assertNotNullForConfigs() FAILED
    java.lang.ClassCastException at OAuthConfigTest.java:49

OAuthConfigTest > assertConfigValues() FAILED
    java.lang.ClassCastException at OAuthConfigTest.java:49

OAuthConfigTest > testConfig() FAILED
    java.lang.ClassCastException at OAuthConfigTest.java:49

PlainTextAuthConfigTest > testConfigValues() FAILED
    java.lang.ClassCastException at PlainTextAuthConfigTest.java:35

SchemaConfigTest > test_schemaConfig_not_null() FAILED
    java.lang.ClassCastException at SchemaConfigTest.java:37

SchemaConfigTest > test_schemaConfig_values() FAILED
    java.lang.ClassCastException at SchemaConfigTest.java:37

KafkaCustomConsumerTest > testJsonDeserializationErrorWithAcknowledgements() FAILED
    java.lang.IllegalAccessError at KafkaCustomConsumerTest.java:162

KafkaCustomConsumerTest > testAwsGlueErrorWithAcknowledgements() FAILED
    java.lang.IllegalAccessError at KafkaCustomConsumerTest.java:162

KafkaCustomConsumerTest > testPlainTextConsumeRecordsWithAcknowledgements() FAILED
    java.lang.IllegalAccessError at KafkaCustomConsumerTest.java:162

KafkaCustomConsumerTest > testJsonConsumeRecords() FAILED
    java.lang.IllegalAccessError at KafkaCustomConsumerTest.java:162

KafkaCustomConsumerTest > testPlainTextConsumeRecordsWithNegativeAcknowledgements() FAILED
    java.lang.IllegalAccessError at KafkaCustomConsumerTest.java:162

KafkaCustomConsumerTest > testPlainTextConsumeRecords() FAILED
    java.lang.IllegalAccessError at KafkaCustomConsumerTest.java:162

TopicPartitionCommitTrackerTest > test(List) > [1] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:51

TopicPartitionCommitTrackerTest > test(List) > [2] [9, 8, 7, 6, 5, 4, 3, 2, 1, 0] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:51

TopicPartitionCommitTrackerTest > test(List) > [3] [1, 0, 2, 3, 5, 9, 4, 8, 6, 7] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:51

TopicPartitionCommitTrackerTest > test(List) > [4] [5, 7, 4, 0, 2, 1, 3, 9, 8, 6] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:51

TopicPartitionCommitTrackerTest > test(List) > [5] [1, 6, 2, 5, 7, 4, 0, 9, 8, 3] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:51

TopicPartitionCommitTrackerTest > test(List) > [6] [1, 2, 6, 5, 3, 9, 4, 7, 0, 8] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:51

TopicPartitionCommitTrackerTest > test(List) > [7] [8, 4, 0, 6, 3, 1, 7, 2, 5, 9] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:51

TopicPartitionCommitTrackerTest > test(List) > [8] [0, 2, 5, 7, 1, 8, 6, 3, 4, 9] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:51

TopicPartitionCommitTrackerTest > test(List) > [9] [3, 8, 2, 6, 7, 1, 5, 0, 9, 4] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:51

TopicPartitionCommitTrackerTest > test(List) > [10] [4, 9, 2, 3, 7, 8, 0, 1, 5, 6] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:51

TopicPartitionCommitTrackerTest > test(List) > [11] [0, 6, 8, 1, 5, 9, 4, 7, 2, 3] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:51

TopicPartitionCommitTrackerTest > test(List) > [12] [9, 7, 0, 1, 2, 8, 5, 6, 3, 4] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:51

TopicPartitionCommitTrackerTest > testNonZeroOffsets(List) > [1] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:75

TopicPartitionCommitTrackerTest > testNonZeroOffsets(List) > [2] [9, 8, 7, 6, 5, 4, 3, 2, 1, 0] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:75

TopicPartitionCommitTrackerTest > testNonZeroOffsets(List) > [3] [1, 4, 7, 5, 0, 6, 8, 2, 9, 3] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:75

TopicPartitionCommitTrackerTest > testNonZeroOffsets(List) > [4] [9, 6, 3, 4, 2, 8, 0, 5, 1, 7] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:75

TopicPartitionCommitTrackerTest > testNonZeroOffsets(List) > [5] [1, 6, 2, 0, 8, 5, 7, 4, 9, 3] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:75

TopicPartitionCommitTrackerTest > testNonZeroOffsets(List) > [6] [3, 2, 4, 0, 5, 1, 7, 8, 9, 6] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:75

TopicPartitionCommitTrackerTest > testNonZeroOffsets(List) > [7] [8, 2, 6, 4, 3, 5, 7, 1, 9, 0] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:75

TopicPartitionCommitTrackerTest > testNonZeroOffsets(List) > [8] [8, 6, 9, 7, 1, 0, 4, 2, 5, 3] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:75

TopicPartitionCommitTrackerTest > testNonZeroOffsets(List) > [9] [3, 1, 8, 2, 6, 5, 4, 9, 7, 0] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:75

TopicPartitionCommitTrackerTest > testNonZeroOffsets(List) > [10] [6, 1, 2, 8, 9, 5, 4, 0, 7, 3] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:75

TopicPartitionCommitTrackerTest > testNonZeroOffsets(List) > [11] [6, 9, 1, 5, 7, 4, 3, 2, 8, 0] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:75

TopicPartitionCommitTrackerTest > testNonZeroOffsets(List) > [12] [9, 8, 0, 6, 7, 4, 5, 2, 3, 1] FAILED
    java.lang.AssertionError at TopicPartitionCommitTrackerTest.java:75

KafkaClusterConfigTest > testConfigWithTestExtension() FAILED
    java.lang.annotation.AnnotationFormatError at KafkaClusterConfigTest.java:40

KafkaCustomProducerTest > testGetGenericRecord() FAILED
    java.lang.ExceptionInInitializerError at KafkaCustomProducerTest.java:249
        Caused by: java.lang.NullPointerException at KafkaCustomProducerTest.java:249

KafkaCustomProducerTest > producePlainTextRecords() FAILED
    java.lang.NullPointerException at KafkaCustomProducer.java:112

KafkaCustomProducerTest > producePlainTextRecords_callbackException() FAILED
    java.lang.NullPointerException at KafkaCustomProducer.java:112

KafkaCustomProducerTest > validateSchema() FAILED
    com.github.fge.jsonschema.core.exceptions.ProcessingException at KafkaCustomProducerTest.java:261

KafkaCustomProducerTest > produceRawDataTest() FAILED
    java.lang.NullPointerException at KafkaCustomProducerTest.java:105

KafkaCustomProducerTest > producePlainTextRecords_sendError() FAILED
    java.lang.NullPointerException at KafkaCustomProducer.java:112

KafkaCustomProducerTest > produceJsonRecordsTest() FAILED
    java.lang.NullPointerException at KafkaCustomProducer.java:112

KafkaCustomProducerTest > produceAvroRecordsTest() FAILED
    java.lang.NoClassDefFoundError at KafkaCustomProducerTest.java:227

KafkaCustomProducerTest > produceRawData_sendError() FAILED
    org.mockito.exceptions.misusing.UnfinishedStubbingException at KafkaCustomProducerTest.java:87

DLQSinkTest > testPerform() FAILED
    java.lang.ClassCastException at DLQSinkTest.java:68

KafkaSinkConfigTest > testDLQConfiguration() FAILED
    java.util.NoSuchElementException at KafkaSinkConfigTest.java:97

KafkaSinkTest > doOutputExceptionTest() FAILED
    java.lang.ClassCastException at KafkaSinkTest.java:105

KafkaSinkTest > doOutputTest() FAILED
    java.lang.ClassCastException at KafkaSinkTest.java:105

KafkaSinkTest > doInitializeNullPointerExceptionTest() FAILED
    java.lang.ClassCastException at KafkaSinkTest.java:105

KafkaSinkTest > isReadyTest() FAILED
    java.lang.ClassCastException at KafkaSinkTest.java:105

KafkaSinkTest > shutdownTest() FAILED
    java.lang.ClassCastException at KafkaSinkTest.java:105

KafkaSinkTest > shutdownExceptionTest() FAILED
    java.lang.ClassCastException at KafkaSinkTest.java:105

KafkaSinkTest > doOutputTestForAutoTopicCreate() FAILED
    java.lang.ClassCastException at KafkaSinkTest.java:105

KafkaSinkTest > doInitializeTest() FAILED
    java.lang.ClassCastException at KafkaSinkTest.java:105

KafkaSinkTest > doOutputEmptyRecordsTest() FAILED
    java.lang.ClassCastException at KafkaSinkTest.java:105

KafkaSourceConfigTest > test_topics_not_null() FAILED
    java.lang.ClassCastException at KafkaSourceConfigTest.java:47

KafkaSourceConfigTest > test_kafka_config_not_null() FAILED
    java.lang.ClassCastException at KafkaSourceConfigTest.java:47

KafkaSourceConfigTest > test_setters() FAILED
    java.lang.ClassCastException at KafkaSourceConfigTest.java:47

KafkaSourceConfigTest > test_bootStrapServers_not_null() FAILED
    java.lang.ClassCastException at KafkaSourceConfigTest.java:47

KafkaSourceTest > test_kafkaSource_basicFunctionality() FAILED
    java.lang.RuntimeException at KafkaSourceTest.java:150
        Caused by: java.lang.NullPointerException at KafkaSourceTest.java:150

KafkaSourceTest > test_kafkaSource_retry_consumer_create() FAILED
    java.lang.RuntimeException at KafkaSource.java:155
        Caused by: java.lang.NullPointerException at KafkaSourceTest.java:165

AuthenticationPropertyConfigurerTest > testSetSaslPlainTextProperties() FAILED
    java.lang.ClassCastException at AuthenticationPropertyConfigurerTest.java:34

AuthenticationPropertyConfigurerTest > testSetSaslSslProperties() FAILED
    java.lang.ClassCastException at AuthenticationPropertyConfigurerTest.java:34

AuthenticationPropertyConfigurerTest > testSetSaslOauthProperties() FAILED
    java.lang.ClassCastException at AuthenticationPropertyConfigurerTest.java:34

AuthenticationTypeTest > getAuthTypeByName(AuthenticationType) > [1] plaintext FAILED
    java.lang.NullPointerException at AuthenticationTypeTest.java:17

AuthenticationTypeTest > getAuthTypeByName(AuthenticationType) > [2] ssl FAILED
    java.lang.NullPointerException at AuthenticationTypeTest.java:17

AuthenticationTypeTest > getAuthTypeByName(AuthenticationType) > [3] oauth FAILED
    java.lang.NullPointerException at AuthenticationTypeTest.java:17

KafkaTopicConsumerMetricsTests > KafkaTopicMetricTest_checkMetricUpdates(int) > [1] 1 FAILED
    java.lang.NullPointerException at KafkaTopicConsumerMetricsTests.java:211

KafkaTopicConsumerMetricsTests > KafkaTopicMetricTest_checkMetricUpdates(int) > [2] 5 FAILED
    java.lang.NullPointerException at KafkaTopicConsumerMetricsTests.java:211

KafkaTopicConsumerMetricsTests > KafkaTopicMetricTest_checkMetricUpdates(int) > [3] 10 FAILED
    java.lang.NullPointerException at KafkaTopicConsumerMetricsTests.java:211

KafkaTopicProducerMetricsTests > KafkaTopicMetricTest_checkMetricUpdates(int) > [1] 1 FAILED
    java.lang.NullPointerException at KafkaTopicProducerMetricsTests.java:169

KafkaTopicProducerMetricsTests > KafkaTopicMetricTest_checkMetricUpdates(int) > [2] 5 FAILED
    java.lang.NullPointerException at KafkaTopicProducerMetricsTests.java:169

KafkaTopicProducerMetricsTests > KafkaTopicMetricTest_checkMetricUpdates(int) > [3] 10 FAILED
    java.lang.NullPointerException at KafkaTopicProducerMetricsTests.java:169

SinkPropertyConfigurerTest > testGetProducerPropertiesForPlainText() FAILED
    java.lang.ClassCastException at SinkPropertyConfigurerTest.java:43

SinkPropertyConfigurerTest > testGetProducerPropertiesForNoSerde() FAILED
    java.lang.ClassCastException at SinkPropertyConfigurerTest.java:43

SinkPropertyConfigurerTest > testGetProducerPropertiesForAvro() FAILED
    java.lang.ClassCastException at SinkPropertyConfigurerTest.java:43

SinkPropertyConfigurerTest > testGetProducerPropertiesForJson() FAILED
    java.lang.ClassCastException at SinkPropertyConfigurerTest.java:43

258 tests completed, 132 failed

https://github.com/opensearch-project/data-prepper/actions/runs/7479248734/job/20362415558?pr=3937

@dlvenable
Copy link
Member

Seen today:

ExportPartitionWorkerTest > testProcessPartitionSuccess(String) > [1] test.collection|0|1|java.lang.Integer FAILED
    org.mockito.exceptions.verification.WantedButNotInvoked at ExportPartitionWorkerTest.java:120

@dlvenable dlvenable mentioned this issue Mar 19, 2024
Closed
@dlvenable
Copy link
Member

Seen today:

HistogramAggregateActionTests > testHistogramAggregateOTelFormatWithStartAndEndTimesInTheEvent(int) > [1] 10 FAILED
    java.lang.AssertionError at HistogramAggregateActionTests.java:370

@dlvenable
Copy link
Member

Some investigation on AggregateProcessorIT.

[2] 15.0

java.lang.AssertionError: 
Expected: a numeric value within <1.0> of <3000.0>
     but: <2994.0> differed by <5.0> more than delta <1.0>
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
	at org.opensearch.dataprepper.plugins.processor.aggregate.AggregateProcessorIT.aggregateWithPercentSamplerAction(AggregateProcessorIT.java:292)

This assertion is failing:

assertThat((double)allowedEventsCount.get(), closeTo(NUM_THREADS * NUM_EVENTS_PER_BATCH * testPercent/100, 1.0));

@dlvenable
Copy link
Member

A new failure:

LambdaSinkServiceTest > lambda_sink_test_batch_enabled() FAILED
    org.mockito.exceptions.verification.WantedButNotInvoked at LambdaSinkServiceTest.java:316

@dlvenable dlvenable mentioned this issue Jul 10, 2024
Merged
4 tasks
@dlvenable
Copy link
Member

New failure: > Task :data-prepper-plugins:log-generator-source:test

LogGeneratorSourceTest > GIVEN_logGeneratorSourceAndBlockingBuffer_WHEN_reachedLimit_THEN_stopsWritingToBuffer() FAILED
org.mockito.exceptions.verification.WantedButNotInvoked at LogGeneratorSourceTest.java:115

This was referenced Jul 16, 2024
Merged
Merged
@dlvenable
Copy link
Member

Another AggregateProcessorIT:

AggregateProcessorIT > aggregateWithCountAggregateActionWithCondition() > repetition 2 of 2 FAILED
    java.lang.AssertionError at AggregateProcessorIT.java:439

@dlvenable
Copy link
Member

dlvenable commented Oct 4, 2024 •
edited
Loading

RDS now has a flaky test:

> Task :data-prepper-plugins:rds-source:test

ExportSchedulerTest > test_shutDown() FAILED
    org.mockito.exceptions.verification.NoInteractionsWanted at ExportSchedulerTest.java:271

https://github.com/opensearch-project/data-prepper/actions/runs/11172396206/job/31058770072?pr=5021

@oeyh , Can you look at this one?

@dlvenable
Copy link
Member 

> Task :data-prepper-plugins:rds-source:test

StreamSchedulerTest > test_shutdown() FAILED
    org.mockito.exceptions.verification.NoInteractionsWanted at StreamSchedulerTest.java:118

@oeyh , Here is another failure in RDS.

@oeyh
Copy link
Collaborator

oeyh commented Oct 10, 2024 •
edited
Loading

Truncate processor test. https://github.com/opensearch-project/data-prepper/actions/runs/11278262054/job/31366340484?pr=5043

> Task :data-prepper-plugins:truncate-processor:test

TruncateProcessorConfigTests > testInvalidConfiguration_StartAt_Length_Negative() FAILED
    org.opentest4j.AssertionFailedError: expected: <false> but was: <true>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working maintenance Issues to help maintain the project, such as improving builds, testing, etc.
Projects
Data Prepper Tracking Board
Status: Unplanned
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix e2e acks test kkondaka/kk-data-prepper-f2
3 participants
@dlvenable @oeyh @graytaylor0