[BUG] Flaky test in DataFileLoaderTest

[dlvenable]

Describe the bug

> Task :data-prepper-plugins:dynamodb-source:test

DataFileLoaderTest > test_run_loadFile_correctly() FAILED
    org.mockito.exceptions.verification.WantedButNotInvoked at DataFileLoaderTest.java:139

Example failure:

https://github.com/opensearch-project/data-prepper/actions/runs/6470850267/job/17567944366?pr=3469

To Reproduce

This is flaky, so this might require some investigation.

Expected behavior

This test should consistently pass.

[dlvenable]

@daixba , Could you take a look at this flaky test?

[daixba]

Sure, I will work on this.

I have seen this before, it's strange that it does not fail all the time and I can't reproduce this failure in my local. My best guess is that there is no enough 'time' before the 'expected invoke' to happen due in the github ci instance.

[dlvenable]

Recently failing test running on Java 11:

> Task :data-prepper-plugins:dynamodb-source:test

DataFileLoaderTest > run_loadFile_with_acknowledgments_processes_correctly() FAILED
    org.mockito.exceptions.verification.WantedButNotInvoked at DataFileLoaderTest.java:222

[daixba]

The Flaky issue (DataFileLoaderTest.java:222) regarding flush() wanted but not invoked is because there was

private final int total = random.nextInt(10);

in the test code to generate random number of test records.

However, I just noticed that the 0 is inclusive for nextInt() method, hence, there is a small chance that the total records is 0. In which case, the flush() was not invoked.

I have run the test again and again and confirm that only when total == 0, the test failed. I was trying to fix this issue by adding at least 1 to the total count. However, this test was already modified.

Let's see if this issue still exists.

[daixba]

See change as below:
https://github.com/opensearch-project/data-prepper/pull/3592/files#diff-d0164dc5e11eb3ef88aa4fabf7ae3fd4b0ebda3c17120c2fcc7849a4d43578c4L96-R96

[dlvenable]

Yes, roughly 1 out of 10 times that condition should occur, so I think that is likely the fix. For the future, would be nice to have had a separate PR so that such a nice fix is not blocked by other PR comments.

[dlvenable]

Resolved by #3592.