Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky test TestMemoryConnectorTest.testCustomMetricsScanFilter #8691

Closed
findepi opened this issue Jul 28, 2021 · 17 comments · Fixed by #9336 or #19951
Closed

Flaky test TestMemoryConnectorTest.testCustomMetricsScanFilter #8691

findepi opened this issue Jul 28, 2021 · 17 comments · Fixed by #9336 or #19951
Labels
bug Something isn't working

Comments

@findepi
Copy link
Member

findepi commented Jul 28, 2021

https://github.com/trinodb/trino/runs/3179633364

Error:  testCustomMetricsScanFilter(io.trino.plugin.memory.TestMemoryConnectorTest)  Time elapsed: 0.125 s  <<< FAILURE!
org.junit.ComparisonFailure: expected:<LongCount{total=[2000]}> but was:<LongCount{total=[1001]}>
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at io.trino.plugin.memory.TestMemoryConnectorTest.testCustomMetricsScanFilter(TestMemoryConnectorTest.java:156)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

cc @rzeyde-varada
@findepi findepi added the bug Something isn't working label Jul 28, 2021
@findepi findepi mentioned this issue Jul 28, 2021
Merged
@findepi
Copy link
Member Author

findepi commented Jul 28, 2021 •
edited
Loading

https://github.com/trinodb/trino/runs/3181603026 cc @rzeyde-varada

Error:  testCustomMetricsScanFilter(io.trino.plugin.memory.TestMemoryConnectorTest)  Time elapsed: 0.179 s  <<< FAILURE!
org.junit.ComparisonFailure: expected:<LongCount{total=[2000]}> but was:<LongCount{total=[1001]}>
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at io.trino.plugin.memory.TestMemoryConnectorTest.testCustomMetricsScanFilter(TestMemoryConnectorTest.java:156)

@findepi findepi mentioned this issue Jul 28, 2021
Closed
@rzeyde-varada
Copy link
Contributor

Reproduced locally (failed 2 out of 10k runs of testCustomMetricsScanFilter)

org.junit.ComparisonFailure: 
Expected :LongCount{total=2000}
Actual   :LongCount{total=1502}

	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at io.trino.plugin.memory.TestMemoryConnectorTest.testCustomMetricsScanFilter(TestMemoryConnectorTest.java:154)

@rzeyde-varada
Copy link
Contributor

rzeyde-varada commented Jul 29, 2021 •
edited
Loading

testCustomMetricsScanOnly also failed locally 3 times (out of 10k runs) with:

org.junit.ComparisonFailure: 
Expected :LongCount{total=2000}
Actual   :LongCount{total=1162}
(happened twice)

org.junit.ComparisonFailure: 
Expected :LongCount{total=2000}
Actual   :LongCount{total=1834}
(happened once)

@rzeyde-varada
Copy link
Contributor

It seems that also OperatorStats#getInputPositions (which is a "non-custom metric") aggregation returns the wrong result when the above error occurs:

inputRows: 1834
metrics: {rows=LongCount{total=1834}, finished=LongCount{total=8}, started=LongCount{total=8}}

org.junit.ComparisonFailure: 
Expected :LongCount{total=2000}
Actual   :LongCount{total=1834}

I have used the following snippet to compute the total input positions:
rzeyde-varada@4399126#diff-920e9ca800575089f913cc4063da9962bfe7bd4adbd617668c973a7004cd2c99R178

@findepi
Copy link
Member Author

findepi commented Jul 29, 2021

Is it another incarnation of #5172?
We have marked many tests as @Flaky attributing flakiness to that issue.

@findepi
Copy link
Member Author

findepi commented Jul 29, 2021

cc @sopel39

@wendigo wendigo mentioned this issue Aug 2, 2021
Merged
@findepi
Copy link
Member Author

findepi commented Aug 2, 2021

see #8739 (comment)

@rzeyde-varada rzeyde-varada mentioned this issue Aug 5, 2021
Closed
@ebyhr ebyhr mentioned this issue Aug 9, 2021
Merged
@losipiuk losipiuk mentioned this issue Aug 16, 2021
Merged
@kokosing kokosing mentioned this issue Aug 18, 2021
Merged
@findepi findepi mentioned this issue Aug 24, 2021
Merged
@findepi
Copy link
Member Author

findepi commented Aug 24, 2021

https://github.com/trinodb/trino/runs/3409206985

@findepi
Copy link
Member Author

findepi commented Aug 24, 2021

https://github.com/trinodb/trino/runs/3410107887

@findepi findepi mentioned this issue Aug 26, 2021
Merged
@findepi
Copy link
Member Author

findepi commented Aug 26, 2021

https://github.com/trinodb/trino/runs/3429667435 cc @rzeyde-varada

@findepi findepi mentioned this issue Aug 26, 2021
Merged
@findepi
Copy link
Member Author

findepi commented Sep 16, 2021

https://github.com/trinodb/trino/runs/3614059303

This was referenced Sep 16, 2021
Merged
Merged
@findepi
Copy link
Member Author

findepi commented Sep 21, 2021

https://github.com/trinodb/trino/runs/3636321727

@findepi findepi mentioned this issue Sep 22, 2021
Merged
@findepi
Copy link
Member Author

findepi commented Sep 22, 2021

https://github.com/trinodb/trino/runs/3666711537 cc @sopel39 @raunaqmorarka

@sopel39 sopel39 mentioned this issue Sep 23, 2021
Merged
@ebyhr
Copy link
Member

ebyhr commented Oct 8, 2021

https://github.com/trinodb/trino/runs/3835521895

@ebyhr ebyhr mentioned this issue Oct 8, 2021
Merged
@wendigo
Copy link
Contributor

wendigo commented Oct 11, 2021

https://github.com/trinodb/trino/pull/9517/checks

@findepi
Copy link
Member Author

findepi commented Oct 11, 2021

closed this in #9336 18 days ago

seems it wasn't sufficient, per above

@findepi findepi reopened this Oct 11, 2021
@losipiuk losipiuk mentioned this issue Oct 12, 2021
Merged
@ebyhr ebyhr mentioned this issue Oct 13, 2021
Merged
@ebyhr
Copy link
Member

ebyhr commented Oct 13, 2021

https://github.com/trinodb/trino/runs/3876699524

This was referenced Oct 13, 2021
Merged
Merged
@findepi findepi mentioned this issue Nov 29, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Milestone
No milestone
4 participants
@wendigo @findepi @ebyhr @rzeyde-varada