You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
we added in hasSideEffects a long time ago to be able to deal with conditional expressions where an exception might be thrown. Spark does lazy evaluation on the CPU but we tend to evaluation both sides greedily. We detect when we need to be lazy based on the hasSideEffects method. But this does not look like it was properly maintained. Things like GpuBinaryArithmetic expressions that just check for ANSI mode instead of checking to see if failOnError returns true.
This ends up being both a performance problem in some cases and a correctness problem in others.
Steps/Code to reproduce bug
Bitwise operations in ANSI mode says it could throw an exception, but it cannot.
GpuAdd in ANSI mode says it has side effects in cases it does not.
I just don't trust that all of our expressions are doing the right thing here and we need to check them all and add tests to verify that they are doing what we expect.
The text was updated successfully, but these errors were encountered:
Describe the bug
we added in hasSideEffects a long time ago to be able to deal with conditional expressions where an exception might be thrown. Spark does lazy evaluation on the CPU but we tend to evaluation both sides greedily. We detect when we need to be lazy based on the hasSideEffects method. But this does not look like it was properly maintained. Things like GpuBinaryArithmetic expressions that just check for ANSI mode instead of checking to see if failOnError returns true.
This ends up being both a performance problem in some cases and a correctness problem in others.
Steps/Code to reproduce bug
Bitwise operations in ANSI mode says it could throw an exception, but it cannot.
GpuAdd in ANSI mode says it has side effects in cases it does not.
I just don't trust that all of our expressions are doing the right thing here and we need to check them all and add tests to verify that they are doing what we expect.
The text was updated successfully, but these errors were encountered: