Pre-allocate error vector in TRY #9986
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: TRY(CAST(...)) is up to 4x slower than TRY_CAST when many rows fail. The profile reveals that significant percentage of cpu time goes to EvalCtx::ensureErrorsVectorSize. For every row that fails, we call EvalCtx::ensureErrorsVectorSize to resize the error vector to accommodate that row. When many rows fail we end up resizing a lot: resize(1), resize(2), resize (3),....resize(n). Fix this by pre-allocating error vector in TryExpr. An earlier attempt at fixing this facebookincubator#9911 caused 2x memory regression in one of the streaming pipelines. The change was reverted: facebookincubator#9971 The regression was due to TRY starting to allocate 'nulls' buffer in results unconditionally. Even if there were no errors, TRY would still allocate 'nulls' buffer. When result is a boolean vector, allocating unnecessary 'nulls' buffer increases memory usage for 'result' by 2x. This fix makes sure not to do that and adds a test. Also, this change creates ErrorVector with only nulls buffer allocated. The 'values' buffer that requires ~20 bytes per row is allocated only if an error occurs. Before: ``` ============================================================================ [...]hmarks/ExpressionBenchmarkBuilder.cpp relative time/iter iters/s ============================================================================ cast##try_cast_invalid_empty_input 2.27ms 440.97 cast##tryexpr_cast_invalid_empty_input 8.96ms 111.56 cast##try_cast_invalid_nan 5.49ms 182.26 cast##tryexpr_cast_invalid_nan 12.96ms 77.17 ``` After: ``` cast##try_cast_invalid_empty_input 2.22ms 451.34 cast##tryexpr_cast_invalid_empty_input 4.52ms 221.06 cast##try_cast_invalid_nan 5.79ms 172.69 cast##tryexpr_cast_invalid_nan 8.16ms 122.48 ``` Differential Revision: D57968341
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D57968341 |
✅ Deploy Preview for meta-velox canceled.
|
xiaoxmeng
approved these changes
May 30, 2024
bikramSingh91
approved these changes
May 31, 2024
|
This pull request has been merged in 76424c0. |
|
Conbench analyzed the 1 benchmark run on commit There were no benchmark performance regressions. 🎉 The full Conbench report has more details. |
Joe-Abraham
pushed a commit
to Joe-Abraham/velox
that referenced
this pull request
Jun 7, 2024
Summary: Pull Request resolved: facebookincubator#9986 TRY(CAST(...)) is up to 4x slower than TRY_CAST when many rows fail. The profile reveals that significant percentage of cpu time goes to EvalCtx::ensureErrorsVectorSize. For every row that fails, we call EvalCtx::ensureErrorsVectorSize to resize the error vector to accommodate that row. When many rows fail we end up resizing a lot: resize(1), resize(2), resize (3),....resize(n). Fix this by pre-allocating error vector in TryExpr. An earlier attempt at fixing this facebookincubator#9911 caused 2x memory regression in one of the streaming pipelines. The change was reverted: facebookincubator#9971 The regression was due to TRY starting to allocate 'nulls' buffer in results unconditionally. Even if there were no errors, TRY would still allocate 'nulls' buffer. When result is a boolean vector, allocating unnecessary 'nulls' buffer increases memory usage for 'result' by 2x. This fix makes sure not to do that and adds a test. Also, this change creates ErrorVector with only nulls buffer allocated. The 'values' buffer that requires ~20 bytes per row is allocated only if an error occurs. Before: ``` ============================================================================ [...]hmarks/ExpressionBenchmarkBuilder.cpp relative time/iter iters/s ============================================================================ cast##try_cast_invalid_empty_input 2.27ms 440.97 cast##tryexpr_cast_invalid_empty_input 8.96ms 111.56 cast##try_cast_invalid_nan 5.49ms 182.26 cast##tryexpr_cast_invalid_nan 12.96ms 77.17 ``` After: ``` cast##try_cast_invalid_empty_input 2.22ms 451.34 cast##tryexpr_cast_invalid_empty_input 4.52ms 221.06 cast##try_cast_invalid_nan 5.79ms 172.69 cast##tryexpr_cast_invalid_nan 8.16ms 122.48 ``` Reviewed By: xiaoxmeng, bikramSingh91 Differential Revision: D57968341 fbshipit-source-id: d9f44aeda56596d9efb035ff9fada5eae22bea1d
Joe-Abraham
pushed a commit
to Joe-Abraham/velox
that referenced
this pull request
Jun 7, 2024
Summary: Pull Request resolved: facebookincubator#9986 TRY(CAST(...)) is up to 4x slower than TRY_CAST when many rows fail. The profile reveals that significant percentage of cpu time goes to EvalCtx::ensureErrorsVectorSize. For every row that fails, we call EvalCtx::ensureErrorsVectorSize to resize the error vector to accommodate that row. When many rows fail we end up resizing a lot: resize(1), resize(2), resize (3),....resize(n). Fix this by pre-allocating error vector in TryExpr. An earlier attempt at fixing this facebookincubator#9911 caused 2x memory regression in one of the streaming pipelines. The change was reverted: facebookincubator#9971 The regression was due to TRY starting to allocate 'nulls' buffer in results unconditionally. Even if there were no errors, TRY would still allocate 'nulls' buffer. When result is a boolean vector, allocating unnecessary 'nulls' buffer increases memory usage for 'result' by 2x. This fix makes sure not to do that and adds a test. Also, this change creates ErrorVector with only nulls buffer allocated. The 'values' buffer that requires ~20 bytes per row is allocated only if an error occurs. Before: ``` ============================================================================ [...]hmarks/ExpressionBenchmarkBuilder.cpp relative time/iter iters/s ============================================================================ cast##try_cast_invalid_empty_input 2.27ms 440.97 cast##tryexpr_cast_invalid_empty_input 8.96ms 111.56 cast##try_cast_invalid_nan 5.49ms 182.26 cast##tryexpr_cast_invalid_nan 12.96ms 77.17 ``` After: ``` cast##try_cast_invalid_empty_input 2.22ms 451.34 cast##tryexpr_cast_invalid_empty_input 4.52ms 221.06 cast##try_cast_invalid_nan 5.79ms 172.69 cast##tryexpr_cast_invalid_nan 8.16ms 122.48 ``` Reviewed By: xiaoxmeng, bikramSingh91 Differential Revision: D57968341 fbshipit-source-id: d9f44aeda56596d9efb035ff9fada5eae22bea1d
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
TRY(CAST(...)) is up to 4x slower than TRY_CAST when many rows fail.
The profile reveals that significant percentage of cpu time goes to
EvalCtx::ensureErrorsVectorSize. For every row that fails, we call
EvalCtx::ensureErrorsVectorSize to resize the error vector to accommodate that
row. When many rows fail we end up resizing a lot: resize(1), resize(2), resize
(3),....resize(n). Fix this by pre-allocating error vector in TryExpr.
An earlier attempt at fixing this #9911 caused 2x memory regression in one of the streaming pipelines. The change
was reverted: #9971
The regression was due to TRY starting to allocate 'nulls' buffer in results
unconditionally. Even if there were no errors, TRY would still allocate 'nulls'
buffer. When result is a boolean vector, allocating unnecessary 'nulls' buffer
increases memory usage for 'result' by 2x. This fix makes sure not to do that
and adds a test.
Also, this change creates ErrorVector with only nulls buffer allocated.
The 'values' buffer that requires ~20 bytes per row is allocated only if an
error occurs.
Before:
After:
Differential Revision: D57968341