Improve numerical stability of CCA variance #629
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stephenswat
added
tests
stephenswat
changed the title
|
I also had to update the CPU measurement creation algorithm which, as I discovered in this PR, wasn't actually computing the variance at all and just defaulting to |
krasznaa
requested changes
Jun 22, 2024
device/common/include/traccc/clusterization/device/impl/aggregate_cluster.ipp
Outdated
Show resolved
Hide resolved
|
Updated. 👍 |
I noticed that, at some point, a factor of $\frac{1}{12}$ was added to
the variance of measurements and this slipped through because the
tolerance on the variance test was extremely large, i.e. no smaller than
0.1. This is an unacceptably high tolerance, and so I decided that the
variance computation was in need of an update. I decided to adopt two
strategies to do this. The first is the implementation of Welford's
online algorithm, which relies on the following recurrence relation:
$$\sigma^2_n = (1 - \frac{w_n}{W_n}) \sigma^2_{n-1} + \frac{w_n}{W_n} *
(x_n - \mu_n) (x_n - \mu_{n-1})$$
This is significantly less prone to catastrophic cancellation. Second, I
shifted the entire computation by the position of the first cell, which
brings the computation closer to zero where floating point computation
is more accurate. This depends on two equivalences:
$$\mu(x_1, \ldots, x_n) = \mu(x_1 - C, \ldots, x_n - C) + C$$
and
$$\sigma^2(x_1, \ldots, x_n) = \sigma^2(x_1 - C, \ldots, x_n - C)$$
Combined, these factors allow me to drop the tolerance in the tests from
a _minimum_ of 0.1 to a fixed value of 0.0001.
krasznaa
approved these changes
Jun 22, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I noticed that, at some point, a factor of$\frac{1}{12}$ was added to the variance of measurements and this slipped through because the tolerance on the variance test was extremely large, i.e. no smaller than 0.1. This is an unacceptably high tolerance, and so I decided that the variance computation was in need of an update. I decided to adopt two strategies to do this. The first is the implementation of Welford's online algorithm, which relies on the following recurrence relation:
This is significantly less prone to catastrophic cancellation. Second, I shifted the entire computation by the position of the first cell, which brings the computation closer to zero where floating point computation is more accurate. This depends on two equivalences:
and
Combined, these factors allow me to drop the tolerance in the tests from a minimum of 0.1 to a fixed value of 0.0001.