This code complements the submission “Variation in False Negative Rate of RT-PCR Based SARS-CoV-2 Tests by Time Since Exposure”
Zhao et al. (2020), Liu et al. (2020), Guo et al. (2020), Wölfel et al. (2020), Danis et al. (2020), Kujawski et al. (2020), Kim et al. (2020), and Young et al. (2020) each looked at the sensitivity of the RT-PCR by time since symptom onset.
The sensitivity rises to a peak 4 days after symptom onset then declines for the next couple of weeks.
If we know the risk of an individual, we can find the negative predictive value – the probability that someone who tests negative is actually negative. From Bi et al., we know that about 15% (77/517) household contacts later tested positive for covid-19.
We use logistic regression for the sensitivity of the RT-PCR with a cubic polynomial for the log of time since exposure and use that, along with the probability of infection given exposure, to estimate the negative predictive value of the RT-PCR. We use estimates of the incubation period from Lauer, Grantz, et al. (2020). From this, we can find the probability of having a covid-19 infection despite being RT-PCR negative.
We use Stan for this analysis.
| day | False negative rate (95% CI) | False omission rate (95% CI) | Rel. reduction in prob. pos. | Abs. reduction in prob. pos. |
|---|---|---|---|---|
| 1 | 100.0% (100.0-100.0%) | 11.3% (9.0-13.8%) | 0.0% (-0.0-0.0%) | 0.0% (-0.0-0.0%) |
| 2 | 100.0% (96.0-100.0%) | 11.2% (8.9-13.8%) | 0.0% (0.0-3.6%) | 0.0% (0.0-0.4%) |
| 3 | 97.7% (58.8-99.9%) | 10.7% (6.7-13.5%) | 2.0% (0.1-38.3%) | 0.2% (0.0-4.2%) |
| 4 | 71.0% (29.6-94.1%) | 8.1% (3.5-11.8%) | 26.5% (5.3-67.7%) | 3.0% (0.6-7.8%) |
| 5 | 38.7% (18.4-64.6%) | 4.7% (2.2-7.9%) | 58.5% (32.7-79.7%) | 6.5% (3.5-9.6%) |
| 6 | 24.8% (14.0-39.9%) | 3.0% (1.7-5.1%) | 72.9% (57.3-84.4%) | 8.2% (6.0-10.5%) |
| 7 | 20.1% (12.5-31.0%) | 2.5% (1.5-4.0%) | 77.9% (66.3-86.2%) | 8.7% (6.7-11.0%) |
| 8 | 19.1% (12.0-29.1%) | 2.4% (1.4-3.8%) | 78.9% (68.3-86.6%) | 8.9% (6.9-11.2%) |
| 9 | 20.0% (12.8-30.2%) | 2.5% (1.5-4.0%) | 78.0% (67.4-85.8%) | 8.8% (6.8-11.0%) |
| 10 | 22.1% (14.5-32.7%) | 2.7% (1.7-4.3%) | 75.8% (64.7-84.0%) | 8.5% (6.6-10.7%) |
| 11 | 25.0% (16.7-36.0%) | 3.1% (1.9-4.7%) | 72.7% (61.1-81.5%) | 8.2% (6.3-10.3%) |
| 12 | 28.6% (19.5-40.1%) | 3.5% (2.3-5.2%) | 69.0% (57.0-78.6%) | 7.7% (5.9-9.9%) |
| 13 | 32.5% (22.6-44.4%) | 3.9% (2.6-5.8%) | 64.8% (52.6-75.1%) | 7.3% (5.5-9.3%) |
| 14 | 36.8% (26.2-49.0%) | 4.4% (3.0-6.4%) | 60.4% (47.9-71.5%) | 6.8% (5.1-8.8%) |
| 15 | 41.2% (30.0-53.6%) | 4.9% (3.4-7.0%) | 55.8% (43.5-67.5%) | 6.3% (4.6-8.2%) |
| 16 | 45.5% (33.8-58.0%) | 5.4% (3.8-7.6%) | 51.5% (39.1-63.5%) | 5.8% (4.1-7.7%) |
| 17 | 49.6% (37.6-62.1%) | 5.9% (4.2-8.1%) | 47.4% (35.2-59.6%) | 5.3% (3.7-7.2%) |
| 18 | 53.5% (41.2-65.7%) | 6.3% (4.6-8.6%) | 43.5% (31.8-55.9%) | 4.9% (3.4-6.7%) |
| 19 | 57.0% (44.6-68.9%) | 6.7% (4.9-9.0%) | 40.1% (28.7-52.3%) | 4.5% (3.1-6.2%) |
| 20 | 60.2% (47.8-71.6%) | 7.1% (5.2-9.4%) | 37.0% (25.9-49.1%) | 4.2% (2.8-5.8%) |
| 21 | 63.0% (50.8-74.2%) | 7.4% (5.5-9.7%) | 34.3% (23.6-46.2%) | 3.9% (2.6-5.5%) |
With little data on RT-PCR to time prior to symptom onset, the estimates of sensitivity at or below day five are low with large credible intervals. Due to the decline in sensitivity over time, the RT-PCR test is best deployed about a week after exposure. A day or two after exposure (3 or 4 days prior to symptoms), the test may have no utility at all, and thus the probability of having been infected is would be the same with or without an RT-PCR. Seven to nine days after exposure (roughly 2 to 4 days after symptom onset), the negative predictive value is around 97.5%, meaning there is about a 2.5% chance of actually being covid-19 positive despite testing negative (given a 15% attack rate).
What if the specificity of the test is less than 100%? To test this we fit the same model except with a specificity of 90% and compared the results to the original (the sensitivity remained the same).
| Day | False omission rate (90% spec.) | False omission rate (100% spec.) | Rel. increase in FOR | Abs. increase in FOR |
|---|---|---|---|---|
| 1 | 12.4% | 11.3% | 9.9% | 1.1% |
| 2 | 12.4% | 11.2% | 10.0% | 1.1% |
| 3 | 11.8% | 10.7% | 9.6% | 1.0% |
| 4 | 8.8% | 8.1% | 7.6% | 0.6% |
| 5 | 5.0% | 4.7% | 7.2% | 0.3% |
| 6 | 3.3% | 3.0% | 7.5% | 0.2% |
| 7 | 2.7% | 2.5% | 7.5% | 0.2% |
| 8 | 2.6% | 2.4% | 8.1% | 0.2% |
| 9 | 2.7% | 2.5% | 8.8% | 0.2% |
| 10 | 3.0% | 2.7% | 8.8% | 0.2% |
| 11 | 3.3% | 3.1% | 9.1% | 0.3% |
| 12 | 3.8% | 3.5% | 9.1% | 0.3% |
| 13 | 4.3% | 3.9% | 9.3% | 0.4% |
| 14 | 4.9% | 4.4% | 9.5% | 0.4% |
| 15 | 5.4% | 4.9% | 9.4% | 0.5% |
| 16 | 5.9% | 5.4% | 9.5% | 0.5% |
| 17 | 6.5% | 5.9% | 9.5% | 0.6% |
| 18 | 6.9% | 6.3% | 9.5% | 0.6% |
| 19 | 7.4% | 6.7% | 9.6% | 0.6% |
| 20 | 7.7% | 7.1% | 9.7% | 0.7% |
| 21 | 8.1% | 7.4% | 9.5% | 0.7% |
The shape of the curve for 90% specificity is similar to that of 100%, though slightly elevated. The best time to test is still 2-4 days post-symptom onset.
We tested pre-test probabilities of half, twice, and four times that of Bi et al. (2020).
| day | attack rate | FOR | FOR LB | FOR UB |
|---|---|---|---|---|
| 1 | half | 5.6 | 4.1 | 7.4 |
| 2 | half | 5.6 | 4.0 | 7.4 |
| 3 | half | 5.3 | 3.1 | 7.2 |
| 4 | half | 3.9 | 1.6 | 6.1 |
| 5 | half | 2.2 | 1.0 | 4.0 |
| 6 | half | 1.4 | 0.7 | 2.5 |
| 7 | half | 1.1 | 0.6 | 2.0 |
| 8 | half | 1.1 | 0.6 | 1.9 |
| 9 | half | 1.2 | 0.7 | 1.9 |
| 1 | 2x | 22.5 | 19.4 | 25.8 |
| 2 | 2x | 22.4 | 19.2 | 25.7 |
| 3 | 2x | 21.6 | 14.3 | 25.2 |
| 4 | 2x | 16.8 | 7.8 | 22.6 |
| 5 | 2x | 9.9 | 4.9 | 16.1 |
| 6 | 2x | 6.6 | 3.8 | 10.6 |
| 7 | 2x | 5.4 | 3.3 | 8.4 |
| 8 | 2x | 5.2 | 3.2 | 7.9 |
| 9 | 2x | 5.4 | 3.4 | 8.1 |
| 1 | 4x | 44.9 | 41.3 | 48.6 |
| 2 | 4x | 44.8 | 40.8 | 48.6 |
| 3 | 4x | 43.7 | 31.0 | 47.9 |
| 4 | 4x | 36.0 | 18.6 | 44.5 |
| 5 | 4x | 23.5 | 12.6 | 34.9 |
| 6 | 4x | 16.6 | 9.9 | 24.4 |
| 7 | 4x | 13.9 | 8.8 | 20.0 |
| 8 | 4x | 13.3 | 8.6 | 18.7 |
| 9 | 4x | 13.9 | 9.1 | 19.2 |
We originally assumed a 5-day incubation period, what if that was 3 or 7 days instead?
| day | inc period | FOR | FOR LB | FOR UB |
|---|---|---|---|---|
| 1 | 3d | 11.3 | 8.9 | 13.7 |
| 2 | 3d | 10.0 | 4.9 | 12.9 |
| 3 | 3d | 5.3 | 2.4 | 8.9 |
| 4 | 3d | 2.9 | 1.6 | 5.0 |
| 5 | 3d | 2.3 | 1.4 | 3.8 |
| 6 | 3d | 2.2 | 1.3 | 3.6 |
| 7 | 3d | 2.4 | 1.5 | 3.8 |
| 8 | 3d | 2.7 | 1.7 | 4.1 |
| 9 | 3d | 3.1 | 2.0 | 4.6 |
| 10 | 3d | 3.5 | 2.3 | 5.2 |
| 11 | 3d | 4.0 | 2.7 | 5.7 |
| 12 | 3d | 4.5 | 3.1 | 6.3 |
| 13 | 3d | 5.0 | 3.5 | 6.9 |
| 1 | 7d | 11.3 | 9.1 | 13.8 |
| 2 | 7d | 11.3 | 9.1 | 13.8 |
| 3 | 7d | 11.3 | 9.1 | 13.8 |
| 4 | 7d | 11.1 | 8.2 | 13.6 |
| 5 | 7d | 9.8 | 5.1 | 12.8 |
| 6 | 7d | 6.8 | 3.1 | 10.7 |
| 7 | 7d | 4.3 | 2.2 | 7.3 |
| 8 | 7d | 3.1 | 1.7 | 5.0 |
| 9 | 7d | 2.6 | 1.5 | 4.1 |
| 10 | 7d | 2.4 | 1.5 | 3.8 |
| 11 | 7d | 2.5 | 1.5 | 3.9 |
| 12 | 7d | 2.8 | 1.7 | 4.1 |
| 13 | 7d | 3.1 | 1.9 | 4.6 |
Changing the length of the incubation period changes the progression of the false omission rate (post-test probability given test negative). Since the sensitivity is calibrated with respect to the time of symptom onset, an earlier onset time leads to a quicker drop in false omission rate and a later onset time leads to a slower drop.
The timing of the days since symptom onset are ambiguous in Guo et al. and Kim et al., where day 1 may mean one day since symptom onset or the first day of symptoms.
Shifting the timing of ambiguous days one day earlier improves the median sensitivity of the early tests, such that the best days for testing would shift from 2-4 days post-symptom onset to 1-3 days post-symptom onset. The log-likelihoods for each model are nearly identical with the shifted days being minutely better. The overlap in credible intervals indicates that the estimates are roughly equivalent.
Kujawski et al. and Danis et al. reported 21 inconclusive tests combined in their data. In our main analysis, we omit these swabs, but they could count as negative or positive tests instead.
The results are barely changed by including inconclusive tests as negatives or positives instead of being omitted.
To see whether any study has undue influence over our results, we ran a leave-one-study out analysis to see if the results changed.
## Running /Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB foo.c
## clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/Rcpp/include/" -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/" -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/unsupported" -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/BH/include" -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/src/" -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/" -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/rstan/include" -DEIGEN_NO_DEBUG -D_REENTRANT -DBOOST_DISABLE_ASSERTS -DBOOST_PENDING_INTEGER_LOG2_HPP -include stan/math/prim/mat/fun/Eigen.hpp -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fPIC -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -c foo.c -o foo.o
## In file included from <built-in>:1:
## In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
## In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/Dense:1:
## In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/Core:88:
## /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/src/Core/util/Macros.h:613:1: error: unknown type name 'namespace'
## namespace Eigen {
## ^
## /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/src/Core/util/Macros.h:613:16: error: expected ';' after top level declarator
## namespace Eigen {
## ^
## ;
## In file included from <built-in>:1:
## In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
## In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/Dense:1:
## /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/Core:96:10: fatal error: 'complex' file not found
## #include <complex>
## ^~~~~~~~~
## 3 errors generated.
## make: *** [foo.o] Error 1
| days_since_exposure | fnr_med | fnr_lb | fnr_ub | for_med | for_lb | for_ub | rr_med | rr_lb | rr_ub | abs_med | abs_lb | abs_ub |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 100.0 | 100.0 | 100.0 | 11.3 | 8.9 | 13.9 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 2 | 100.0 | 86.8 | 100.0 | 11.2 | 8.4 | 13.9 | 0.0 | 0.0 | 11.7 | 0.0 | 0.0 | 1.4 |
| 3 | 99.1 | 44.8 | 100.0 | 10.9 | 5.1 | 13.6 | 0.8 | 0.0 | 52.4 | 0.1 | 0.0 | 5.8 |
| 4 | 80.2 | 24.8 | 99.0 | 9.0 | 3.0 | 12.6 | 17.9 | 0.9 | 72.8 | 2.0 | 0.1 | 8.4 |
| 5 | 45.7 | 17.5 | 83.4 | 5.5 | 2.1 | 10.1 | 51.2 | 14.9 | 80.8 | 5.7 | 1.7 | 9.7 |
| 6 | 27.9 | 14.0 | 52.3 | 3.4 | 1.7 | 6.5 | 69.6 | 44.7 | 84.5 | 7.8 | 4.8 | 10.4 |
| 7 | 21.3 | 12.3 | 35.1 | 2.6 | 1.4 | 4.5 | 76.6 | 62.2 | 86.4 | 8.6 | 6.4 | 10.9 |
| 8 | 19.3 | 11.8 | 29.9 | 2.4 | 1.4 | 3.9 | 78.7 | 67.7 | 87.0 | 8.9 | 6.8 | 11.1 |
| 9 | 19.5 | 12.3 | 29.7 | 2.4 | 1.4 | 3.8 | 78.5 | 67.8 | 86.4 | 8.8 | 6.8 | 11.0 |
| 10 | 21.0 | 13.3 | 31.6 | 2.6 | 1.5 | 4.1 | 76.9 | 65.8 | 85.2 | 8.6 | 6.7 | 10.8 |
| 11 | 23.5 | 15.1 | 34.9 | 2.9 | 1.8 | 4.5 | 74.3 | 62.4 | 83.3 | 8.3 | 6.4 | 10.5 |
| 12 | 26.8 | 17.5 | 38.7 | 3.3 | 2.1 | 5.0 | 70.9 | 58.5 | 80.7 | 8.0 | 6.0 | 10.1 |
| 13 | 30.4 | 20.4 | 42.9 | 3.7 | 2.4 | 5.5 | 67.0 | 54.1 | 77.6 | 7.5 | 5.6 | 9.7 |
| 14 | 34.5 | 23.7 | 47.4 | 4.2 | 2.7 | 6.1 | 62.7 | 49.6 | 74.1 | 7.1 | 5.2 | 9.2 |
| 15 | 38.7 | 27.3 | 51.9 | 4.7 | 3.1 | 6.7 | 58.3 | 45.1 | 70.3 | 6.6 | 4.7 | 8.7 |
| 16 | 43.1 | 31.0 | 56.2 | 5.2 | 3.5 | 7.2 | 53.9 | 40.8 | 66.3 | 6.1 | 4.3 | 8.1 |
| 17 | 47.4 | 34.9 | 60.5 | 5.7 | 3.9 | 7.8 | 49.6 | 36.8 | 62.4 | 5.6 | 3.9 | 7.6 |
| 18 | 51.4 | 38.7 | 64.2 | 6.1 | 4.3 | 8.3 | 45.6 | 33.2 | 58.4 | 5.1 | 3.5 | 7.1 |
| 19 | 55.1 | 42.3 | 67.5 | 6.5 | 4.7 | 8.7 | 42.0 | 30.0 | 54.7 | 4.7 | 3.2 | 6.6 |
| 20 | 58.4 | 45.6 | 70.4 | 6.9 | 5.0 | 9.1 | 38.7 | 27.2 | 51.3 | 4.3 | 2.9 | 6.1 |
| 21 | 61.4 | 48.9 | 72.9 | 7.2 | 5.3 | 9.5 | 35.8 | 24.8 | 48.0 | 4.0 | 2.6 | 5.7 |
When the incubation period is treated as a random variable instead of a fixed value, the confidence intervals for the false negative rate are wider, especially in the first week of exposure.