Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix snapshot differences (it may need muting h_ppmeans()) #1072

Merged
merged 4 commits into from
Oct 2, 2023

Conversation

Melkiades
Copy link
Contributor

@Melkiades Melkiades commented Sep 29, 2023

Fixes #1058

The error comes ONLY from integration tests (mismatch of output rates and rate CI in h_ppmeans()). @ayogasekaram tested on docker image, @edelarua and I tested it (I think window-based rstudio and Ubuntu rstudio-server) and there are no differences in our snapshots, i.e. it is passing for all of us. Now I think this needs to come from stats::predict step in h_ppmeans as also rates are different. It seems we tested everything (adding random seed does not change my local snapshots). Could it have something to do with the system libraries? what do you think @cicdguy? If we cannot fix it, and it is not a fundamental feature we might want to turn it off. Is this function used in the biomarker catalog @danielinteractive? To my knowledge, it is not used in the TLG-c, right @ayogasekaram @edelarua?

Please refer to #1059 and integration tests for the changes that exclude the fit itself from having any changes local vs integration tests.

@Melkiades Melkiades added bug Something isn't working sme automation labels Sep 29, 2023
@danielinteractive
Copy link
Contributor

This is not used in BAC AFAIK 👍🏼

@github-actions
Copy link
Contributor

badge

Code Coverage Summary

Filename                                   Stmts    Miss  Cover    Missing
---------------------------------------  -------  ------  -------  -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
R/abnormal_by_baseline.R                      63       0  100.00%
R/abnormal_by_marked.R                        53       5  90.57%   115-119
R/abnormal_by_worst_grade_worsen.R           114       3  97.37%   233-235
R/abnormal_by_worst_grade.R                   38       0  100.00%
R/abnormal.R                                  41       0  100.00%
R/analyze_variables.R                        192       5  97.40%   483, 686-687, 704-705
R/analyze_vars_in_cols.R                     166      29  82.53%   199-204, 219, 233-234, 242-247, 253-259, 334-340
R/combination_function.R                       9       0  100.00%
R/compare_variables.R                        120      15  87.50%   127-131, 243, 321-330, 385
R/control_incidence_rate.R                    20       8  60.00%   32-35, 38-41
R/control_logistic.R                           7       0  100.00%
R/control_step.R                              23       1  95.65%   58
R/control_survival.R                          15       0  100.00%
R/count_cumulative.R                          48       1  97.92%   63
R/count_missed_doses.R                        32       0  100.00%
R/count_occurrences_by_grade.R               103       4  96.12%   156-158, 161
R/count_occurrences.R                         73       1  98.63%   92
R/count_patients_events_in_cols.R             67       1  98.51%   62
R/count_patients_with_event.R                 45       0  100.00%
R/count_patients_with_flags.R                 56       4  92.86%   71-72, 77-78
R/count_values.R                              25       0  100.00%
R/cox_regression_inter.R                     154       0  100.00%
R/cox_regression.R                           161       0  100.00%
R/coxph.R                                    167       7  95.81%   191-195, 239, 254, 262, 268-269
R/d_pkparam.R                                406       0  100.00%
R/decorate_grob.R                            169      40  76.33%   232-263, 323-325, 332, 353-390
R/desctools_binom_diff.R                     663      66  90.05%   52, 87-88, 128-129, 132, 211, 237-246, 285, 287, 307, 311, 315, 319, 375, 378, 381, 384, 445, 453, 465-466, 472-475, 483, 486, 495, 498, 546-547, 549-550, 552-553, 555-556, 626, 638-651, 656, 703, 716, 720
R/df_explicit_na.R                            30       0  100.00%
R/estimate_multinomial_rsp.R                  48       1  97.92%   60
R/estimate_proportion.R                      200      12  94.00%   75-82, 86, 91, 296, 463, 568
R/fit_rsp_step.R                              36       0  100.00%
R/fit_survival_step.R                         36       0  100.00%
R/formatting_functions.R                     162       3  98.15%   145, 155, 243
R/g_forest.R                                 438      23  94.75%   199, 251-252, 319, 336-337, 342-343, 356, 372, 419, 450, 526, 535, 616-620, 630, 705, 708, 832
R/g_lineplot.R                               199      30  84.92%   160, 173, 201, 227-230, 307-314, 332-333, 339-349, 441, 447, 449
R/g_step.R                                    68       1  98.53%   109
R/g_waterfall.R                               47       0  100.00%
R/h_adsl_adlb_merge_using_worst_flag.R        73       0  100.00%
R/h_biomarkers_subgroups.R                    40       0  100.00%
R/h_cox_regression.R                         110       0  100.00%
R/h_logistic_regression.R                    468       3  99.36%   206-207, 276
R/h_map_for_count_abnormal.R                  54       0  100.00%
R/h_pkparam_sort.R                            15       0  100.00%
R/h_response_biomarkers_subgroups.R           75       0  100.00%
R/h_response_subgroups.R                     171      12  92.98%   257-270
R/h_stack_by_baskets.R                        64       1  98.44%   89
R/h_step.R                                   180       0  100.00%
R/h_survival_biomarkers_subgroups.R           79       0  100.00%
R/h_survival_duration_subgroups.R            200      12  94.00%   259-271
R/imputation_rule.R                           17       2  88.24%   54-55
R/incidence_rate.R                            94       7  92.55%   53-60
R/individual_patient_plot.R                  133       0  100.00%
R/kaplan_meier_plot.R                        683      65  90.48%   230-233, 273-308, 317-321, 530, 717-719, 727-729, 761-762, 935-938, 1161, 1478-1489
R/logistic_regression.R                      101       0  100.00%
R/missing_data.R                              21       3  85.71%   32, 66, 76
R/odds_ratio.R                               107       0  100.00%
R/prop_diff_test.R                            89       0  100.00%
R/prop_diff.R                                261      16  93.87%   72-75, 107, 269-276, 415, 475, 580
R/prune_occurrences.R                         57      10  82.46%   138-142, 188-192
R/response_biomarkers_subgroups.R             60       0  100.00%
R/response_subgroups.R                       165       4  97.58%   268, 312-314
R/riskdiff.R                                  48       7  85.42%   85-88, 95, 105-106
R/rtables_access.R                            38       4  89.47%   159-162
R/score_occurrences.R                         20       1  95.00%   124
R/split_cols_by_groups.R                      49       0  100.00%
R/stat.R                                      59       3  94.92%   73-74, 129
R/summarize_ancova.R                          96       1  98.96%   180
R/summarize_change.R                          28       0  100.00%
R/summarize_colvars.R                          9       0  100.00%
R/summarize_coxreg.R                         165       2  98.79%   198, 420
R/summarize_glm_count.R                      165      29  82.42%   158, 162-213, 261-262
R/summarize_num_patients.R                    97       9  90.72%   103-105, 153-154, 235-240
R/summarize_patients_exposure_in_cols.R       97       1  98.97%   56
R/survival_biomarkers_subgroups.R             60       0  100.00%
R/survival_coxph_pairwise.R                   74       9  87.84%   59-67
R/survival_duration_subgroups.R              172       0  100.00%
R/survival_time.R                             48       0  100.00%
R/survival_timepoint.R                       118       7  94.07%   126-132
R/utils_checkmate.R                           68       0  100.00%
R/utils_default_stats_formats_labels.R        95       3  96.84%   408-411
R/utils_factor.R                             109       2  98.17%   84, 302
R/utils_grid.R                               111       5  95.50%   149, 258-264
R/utils_rtables.R                             90       7  92.22%   24, 31-35, 376-377
R/utils.R                                    137      10  92.70%   92, 94, 98, 118, 121, 124, 128, 137-138, 311
TOTAL                                       9564     495  94.82%

Diff against main

Filename                   Stmts    Miss  Cover
-----------------------  -------  ------  -------
R/summarize_glm_count.R        0     +24  -14.55%
TOTAL                          0     +24  -0.25%

Results for commit: f9c5826

Minimum allowed coverage is 80%

♻️ This comment has been updated with latest results

@github-actions
Copy link
Contributor

github-actions bot commented Sep 29, 2023

Unit Tests Summary

       1 files       80 suites   1m 6s ⏱️
   776 tests    767 ✔️     9 💤 0
1 652 runs  1 038 ✔️ 614 💤 0

Results for commit a82e1fa.

♻️ This comment has been updated with latest results.

@edelarua
Copy link
Contributor

In TLG-C, summarize_glm_count is used only in RATET01.

See context:

  ...
  ) %>%
  summarize_glm_count(
    vars = "AVAL",
    variables = list(arm = "ARM", offset = "lgTMATRSK", covariates = NULL),
    conf_level = 0.95,
    distribution = "poisson",
    rate_mean_method = "emmeans",
    var_labels = "Unadjusted exacerbation rate (per year)",
    table_names = "unadj",
    .stats = c("rate"),
    .labels = c(rate = "Rate")
  ) %>%
  summarize_glm_count(
    vars = "AVAL",
    variables = list(arm = "ARM", offset = "lgTMATRSK", covariates = c("REGION1")),
    conf_level = 0.95,
    distribution = "quasipoisson",
    rate_mean_method = "ppmeans",
    var_labels = "Adjusted (QP) exacerbation rate (per year)",
    table_names = "adj",
    .stats = c("rate", "rate_ci", "rate_ratio", "rate_ratio_ci", "pval"),
    .labels = c(
      rate = "Rate", rate_ci = "Rate CI", rate_ratio = "Rate Ratio",
      rate_ratio_ci = "Rate Ratio CI", pval = "p value"
    )
  )

@@ -186,6 +187,7 @@ testthat::test_that("h_ppmeans works with healthy input", {

testthat::expect_snapshot(fits2)

# XXX ppmeans fails snapshot diff in integration tests
result <- h_ppmeans(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping this failing to see if set.seed helps and if also AVR is triggered by this change

@Melkiades Melkiades enabled auto-merge (squash) October 2, 2023 09:48
@Melkiades Melkiades merged commit 36a378b into main Oct 2, 2023
@Melkiades Melkiades deleted the 1058_fix_snaps_diff_v2@main branch October 2, 2023 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automation bug Something isn't working sme
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate variable printout forsummarize_glm_count
4 participants