Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stat overflow #151

Merged
merged 4 commits into from
Aug 13, 2021
Merged

Stat overflow #151

merged 4 commits into from
Aug 13, 2021

Conversation

jmid
Copy link
Collaborator

@jmid jmid commented Aug 10, 2021

This fixes a rare overflow in the histogram labels (and simplifies a couple of other things).

Before (notice the negative labels on the last bucket):

utop # #require "qcheck";;
utop # open QCheck;;
utop # let t = Test.make ~count:1_000 (add_stat ("dist",fun x -> x) (oneof [small_int_corners ();int])) (fun _ -> true);;
val t : Test.t = QCheck.Test.Test <abstr>
utop # QCheck_runner.run_tests ~verbose:true [t];;
random seed: 153870556
generated error fail pass / total     time test name
[✓] 1000    0    0 1000 / 1000     0.0s anon_test_1

+++ Stats for anon_test_1 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++

stats dist:
  num: 1000, avg: -55083208105414400.00, stddev: 1847115855773139200.00, median 9, min -4590718933436425025, max 4611686018427387903
  -4590718933436425025..-4130598685843234370: ##                                                               26
  -4130598685843234369..-3670478438250043714: #                                                                13
  -3670478438250043713..-3210358190656853058: ###                                                              37
  -3210358190656853057..-2750237943063662402: ###                                                              30
  -2750237943063662401..-2290117695470471746: ##                                                               27
  -2290117695470471745..-1829997447877281090: ##                                                               24
  -1829997447877281089..-1369877200284090434: ##                                                               27
  -1369877200284090433.. -909756952690899778: ##                                                               27
   -909756952690899777.. -449636705097709122: ##                                                               21
   -449636705097709121..   10483542495481534: #######################################################         531
     10483542495481535..  470603790088672190: ##                                                               21
    470603790088672191..  930724037681862846: ##                                                               27
    930724037681862847.. 1390844285275053502: ##                                                               24
   1390844285275053503.. 1850964532868244158: ##                                                               25
   1850964532868244159.. 2311084780461434814: ##                                                               28
   2311084780461434815.. 2771205028054625470: ##                                                               23
   2771205028054625471.. 3231325275647816126: ##                                                               23
   3231325275647816127.. 3691445523241006782: ##                                                               25
   3691445523241006783.. 4151565770834197438: #                                                                17
   4151565770834197439.. 4611686018427387903: ##                                                               24
  -4611686018427387713..-4151565770834197058:                                                                   0
================================================================================
success (ran 1 tests)
- : int = 0

After the change it behaves better:

utop # open QCheck;;
utop # QCheck_runner.set_seed 153870556;;
random seed: 153870556
- : unit = ()
utop # let t = Test.make ~count:1_000 (add_stat ("dist",fun x -> x) (oneof [small_int_corners ();int])) (fun _ -> true);;
val t : Test.t = QCheck2.Test.Test <abstr>
utop # QCheck_runner.run_tests ~verbose:true [t];;
generated error fail pass / total     time test name
[✓] 1000    0    0 1000 / 1000     0.0s anon_test_1

+++ Stats for anon_test_1 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++

stats dist:
  num: 1000, avg: -55083208105414400.00, stddev: 1847115855773139200.00, median 9, min -4590718933436425025, max 4611686018427387903
  -4590718933436425025..-4130598685843234370: ##                                                               26
  -4130598685843234369..-3670478438250043714: #                                                                13
  -3670478438250043713..-3210358190656853058: ###                                                              37
  -3210358190656853057..-2750237943063662402: ###                                                              30
  -2750237943063662401..-2290117695470471746: ##                                                               27
  -2290117695470471745..-1829997447877281090: ##                                                               24
  -1829997447877281089..-1369877200284090434: ##                                                               27
  -1369877200284090433.. -909756952690899778: ##                                                               27
   -909756952690899777.. -449636705097709122: ##                                                               21
   -449636705097709121..   10483542495481534: #######################################################         531
     10483542495481535..  470603790088672190: ##                                                               21
    470603790088672191..  930724037681862846: ##                                                               27
    930724037681862847.. 1390844285275053502: ##                                                               24
   1390844285275053503.. 1850964532868244158: ##                                                               25
   1850964532868244159.. 2311084780461434814: ##                                                               28
   2311084780461434815.. 2771205028054625470: ##                                                               23
   2771205028054625471.. 3231325275647816126: ##                                                               23
   3231325275647816127.. 3691445523241006782: ##                                                               25
   3691445523241006783.. 4151565770834197438: #                                                                17
   4151565770834197439.. 4611686018427387903: ##                                                               24
================================================================================
success (ran 1 tests)
- : int = 0

I'm still not too happy about the statistics code, but this is at least a step in the right direction.

@c-cube c-cube merged commit a61a7e3 into c-cube:master Aug 13, 2021
@c-cube
Copy link
Owner

c-cube commented Aug 13, 2021

Good catch, thank you. I don't like the stats code either, but it has never been clean and it's the best we have right now :)

jmid added a commit to jmid/qcheck that referenced this pull request Aug 17, 2021
@jmid jmid deleted the stat-overflow branch September 8, 2021 07:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants