Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different Gen.int distributions #164

Closed
jmid opened this issue Aug 18, 2021 · 1 comment
Closed

Different Gen.int distributions #164

jmid opened this issue Aug 18, 2021 · 1 comment

Comments

@jmid
Copy link
Collaborator

jmid commented Aug 18, 2021

The regression diff -y finds a difference in average, standard deviation, etc. of Gen.int whereas the remaining generators reveal the exact same distribution:

+++ Stats for int dist ++++++++++++++++++++++++++++++++++++++++++++++++++++++	+++ Stats for int dist ++++++++++++++++++++++++++++++++++++++++++++++++++++++

stats dist:									stats dist:
  num: 100000, avg: 2541076923587387.50, stddev: 2660730801206827008.00, medi |	  num: 100000, avg: 6126662802445055.00, stddev: 2661484817981980672.00, medi
  -4611522359435274428..-4150369195341695293: ############################### |	  -4611578348806740501..-4150415539984062486: ###############################
  -4150369195341695292..-3689216031248116157: ############################### |	  -4150415539984062485..-3689252731161384470: ###############################
  -3689216031248116156..-3228062867154537021: ############################### |	  -3689252731161384469..-3228089922338706454: ###############################
  -3228062867154537020..-2766909703060957885: ############################### |	  -3228089922338706453..-2766927113516028438: ###############################
  -2766909703060957884..-2305756538967378749: ############################### |	  -2766927113516028437..-2305764304693350422: ###############################
  -2305756538967378748..-1844603374873799613: ############################### |	  -2305764304693350421..-1844601495870672406: ###############################
  -1844603374873799612..-1383450210780220477: ############################### |	  -1844601495870672405..-1383438687047994390: ###############################
  -1383450210780220476.. -922297046686641341: ############################### |	  -1383438687047994389.. -922275878225316374: ###############################
   -922297046686641340.. -461143882593062205: ############################### |	   -922275878225316373.. -461113069402638358: ###############################
   -461143882593062204..       9281500516931: ############################### |	   -461113069402638357..      49739420039658: ###############################
         9281500516932..  461162445594096067: ############################### |	        49739420039659..  461212548242717674: ###############################
    461162445594096068..  922315609687675203: ############################### |	    461212548242717675..  922375357065395690: ###############################
    922315609687675204.. 1383468773781254339: ############################### |	    922375357065395691.. 1383538165888073706: ###############################
   1383468773781254340.. 1844621937874833475: ############################### |	   1383538165888073707.. 1844700974710751722: ###############################
   1844621937874833476.. 2305775101968412611: ############################### |	   1844700974710751723.. 2305863783533429738: ###############################
   2305775101968412612.. 2766928266061991747: ############################### |	   2305863783533429739.. 2767026592356107754: ###############################
   2766928266061991748.. 3228081430155570883: ############################### |	   2767026592356107755.. 3228189401178785770: ###############################
   3228081430155570884.. 3689234594249150019: ############################### |	   3228189401178785771.. 3689352210001463786: ###############################
   3689234594249150020.. 4150387758342729155: ############################### |	   3689352210001463787.. 4150515018824141802: ###############################
   4150387758342729156.. 4611540922436308291: ############################### |	   4150515018824141803.. 4611677827646819818: ###############################

I believe this is expected, since QCheck2.Gen.int relies on a reworked pint generator from
bdfc905. The two underlying implementations look very much the same if one compares QCheck:

  (* Uniform random int generator *)
  let pint =
    if Sys.word_size = 32 then
      fun st -> RS.bits st
    else (* word size = 64 *)
      fun st ->
        RS.bits st                        (* Bottom 30 bits *)
        lor (RS.bits st lsl 30)           (* Middle 30 bits *)
        lor ((RS.bits st land 3) lsl 60)  (* Top 2 bits *)  (* top bit = 0 *)

to QCheck2:

  let pint_raw (st : RS.t) : int =
    if Sys.word_size = 32
    then RS.bits st
    else (* word size = 64 *)
      (* Bottom 30 bits *)
      let right = RS.bits st in
      (* Middle 30 bits *)
      let middle = (RS.bits st lsl 30) in
      (* Technically we could write [3] but this is clearer *)
      let two_bits_mask = 0b11 in
      (* Top 2 bits *)
      let left = ((RS.bits st land two_bits_mask) lsl 60) in
      left lor middle lor right

As far as I can tell, it is better documented (yay!) and the call order to RS.bits is fixed with let-bindings (also yay!). As such, I suspect the difference is actually caused by our good old friend - unspecified evaluation order of function calls 😬
With right-to-left evaluation I thus suspect the QCheck version generates the top, middle, bottom parts in that order, whereas QCheck2 enforces a bottom, middle, top order. Here I think QCheck should follow suit:

  • relying on the evaluation order is bad for reproducability
  • there would be no diff -y for the Gen.int distribution (like all other stat tests so far)

Originally posted by @jmid in #153 (comment)

@jmid
Copy link
Collaborator Author

jmid commented Aug 22, 2021

Addressed by #165.

@jmid jmid closed this as completed Aug 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant