Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Refactor] Update RNG code from upstream #2278

Merged
merged 55 commits into from
Apr 14, 2021

Commits on Apr 14, 2021

  1. Make FastRandomContext support standard C++11 RNG interface

    This makes it possible to plug it into the various standard C++11 random
    distribution algorithms and other functions like std::shuffle.
    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    94b2ead View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e862564 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    1cdf124 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    746d466 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    af52bf5 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    972effa View commit details
    Browse the repository at this point in the history
  7. Simplify testing RNG code

    Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    32e6c42 View commit details
    Browse the repository at this point in the history
  8. Don't log RandAddSeedPerfmon details

    These are hard to deal with, as in a follow-up this function can get
    called before the logging infrastructure is initialized.
    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    1a5dbc5 View commit details
    Browse the repository at this point in the history
  9. Automatically initialize RNG on first use.

    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    d76ee83 View commit details
    Browse the repository at this point in the history
  10. Rename some hardware RNG related functions

    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    2326535 View commit details
    Browse the repository at this point in the history
  11. Add thread safety annotations to RNG state

    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    298f97c View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    5f20e62 View commit details
    Browse the repository at this point in the history
  13. Integrate util/system's CInit into RNGState

    This guarantees that OpenSSL is initialized properly whenever randomness
    is used, even when that randomness is invoked from global constructors.
    
    Note that this patch uses Mutex directly, rather than CCriticalSection.
    This is because the lock-detection code is not necessarily initialized
    during global constructors.
    Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    038a45a View commit details
    Browse the repository at this point in the history
  14. Switch all RNG code to the built-in PRNG.

    It includes the following policy changes:
    * All GetRand* functions seed the stack pointer and rdrand result
      (in addition to the performance counter)
    * The periodic entropy added by the idle scheduler now seeds stack pointer,
      rdrand and perfmon data (once every 10 minutes) in addition to
      just a sleep timing.
    * The entropy added when calling GetStrongRandBytes no longer includes
      the once-per-10-minutes perfmon data on windows (it is moved to the
      idle scheduler instead, where latency matters less).
    
    Other changes:
    * OpenSSL is no longer seeded directly anywhere. Instead, any generated
      randomness through our own RNG is fed back to OpenSSL (after an
      additional hashing step to prevent leaking our RNG state).
    * Seeding that was previously done directly in RandAddSeedSleep is now
      moved to SeedSleep(), which is indirectly invoked through ProcRand
      from RandAddSeedSleep.
    * Seeding that was previously done directly in GetStrongRandBytes()
      is now moved to SeedSlow(), which is indirectly invoked through
      ProcRand from GetStrongRandBytes().
    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    698d133 View commit details
    Browse the repository at this point in the history
  15. Remove hwrand_initialized.

     All access to hwrand is now gated by GetRNGState, which initializes the hwrand code.
    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    774899f View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    5bc2583 View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    787d72f View commit details
    Browse the repository at this point in the history
  18. Encapsulate RNGState better

    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    080deb3 View commit details
    Browse the repository at this point in the history
  19. Use secure allocator for RNG state

    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    2b6381e View commit details
    Browse the repository at this point in the history
  20. Document RNG design in random.h

    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    4ffda1f View commit details
    Browse the repository at this point in the history
  21. Use RdSeed when available, and reduce RdRand load

    This introduces support for autodetecting and using the RdSeed instruction.
    
    In addition:
    * In SeedFast, only 64 bits of entropy are generated through RdRand (256 was relatively slow).
    * In SeedStartup, 256 bits of entropy are generated, using RdSeed (preferably) or RdRand (otherwise).
    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    67e336d View commit details
    Browse the repository at this point in the history
  22. Add hash strengthening to the RNG

    Once every minute, this will feed the RNG state through repeated SHA512
    for 10ms. The timings of that operation are used as entropy source as
    well.
    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    0190dec View commit details
    Browse the repository at this point in the history
  23. Document strenghtening

    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    7b33223 View commit details
    Browse the repository at this point in the history
  24. test: Make bloom tests deterministic

    MarcoFalke authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    c82e359 View commit details
    Browse the repository at this point in the history
  25. Configuration menu
    Copy the full SHA
    28c9cdb View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    6966aa9 View commit details
    Browse the repository at this point in the history
  27. Add ChaCha20 bench

    jonasschnelli authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    79e7fd3 View commit details
    Browse the repository at this point in the history
  28. Configuration menu
    Copy the full SHA
    22a7121 View commit details
    Browse the repository at this point in the history
  29. gui: remove OpenSSL PRNG seeding (Windows, Qt only)

    This removes the code introduced in [dashpay#4399](bitcoin#4399)
    that attempts to add additional entroy to the OpenSSL PRNG using Windows messages.
    Note that this is specific to bitcoin-qt running on Windows.
    
    ```
    RAND_event() collects the entropy from Windows events such as mouse movements and other user interaction.
    It should be called with the iMsg, wParam and lParam arguments of all messages sent to the window procedure.
    It will estimate the entropy contained in the event message (if any), and add it to the PRNG.
    The program can then process the messages as usual.
    ```
    
    Besides BIP70, this is the last place we are directly using OpenSSL in the
    GUI code. All other OpenSSL usage is in random.cpp.
    
    Note that we are still also doing Windows specific entropy gathering in multiple
    other places. Such as [RandAddSeedPerfmon](https://github.com/bitcoin/bitcoin/blob/master/src/random.cpp#L268)
    and [RAND_screen()](https://github.com/bitcoin/bitcoin/blob/master/src/random.cpp#L600).
    
    Also note that if RAND_event returns 0 (PRNG has NOT been seeded with enough data), we're
    just logging a message and continuing on, which seems less than ideal.
    fanquake authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    ada9868 View commit details
    Browse the repository at this point in the history
  30. random: remove call to RAND_screen() (Windows only)

    Follow up to bitcoin#17151 where
    there were multiple calls to also remove our call to RAND_screen().
    fanquake authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    5eed08c View commit details
    Browse the repository at this point in the history
  31. Configuration menu
    Copy the full SHA
    630931f View commit details
    Browse the repository at this point in the history
  32. Fix FreeBSD build by including utilstrencodings.h

    `random.cpp` needs to explicitly include `utilstrencodings.h` to get
    `ARRAYLEN`. This fixes the FreeBSD build.
    Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    909473e View commit details
    Browse the repository at this point in the history
  33. Configuration menu
    Copy the full SHA
    fccd2b8 View commit details
    Browse the repository at this point in the history
  34. doc: minor corrections in random.cpp

    This should have been part of bitcoin#17151.
    fanquake authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    27cf995 View commit details
    Browse the repository at this point in the history
  35. Configuration menu
    Copy the full SHA
    52b5336 View commit details
    Browse the repository at this point in the history
  36. Configuration menu
    Copy the full SHA
    7bde8b7 View commit details
    Browse the repository at this point in the history
  37. Seed randomness with process id / thread id / various clocks

    This sort of data is also used by OpenSSL.
    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    6142e1f View commit details
    Browse the repository at this point in the history
  38. Gather additional entropy from the environment

    This based on code by Gregory Maxwell.
    
    # Conflicts:
    #	src/randomenv.cpp
    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    67de246 View commit details
    Browse the repository at this point in the history
  39. Use sysctl for seeding on MacOS/BSD

    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    8f5b9c9 View commit details
    Browse the repository at this point in the history
  40. Feed CPUID data into RNG

    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    88d97d0 View commit details
    Browse the repository at this point in the history
  41. Add information gathered through getauxval()

    Suggested by Wladimir van der Laan.
    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    4679181 View commit details
    Browse the repository at this point in the history
  42. Run background seeding periodically instead of unpredictably

    * Instead of calling RandAddSeedSleep anytime the scheduler goes
      idle, call its replacement (RandAddSeedPeriodic) just once per
      minute. This has better guarantees of actually being run, and
      helps limit how frequently the dynamic env data is gathered.
    * Since this code runs once per minute regardless now, we no
      longer need to keep track of the last time strengthening was
      run; just do it always.
    * Make strengthening time context dependent (100 ms at startup,
      10 ms once per minute afterwards).
    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    7d6ddcb View commit details
    Browse the repository at this point in the history
  43. Configuration menu
    Copy the full SHA
    f363ea9 View commit details
    Browse the repository at this point in the history
  44. Configuration menu
    Copy the full SHA
    81d382f View commit details
    Browse the repository at this point in the history
  45. random: mark RandAddPeriodic and SeedPeriodic as noexcept

    The usage of MilliSleep() in SeedPeriodic (previously SeedSleep) was
    removed in bitcoin#17270, meaning it, and its users can now be marked noexcept.
    fanquake authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    88c2ae5 View commit details
    Browse the repository at this point in the history
  46. Move events_hasher into RNGState()

    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    8a9bbb1 View commit details
    Browse the repository at this point in the history
  47. Fix CPUID subleaf iteration

    sipa authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    41ab1ff View commit details
    Browse the repository at this point in the history
  48. Configuration menu
    Copy the full SHA
    fec460c View commit details
    Browse the repository at this point in the history
  49. Configuration menu
    Copy the full SHA
    b70b26f View commit details
    Browse the repository at this point in the history
  50. Configuration menu
    Copy the full SHA
    77bddd7 View commit details
    Browse the repository at this point in the history
  51. build: avoid getifaddrs when unavailable

    theuni authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    3a039d6 View commit details
    Browse the repository at this point in the history
  52. rand: only try and use freeifaddrs if available

    # Conflicts:
    #	src/randomenv.cpp
    fanquake authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    414f405 View commit details
    Browse the repository at this point in the history
  53. Configuration menu
    Copy the full SHA
    e906436 View commit details
    Browse the repository at this point in the history
  54. net: add ifaddrs.h include

    fanquake authored and Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    d9f67da View commit details
    Browse the repository at this point in the history
  55. Use secure.h header for secure allocators

    PIVX-Project#2276 split the `allocators.h` header and has been merged
    Fuzzbawls committed Apr 14, 2021
    Configuration menu
    Copy the full SHA
    cecbf6c View commit details
    Browse the repository at this point in the history