[Refactor] Update RNG code from upstream #2278
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fuzzbawls
changed the title
Fuzzbawls
force-pushed
the
2021_generic-rand
branch
2 times, most recently
from
9208a09
to
77c5f58
Compare
Fuzzbawls
changed the title
random-zebra
previously approved these changes
Apr 8, 2021
|
Added 2 more PRs to this to detect and guard the |
This makes it possible to plug it into the various standard C++11 random distribution algorithms and other functions like std::shuffle.
These are hard to deal with, as in a follow-up this function can get called before the logging infrastructure is initialized.
This guarantees that OpenSSL is initialized properly whenever randomness is used, even when that randomness is invoked from global constructors. Note that this patch uses Mutex directly, rather than CCriticalSection. This is because the lock-detection code is not necessarily initialized during global constructors.
It includes the following policy changes: * All GetRand* functions seed the stack pointer and rdrand result (in addition to the performance counter) * The periodic entropy added by the idle scheduler now seeds stack pointer, rdrand and perfmon data (once every 10 minutes) in addition to just a sleep timing. * The entropy added when calling GetStrongRandBytes no longer includes the once-per-10-minutes perfmon data on windows (it is moved to the idle scheduler instead, where latency matters less). Other changes: * OpenSSL is no longer seeded directly anywhere. Instead, any generated randomness through our own RNG is fed back to OpenSSL (after an additional hashing step to prevent leaking our RNG state). * Seeding that was previously done directly in RandAddSeedSleep is now moved to SeedSleep(), which is indirectly invoked through ProcRand from RandAddSeedSleep. * Seeding that was previously done directly in GetStrongRandBytes() is now moved to SeedSlow(), which is indirectly invoked through ProcRand from GetStrongRandBytes().
All access to hwrand is now gated by GetRNGState, which initializes the hwrand code.
Suggested by Wladimir van der Laan.
* Instead of calling RandAddSeedSleep anytime the scheduler goes idle, call its replacement (RandAddSeedPeriodic) just once per minute. This has better guarantees of actually being run, and helps limit how frequently the dynamic env data is gathered. * Since this code runs once per minute regardless now, we no longer need to keep track of the last time strengthening was run; just do it always. * Make strengthening time context dependent (100 ms at startup, 10 ms once per minute afterwards).
The usage of MilliSleep() in SeedPeriodic (previously SeedSleep) was removed in bitcoin#17270, meaning it, and its users can now be marked noexcept.
# Conflicts: # src/randomenv.cpp
PIVX-Project#2276 split the `allocators.h` header and has been merged
Fuzzbawls
force-pushed
the
2021_generic-rand
branch
from
3ed1055
to
cecbf6c
Compare
furszy
approved these changes
Apr 14, 2021
| /* Test that no values not inserted into the cache are read out of it. | ||
| * | ||
| * There are no repeats in the first 200000 insecure_GetRandHash calls | ||
| */ | ||
| BOOST_AUTO_TEST_CASE(test_cuckoocache_no_fakes) | ||
| { | ||
| insecure_rand = FastRandomContext(true); | ||
| SeedInsecureRand(true); | ||
| CuckooCache::cache<uint256, SignatureCacheHasher> cc{}; | ||
| size_t megabytes = 4; | ||
| cc.setup_bytes(megabytes << 20); | ||
| uint256 v; |
There was a problem hiding this comment.
nit: this v variable is not used anymore
random-zebra
approved these changes
Apr 14, 2021
furszy
added a commit
that referenced
this pull request
May 6, 2021
32473aa build: Allow export of environ symbols (Wladimir J. van der Laan) Pull request description: This export was introduced in #2278 which added ``` //! Necessary on some platforms extern char** environ; ``` Which is one of the causes of the current broken gitian build. Coming from f6e4225 ACKs for top commit: random-zebra: utACK 32473aa Fuzzbawls: utACK 32473aa Tree-SHA512: 3091cdbccb9c8d90a1459479d41d8205a90ccb023a53836c8bc9d84af0c4dce0ce93f174b5b1c19af1f4496839ff48f33dae1124ccdbb2056fad4e1e1de406d8
random-zebra
added a commit
that referenced
this pull request
May 6, 2021
f0f0291 Use thread-safe atomic in perfmon seeder (Pieter Wuille) Pull request description: Follow-up to #2278 now that #2300 has been merged. This pulls in one of the aforementioned commits (bitcoin@64e1e02) that was purposely left out of #2278 ACKs for top commit: furszy: utACK f0f0291 random-zebra: utACK f0f0291 and merging... Tree-SHA512: 2071bc007701acf1cdb2ab758ea3cddea14bc7c447d32095647e7f21e45dbd274bba01939c5f71ec54952577ad1536924e5fcce1928a1e645b91f99f2f714f90
furszy
added a commit
that referenced
this pull request
May 12, 2021
5563331 Snap: remove openssl from nightly snapcraft build requirements (Fuzzbawls) 686bfad doc: Add OpenSSL removal to release notes (Fuzzbawls) f669248 ci: remove OpenSSL installation (Fuzzbawls) 9660aec doc: remove OpenSSL from build instructions and licensing info (Fuzzbawls) 9b2e35d depends: remove OpenSSL package (Fuzzbawls) 9a81d8e CMake: remove OpenSSL detection and libs (Fuzzbawls) 53576bc build: remove OpenSSL detection and libs (fanquake) 5f30c2b Stop using OpenSSL's sha hashing in bip38 code (Fuzzbawls) d531bf2 Use our own hmac_sha256 instead of OpenSSL's in scrypt.cpp (Fuzzbawls) b687f8e Use ctaes instead of OpenSSL's AES in bip38 code (Fuzzbawls) 86c978a Remove unused openssl includes (Fuzzbawls) ab830e5 remove unused EncodeBase64Secure (Fuzzbawls) 690c938 random: Remove remaining OpenSSL calls and locking infrastructure (fanquake) 602c0b2 random: stop retrieving random bytes from OpenSSL (fanquake) b1c8396 random: stop feeding RNG output back into OpenSSL (fanquake) Pull request description: The natural follow-up to #2278, #2286, and #2288. With these three PRs merged, there are only a few minor pieces of code that still rely on OpenSSL: - a call to `RAND_bytes` during the ::SLOW path of ProcRand - feeding output from our RNG back into OpenSSL via `RAND_add` during the ::SLOW and ::SLEEP paths. - an unused function in `utilstrencodings.cpp` (`DecodeBase64Secure()`, now removed) - some stale (un-needed/un-used) header includes - bip38 exclusive usages including the following: - using OpenSSL's AES for encryption, now switched to using ctaes - using OpenSSL to do HMAC_SHA256 hashing in `crypto/scrypt.cpp`, now switched to using our native HMAC_SHA256 header - an unused function in `hash.h` (`std::string Hash(std::string input)`), now removed - a SHA256 Hash function to compute a void pointer, switched to using template objects Upstream PRs backported: bitcoin#17265, bitcoin#17515, and bitcoin#18825 The changes to bip38 were tested by doing two-way encryption/decryption between `master` and this PR ACKs for top commit: random-zebra: ACK 5563331 furszy: k, ACK 5563331 and merging.. Tree-SHA512: bfa7445d7b153bb5ea04b7b52bbedaa07ad5acd1a56221425fa5fb7c20ecbf90f392c85273734ad2a277d4fffc43b10a7a660924a8a41c175ba2fc68e6cf820f
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a collection of upstream PRs that have been backported to bring our RNG (
src/random) code more up-to-date. The following upstream PRs have been included here: