bench: add support for custom data directory #31000

furszy · 2024-09-29T21:01:21Z

Expands the benchmark framework with the existing -testdatadir arg,
enabling the ability to change the benchmark data directory.

This is useful for running benchmarks on different storage devices, and
not just under the OS /tmp/ directory.

A good use case is #28574, where we are benchmarking the wallet
migration process on an HDD.

DrahtBot · 2024-09-29T21:01:25Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31000.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	tdb3, hodlinator, maflcko, pablomartin4btc, achow101

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#31260 (WIP: scripted-diff: Type-safe settings retrieval by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

DrahtBot · 2024-09-29T22:24:54Z

🚧 At least one of the CI tasks failed.
_{Debug: https://github.com/bitcoin/bitcoin/runs/30825058976}

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the
affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

maflcko · 2024-09-30T05:43:10Z

Just as a note, it should already be possible to pick the device via an env var, see https://en.cppreference.com/w/cpp/filesystem/temp_directory_path#Notes.

EricMujjona

Efficient code

furszy · 2024-09-30T20:42:47Z

Just as a note, it should already be possible to pick the device via an env var, see https://en.cppreference.com/w/cpp/filesystem/temp_directory_path#Notes.

Hmm, thats a good observation.
Even when that may be possible now, I'm not sure we should start recommending the mixture of env vars and binary args at the user interface level. It seems more natural to plug the already existing -testdatadir arg to the benchmark framework. Still, if we move towards the env var direction, the next step should be revert #26564?.

maflcko · 2024-10-01T05:35:13Z

the next step should be revert #26564?.

IIUC the motivation for 26564 was to have a fully static and fixed path for the unit test datadir. My understanding is that changing the temp storage device was just a side-effect and not the primary motivation. (My comment was just a note to say that your goal is already achievable today)

It seems more natural to plug the already existing -testdatadir arg to the benchmark framework.

I agree. I've also suggested this in #30737 (comment)

furszy · 2024-10-01T18:01:39Z

the next step should be revert #26564?.

IIUC the motivation for 26564 was to have a fully static and fixed path for the unit test datadir. My understanding is that changing the temp storage device was just a side-effect and not the primary motivation. (My comment was just a note to say that your goal is already achievable today)

Practically speaking, we're ultimately talking about pretty much the same feature. It's about customizing the test root directory path so it can be inspected or used in another context. The thing is, 26564 introduced several different features in the same commit: the test no cleanup, the directory name specialization using the test name, and the -testdatadir arg. And what I meant is that this last point can be achieved by setting the environment variable you mentioned. Yet, it seems we agree on not doing it, but it's curious that this can still be used nowadays.

hodlinator

Concept ACK 1ba225c

Good to see these globals implemented for bench. As already mentioned, I've been working on a partially overlapping change in #30737.

hodlinator · 2024-10-03T07:44:02Z

src/bench/bench_bitcoin.cpp

@@ -60,6 +61,17 @@ static uint8_t parsePriorityLevel(const std::string& str) {
    return levels;
 }

+// Parses test setup related arguments
+static std::vector<std::string> parseTestSetupArgs(const ArgsManager& argsman) {


Braces on new lines for classes, functions, methods.

Can see you are following style of function above, but function below follows the dev-notes.

Don't feel as strongly about capitalization.

Suggested change

static std::vector<std::string> parseTestSetupArgs(const ArgsManager& argsman) {

static std::vector<std::string> ParseTestSetupArgs(const ArgsManager& argsman)

{

If you want to make parsePriorityLevel conform to the dev-notes as well, I support it.

Done. Added the jump line. I don't feel strong about the capitalization neither but it looks incorrect to introduce it now when all other functions aren't using it.

I don't feel strong about the capitalization neither but it looks incorrect to introduce it now when all other functions aren't using it.

(SetupBenchArgs is capitalized, main must be non-capitalized, so that only leaves parseAsymptote to change beyond the function under discussion).

src/bench/bench_bitcoin.cpp

src/bench/bench.cpp

furszy · 2024-10-03T19:21:42Z

Thanks for the review @hodlinator. Updated per feedback.

Good to see these globals implemented for bench. As already mentioned, I've been working on a partially overlapping change in #30737.

Cool. It wasn't on my radar. Adding it now.

hodlinator

git range-diff master 1ba225c ded1a6c

Still feel like it would be good to call a common function adding -testdatadir to an ArgsManager instead of duplicating the functionality between test/bench (and having a different description-string).

src/bench/bench_bitcoin.cpp

hodlinator · 2024-10-05T22:22:13Z

src/bench/bench_bitcoin.cpp

@@ -60,6 +61,17 @@ static uint8_t parsePriorityLevel(const std::string& str) {
    return levels;
 }

+// Parses test setup related arguments
+static std::vector<std::string> parseTestSetupArgs(const ArgsManager& argsman) {


I don't feel strong about the capitalization neither but it looks incorrect to introduce it now when all other functions aren't using it.

(SetupBenchArgs is capitalized, main must be non-capitalized, so that only leaves parseAsymptote to change beyond the function under discussion).

src/bench/bench_bitcoin.cpp

pablomartin4btc

tACK ded1a6c

Tested on Ubuntu 22.04 locally and using an external USB.

./build/src/bench/bench_bitcoin -filter=Xor -testdatadir=/tmp/btc
Warning, results might be unstable:
* DEBUG defined
* CPU frequency scaling enabled: CPU 0 between 400.0 and 4,700.0 MHz
* CPU governor is 'powersave' but should be 'performance'
* Turbo is enabled, CPU frequency will fluctuate

Recommendations
* Make sure you compile for Release
* Use 'pyperf system tune' before benchmarking. See https://github.com/psf/pyperf

|             ns/byte |              byte/s |    err% |     total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
|                9.24 |      108,266,754.90 |    1.5% |      0.01 | `Xor`


./build/src/bench/bench_bitcoin -filter=Xor -testdatadir=/media/pablo/USB\ DISK/tmp/btc
Warning, results might be unstable:
* DEBUG defined
* CPU frequency scaling enabled: CPU 0 between 400.0 and 4,700.0 MHz
* CPU governor is 'powersave' but should be 'performance'
* Turbo is enabled, CPU frequency will fluctuate

Recommendations
* Make sure you compile for Release
* Use 'pyperf system tune' before benchmarking. See https://github.com/psf/pyperf

|             ns/byte |              byte/s |    err% |     total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
|               11.52 |       86,792,532.79 |   28.6% |      0.01 | :wavy_dash: `Xor` (Unstable with ~45.2 iters. Increase `minEpochIterations` to e.g. 452)

I think it's a nice feature, even this can be achieved by changing the env var (e.g. TMPDIR=/my_temp ./build/src/bench/bench_bitcoin), using -testdatadir feels more consistent with other binaries use cases.

tdb3

Approach ACK
Useful improvement.

Did a quick sanity check for now (ramdisk at /mnt/tmp/).
Planning to circle back.

$ build/src/bench/bench_bitcoin -testdatadir=/mnt/tmp/
$ ls /mnt/tmp/test_common\ bitcoin
AssembleBlock                 DuplicateInputs        MempoolCheck              WalletBalanceDirty                             WalletIsMineDescriptors
BlockAssemblerAddPackageTxns  LoadExternalBlockFile  MempoolEviction           WalletBalanceMine                              WalletIsMineMigratedDescriptors
BlockFilterIndexSync          LogWithDebug           ReadBlockFromDiskTest     WalletBalanceWatch                             WalletLoadingDescriptors
BlockToJsonVerbose            LogWithoutDebug        ReadRawBlockFromDiskTest  WalletCreateEncrypted
BlockToJsonVerboseWrite       LogWithoutThreadNames  RpcMempool                WalletCreatePlain
CheckBlockIndex               LogWithoutWriteToFile  WalletAvailableCoins      WalletCreateTxUseOnlyPresetInputs
ComplexMemPool                LogWithThreadNames     WalletBalanceClean        WalletCreateTxUsePresetInputsAndCoinSelection

l0rinc · 2024-10-15T09:08:47Z

src/bench/bench_bitcoin.cpp

@@ -27,6 +28,7 @@ static const std::string DEFAULT_PRIORITY{"all"};
 static void SetupBenchArgs(ArgsManager& argsman)
 {
    SetupHelpOptions(argsman);
+    SetupCommonTestArgs(argsman);


I quickly glanced over this PR only, do I understand it correctly that we're treating benchmarks as tests here (I'm not sure I agree with that concept) and adding parameters to the benchmark setup that only apply to a smaller subset of the benchmarks?
Could we configure them via env variables instead and call the benchmark with e.g. TEST_DATA_DIR=a/b/c bench instead?
As an example, if a few of our tests require a for example a timestamp for whatever reason, we wouldn't add it to the testing framework as an additional parameter, right?
So if my understanding is correct, I'm leaning towards a concept NACK - please let me know if I misunderstood it.

I quickly glanced over this PR only, do I understand it correctly that we're treating benchmarks as tests here (I'm not sure I agree with that concept)

I'm not sure this is the best place for the conversation, but generally speaking, benchmarks are a form of testing. The difference is that they evaluate performance against previous runs rather than behavior.

adding parameters to the benchmark setup that only apply to a smaller subset of the benchmarks?

This applies to every benchmark requiring a node context. We currently have 33 of them.

Could we configure them via env variables instead and call the benchmark with e.g. TEST_DATA_DIR=a/b/c bench instead?

Please check the conversation above #31000 (comment).

As an example, if a few of our tests require a for example a timestamp for whatever reason, we wouldn't add it to the testing framework as an additional parameter, right?

Unlike the hardware device type the benchmark runs on, which can't be standardized for all users, software level variables like timestamps can be set in a general manner if needed. I don't think there will be many args like this one.

So if my understanding is correct, I'm leaning towards a concept NACK - please let me know if I misunderstood it.

I think you understood it. Just have a different opinion.

I did see that conversation, but I don't think it's the benchmark's responsibility to reveal the internals of a certain group of tests.

hodlinator

ACK 15cfeeb

Concept

Thanks @furszy for making the dependency explicit! I don't fully understand the upper/lower level argument as bench_bitcoin has a link dependency on test_util already.

Using ENV-vars instead as suggested in other comment seems too sneaky for me, and contradicts my goal of having things explicit. (IMO one original sin of C was main not taking environment args as a parameter). 31 tests are using BasicTestingSetup so I think it's common enough to be supported at a benchmark executor level.

Tested

₿ build/src/bench/bench_bitcoin -testdatadir=foo/
...
Test directory (will not be deleted): "/home/hodlinator/bitcoin/foo/test_common bitcoin/WalletIsMineDescriptors/datadir"
...

₿ ls foo/test_common\ bitcoin/
AssembleBlock                 ComplexMemPool         LogWithoutWriteToFile     RpcMempool            WalletCreateEncrypted                          WalletLoadingDescriptors
BlockAssemblerAddPackageTxns  DuplicateInputs        LogWithThreadNames        WalletAvailableCoins  WalletCreatePlain
BlockFilterIndexSync          LoadExternalBlockFile  MempoolCheck              WalletBalanceClean    WalletCreateTxUseOnlyPresetInputs
BlockToJsonVerbose            LogWithDebug           MempoolEviction           WalletBalanceDirty    WalletCreateTxUsePresetInputsAndCoinSelection
BlockToJsonVerboseWrite       LogWithoutDebug        ReadBlockFromDiskTest     WalletBalanceMine     WalletIsMineDescriptors
CheckBlockIndex               LogWithoutThreadNames  ReadRawBlockFromDiskTest  WalletBalanceWatch    WalletIsMineMigratedDescriptors

maflcko · 2024-10-16T15:02:06Z

lgtm ACK 15cfeeb

pablomartin4btc

re-ACK 15cfeeb

Changes since my last review: removing the -testdatadir arg from bench-bitcoin and reusing it from the utils setup_common.

achow101 · 2024-10-23T21:02:39Z

In 221410b "bench: specialize working directory name" , the commit message says

After this commit, benchmarks are contained within its own directory:
/<OS_tmp_dir>/test_common bitcoin/<benchmark_name>/<random_uint256>/

However I am not observing this behavior. It omits the benchmark name.

The benchmark name is included when -testdatadir is used.

furszy · 2024-10-23T21:34:48Z

In 221410b "bench: specialize working directory name" , the commit message says

After this commit, benchmarks are contained within its own directory:
/<OS_tmp_dir>/test_common bitcoin/<benchmark_name>/<random_uint256>/

However I am not observing this behavior. It omits the benchmark name.

The benchmark name is included when -testdatadir is used.

Yeah, that's because the data directory is erased after execution when -testdatadir isn't used. And the same applies to unit tests. That's why I didn't change it, though I could for consistency, making it easier to find the test data directory in case of failure - which is when the directory is not erased.

tdb3

re ACK b842ad6

Liking the use of time rather than random.

Re-ran #31000 (review)

'tmp/test_common bitcoin/BlockAssemblerAddPackageTxns/1731173028231274080'

src/test/util/setup_common.cpp

tdb3

re ACK 5adb4a4

Change since last push is a simple removal of a semi-accurate comment.

maflcko · 2024-11-11T10:21:06Z

src/test/util/setup_common.cpp

+        const auto rand_str{util::ToString(GetTime<std::chrono::nanoseconds>().count())};
+        m_path_root = fs::temp_directory_path() / TEST_DIR_PATH_ELEMENT / test_name / rand_str;


I don't think it is right to use mockable time here. Also, GetTime is deprecated, so what about TicksSinceEpoch<std::chrono::nanoseconds>(SystemClock::now())?

Also, I think it would be better to switch test_name / rand_str to rand_str / test_name, so that all unit tests from the same time are bundled together. This would also be identical to the functional test runner.

I don't think it is right to use mockable time here. Also, GetTime is deprecated, so what about TicksSinceEpochstd::chrono::nanoseconds(SystemClock::now())?

Yeah sure. This is usually called before mocking the time, but agree to change it.

Also, I think it would be better to switch test_name / rand_str to rand_str / test_name, so that all unit tests from the same time are bundled together. This would also be identical to the functional test runner.

That won’t work as you expect. Think of each unit test as a separate child class inheriting from the testing setup class and implementing a run() method. Each one constructs a new instance of the setup class, thereby creating a new rand_str.

I think we might be able to switch to time / test_name, creating a global testing class with a different Boost framework hook, but I’d prefer to leave that for a follow-up since it’s somewhat unrelated to this PR and will require additional investigation.

Done in #31291

maflcko · 2024-11-11T15:33:39Z

re-ACK 6c7b3bf

tdb3

code review and light test re ACK 6c7b3bf

build/src/bench/bench_bitcoin:
...
/tmp/test_common bitcoin/BlockAssemblerAddPackageTxns/1731341243673972130/regtest/blocks

Update uses TicksSinceEpoch with SystemClock::now() instead of deprecated GetTime().

src/test/util/setup_common.cpp

Since G_TEST_GET_FULL_NAME is not initialized in the benchmark framework, benchmarks using the unit test setup run in the same directory without any clear distinction between them. This poses an extra complication for locating any specific benchmark directory during a failure. In master, unit tests and benchmarks run in the following path: /<OS_tmp_dir>/test_common bitcoin/<random_uint256>/ After this commit, unit tests and benchmarks are contained within its own directory: /<OS_tmp_dir>/test_common bitcoin/<test_name>/<time_in_nanoseconds>/ This makes it easier to find any benchmark run when a failure occurs.

Expands the benchmark framework with the existing '-testdatadir' arg, enabling the ability to change the benchmark data directory. This is useful for running benchmarks on different storage devices, and not just under the OS /tmp/ directory.

furszy · 2024-11-11T16:36:03Z

Updated per feedback. Thanks. Removed unused global g_rng_temp_path from the setup common file.

tdb3

re ACK fa66e08

hodlinator

re-ACK fa66e08

git range-diff master 80d35bc fa66e08

Decimal time since epoch replacing random 256-bit value

Sample of recent now-value is 1731341566994144855.

According to https://www.epochconverter.com/ we won't get another digit until year 2286, so path lengths seem quite stable.

Switching to hex representation does give us constant lengths from now until unsigned 64-bit max, but only shaves off 3 chars, which is not that big of a win.

Determinism

The paths were made more deterministic in 97e16f5, but I suspect it's not that much of a priority.

I tried to follow through on the determinism intent in my #30737 but the approach in your PR should also fix #30696 (might be worth mentioning that you're fixing that issue as a bonus in the PR summary! :) ).

`TEST_DIR_PATH_ELEMENT`

Could potentially still rename TEST_DIR_PATH_ELEMENT -> TESTS_DIR_NAME as previously mentioned.

Testing

Ran build/src/test/test_bitcoin and lsof |grep "/tmp.*bitcoin" in parallel.

maflcko · 2024-11-13T15:46:40Z

re-ACK fa66e08

Only change is removing unused g_rng_temp_path

pablomartin4btc

re-ACK fa66e08

Since my last review: removal of g_rng_temp_path, renaming and moving definition of SetupCommonTestArgs to src/test/util/setup_common.h and using the timestamp for the test dir path instead of a random id.

achow101 · 2024-11-13T23:45:02Z

ACK fa66e08

…158303fe2 48158303fe2 kernel: Add pure kernel bitcoin-chainstate bf80d2f5009 kernel: Add block index utility functions to C header a6ab5345e3b kernel: Add function to read block undo data from disk to C header 845b824d6c7 kernel: Add functions to read block from disk to C header 9324c8c4f67 kernel: Add function for copying block data to C header 368fc93fd80 kernel: Add functions for the block validation state to C header eb6e25ac007 kernel: Add validation interface to C header cdce4484005 kernel: Add interrupt function to C header 7e47ec78768 kernel: Add import blocks function to C header 2b803d50747 kernel: Add chainstate load options for in-memory dbs in C header ea92eb13c4a kernel: Add options for reindexing in C header 8254f2035a7 kernel: Add block validation to C header ad7b880346e Kernel: Add chainstate loading to kernel C header 583820c4487 kernel: Add chainstate manager object to C header ec137a086a0 kernel: Add notifications context option to C header 62a89689266 kerenl: Add chain params context option to C header bb482dcbd30 kernel: Add kernel library context object d114ccfdf8a kernel: Add logging to kernel library C header 44c65c46c43 kernel: Introduce initial kernel C header API 69c03134440 Merge bitcoin/bitcoin#31269: validation: Remove RECENT_CONSENSUS_CHANGE validation result 42282592943 Merge bitcoin/bitcoin#31000: bench: add support for custom data directory 36f5effa178 Merge bitcoin/bitcoin#31235: addrman: cap the `max_pct` to not exceed the maximum number of addresses 98ad249b69f Merge bitcoin/bitcoin#31277: doc: mention `descriptorprocesspsbt` in psbt.md b0222bbb494 Merge bitcoin/bitcoin#30239: Ephemeral Dust 1dda1892b6b Merge bitcoin/bitcoin#31037: test: enhance p2p_orphan_handling 5c2e291060c bench: Add basic CheckEphemeralSpends benchmark 3f6559fa581 Add release note for ephemeral dust 71a6ab4b33d test: unit test for CheckEphemeralSpends 21d28b2f362 fuzz: add ephemeral_package_eval harness 127719f516a test: Add CheckMempoolEphemeralInvariants e2e30e89ba4 functional test: Add ephemeral dust tests 4e68f901390 rpc: disallow in-mempool prioritisation of dusty tx e1d3e81ab4d policy: Allow dust in transactions, spent in-mempool 04b2714fbbc functional test: Add new -dustrelayfee=0 test case ebb6cd82baf doc: mention `descriptorprocesspsbt` in psbt.md 2b33322169b Merge bitcoin/bitcoin#31249: test: Add combinerawtransaction test to rpc_createmultisig 3fb6229dcfd Merge bitcoin/bitcoin#31271: doc: correct typos fa66e0887ca bench: add support for custom data directory ad9c2cceda9 test, bench: specialize working directory name 9c5775c331e addrman: cap the `max_pct` to not exceed the maximum number of addresses 8d340be9247 Merge bitcoin/bitcoin#31181: cmake: Revamp `FindLibevent` module 9a8e5adb161 Merge bitcoin/bitcoin#31267: refactor: Drop deprecated space in operator""_mst 726cbee9553 doc: correct typos 9fdfb73ca84 doc: fix typos af6088701a2 Merge bitcoin/bitcoin#31237: doc: Add missing 'blank=true' option in offline-signing-tutorial.md 7a526653022 Merge bitcoin/bitcoin#31239: test: clarify log messages when handling SOCKS5 proxy connections 900b17239fb Merge bitcoin/bitcoin#31259: doc: Fix missing comma in JSON example in REST-interface.md faf21625652 refactor: Drop deprecated space in operator""_mst c889890e4a3 Merge bitcoin/bitcoin#31264: doc: Fixup bitcoin-wallet manpage chain selection args 0f6d20e43f2 Merge bitcoin/bitcoin#31163: scripted-diff: get rid of remaining "command" terminology in protocol.{h,cpp} 5acd5e7f874 Merge bitcoin/bitcoin#31257: ci: make ctest stop on failure 19f277711eb Merge bitcoin/bitcoin#26593: tracing: Only prepare tracepoint arguments when actually tracing e80e4c6ff91 validation: Remove RECENT_CONSENSUS_CHANGE validation result fa729ab4a27 doc: Fixup bitcoin-wallet manpage chain selection args 5e3b444022c doc: Fix missing comma in JSON example in REST-interface.md 0903ce8dbc2 Merge bitcoin/bitcoin#30592: Remove mempoolfullrbf f842d0801e1 Merge bitcoin/bitcoin#29686: Update manpage descriptions 36a22e56833 ci: make ctest stop on failure 83fab3212c9 test: Add combinerawtransaction test to rpc_createmultisig 018e5fcc462 Merge bitcoin/bitcoin#31190: TxDownloadManager followups 3a5f6027e16 Merge bitcoin/bitcoin#31171: depends: Specify CMake generator explicitly 99d9a093cf6 test: clarify log messages when handling SOCKS5 proxy connections c9e67e214f0 Merge bitcoin/bitcoin#31238: fuzz: Limit wallet_notifications iterations 564238aabf1 Merge bitcoin/bitcoin#31164: net: Use actual memory size in receive buffer accounting fa461d7a43a fuzz: Limit wallet_notifications iterations ec375de39ff doc: Add missing 'blank=true' option in offline-signing-tutorial.md 5a96767e3f5 depends, libevent: Do not install *.pc files and remove patches for them ffda355b5a2 cmake, refactor: Move `HAVE_EVHTTP_...` to `libevent` interface b619bdc3303 cmake: Revamp `FindLibevent` module 2c90f8e08c4 Merge bitcoin/bitcoin#31232: ci: `add second_deadlock_stack=1` to TSAN options 5dc94d13d41 fuzz fix: assert MAX_PEER_TX_ANNOUNCEMENTS is not exceeded 45e2f8f87d8 Merge bitcoin/bitcoin#31173: cmake: Add `FindQRencode` module and enable `libqrencode` package for MSVC 80cb630bd94 Merge bitcoin/bitcoin#31216: Update secp256k1 subtree to v0.6.0 5161c2618cd ci: add second_deadlock_stack=1 to TSAN options 85224f92d52 Merge bitcoin/bitcoin#30811: build: Unify `-logsourcelocations` format 9719d373dc2 Merge bitcoin/bitcoin#30634: ci: Use clang-19 from apt.llvm.org 97235c446e9 build: Disable secp256k1 musig module 9e5089dbb02 build, msvc: Enable `libqrencode` vcpkg package 30089b0cb61 cmake: Add `FindQRencode` module 65b19419366 Merge bitcoin/bitcoin#31186: msvc: Update vcpkg manifest d3388720837 Merge bitcoin/bitcoin#31206: doc: Use relative hyperlinks in release-process.md ffc05fca6f7 Merge bitcoin/bitcoin#31220: doc: Fix word order in developer-notes.md 9f2c8287a24 Merge bitcoin/bitcoin#31192: depends, doc: List packages required to build `qt` package separately 03cff2c1421 Merge bitcoin/bitcoin#31191: build: Make G_FUZZING constexpr, require -DBUILD_FOR_FUZZING=ON to fuzz 44939e5de1b doc: Fix word order in developer-notes.md b934954ad10 Merge bitcoin/bitcoin#30670: doc: Extend developer-notes with file-name-only debugging fix 05aebe3790f Merge bitcoin/bitcoin#30930: netinfo: add peer services column and outbound-only option 0ba680d41b4 Update secp256k1 subtree to v0.6.0 2d46a89386d Squashed 'src/secp256k1/' changes from 2f2ccc46954..0cdc758a563 d22a234ed27 net: Use actual memory size in receive buffer accounting 047b5e2af1f streams: add DataStream::GetMemoryUsage c3a6722f34a net: Use DynamicUsage(m_type) in CSerializedNetMsg::GetMemoryUsage c6594c0b142 memusage: Add DynamicUsage for std::string 7596282a556 memusage: Allow counting usage of vectors with different allocators 6463117a292 Merge bitcoin/bitcoin#31208: doc: archive release notes for v27.2 788c1324f3d build: Unify `-logsourcelocations` format 4747f030956 depends, doc: List packages required to build `qt` package separately 1a05c86ae47 doc: archive release notes for v27.2 9f71cff6ab3 doc: Use relative hyperlinks in release-process.md f1bcf3edc50 Merge bitcoin/bitcoin#31139: test: added test to assert TX decode rpc error on submitpackage rpc 975b115e1a2 Merge bitcoin/bitcoin#31198: init: warn, don't error, when '-upnp' is set 4a0251c05dd Merge bitcoin/bitcoin#31187: ci: Do not error on unused-member-function in test each commit e001dc3dc6e Merge bitcoin/bitcoin#31203: fuzz: fix `implicit-integer-sign-change` in wallet_create_transaction 5a26cf7773e fuzz: fix `implicit-integer-sign-change` in wallet_create_transaction a1b3ccae4be init: warn, don't error, when '-upnp' is set c189eec848e doc: release note for mempoolrullrbf removal d47297c6aab rpc: Mark fullrbf and bip125-replaceable as deprecated 04a5dcee8ab docs: remove requirement to signal bip125 fafbf8acf41 Make G_FUZZING constexpr, require -DBUILD_FOR_FUZZING=ON to execute a fuzz target fae3cf0ffa6 ci: Temporarily disable macOS/Windows fuzz step f6577b71741 build, msvc: Update vcpkg manifest baseline 16e16013bfa build, msvc: Document `libevent` version pinning ec47cd2b508 build, msvc: Drop no longer needed `liblzma` version pinning 9a0734df5f1 build, msvc: Reorder keys in `vcpkg.json` 8351562bec6 [fuzz] allow negative time jumps in txdownloadman_impl 917ab810d93 [doc] comment fixups from n30110 f07a533dfcb Merge bitcoin/bitcoin#24214: Fix unsigned integer overflows in interpreter 62516105536 Merge bitcoin/bitcoin#31015: build: have "make test" depend on "make all" 4a31f8ccc9d Merge bitcoin/bitcoin#31156: test: Don't enforce BIP94 on regtest unless specified by arg 02be3dced71 Merge bitcoin/bitcoin#31166: key: clear out secret data in `DecodeExtKey` 54d07dd37d5 ci: Do not error on unused-member-function in test each commit 47f50c7af55 doc: add bitcoin-qt man description 40b82e3ab0a doc: add bitcoin-util man description a7bf80f3a2d doc: add bitcoin-tx man description 3f9a5168323 doc: add bitcoin-wallet man description d8c0bb23ef8 doc: add bitcoin-cli man description 09abccfa772 doc: add bitcoind man description 97b790e844a Merge bitcoin/bitcoin#29420: test: extend the SOCKS5 Python proxy to actually connect to a destination 6b73eb9a1a2 Merge bitcoin/bitcoin#31064: init: Correct coins db cache size setting 27d12cf17f2 Merge bitcoin/bitcoin#31043: rpc: getorphantxs follow-up 7b66815b16b Merge bitcoin/bitcoin#30110: refactor: TxDownloadManager + fuzzing dc97e7f6dba Merge bitcoin/bitcoin#30903: cmake: Add `FindZeroMQ` module 1b0b9b4c787 Extend possible debugging fixes with file-name-only da10e0bab4a Merge bitcoin/bitcoin#30942: test: Remove dead code from interface_zmq test 111a23d9b36 Remove -mempoolfullrbf option e96ffa98b04 Merge bitcoin/bitcoin#31142: test: fix intermittent failure in p2p_seednode.py, don't connect to random IPs 54c4b09f083 Merge bitcoin/bitcoin#31042: build: Rename `PACKAGE_*` variables to `CLIENT_*` e60cecc8115 doc: add release note for 31156 fc7dfb3df5b test: Don't enforce BIP94 on regtest unless specified by arg fabe90c8242 ci: Use clang-19 from apt.llvm.org 0de3e96e333 tracing: use bitcoind pid in bcc tracing examples 411c6cfc6c2 tracing: only prepare tracepoint args if attached d524c1ec066 tracing: dedup TRACE macros & rename to TRACEPOINT 70713303b63 scripted-diff: Rename `PACKAGE_*` variables to `CLIENT_*` 332655cb52c build: Rename `PACKAGE_*` variables to `CLIENT_*` e6e29e3c94c scripted-diff: Clarify "user agent" variable name e2ba8236715 depends: Specify CMake generator explicitly 1c7ca6e64de Merge bitcoin/bitcoin#31093: Introduce `g_fuzzing` global for fuzzing checks 6e21dedbf2b Merge bitcoin/bitcoin#31130: Drop miniupnp dependency d7fd766feb2 test: added test to assert TX decode rpc error on submitpackage rpc 559a8dd9c0a key: clear out secret data in `DecodeExtKey` 4120c7543ee scripted-diff: get rid of remaining "command" terminology in protocol.{h,cpp} 2a52718d734 Merge bitcoin/bitcoin#31152: functional test: Additional package evaluation coverage 9de9c858d5a test: enhance p2p_orphan_handling 33af14b62e4 test: reduce assert_debug_log reliance 0ea84bc362f test: explicitly check boolean verbosity is disallowed 7a2e6b68cd9 doc: add rpc guidance for boolean verbosity avoidance 698f302df8b rpc: disallow boolean verbosity in getorphantxs 63f5e6ec795 test: add entry and expiration time checks 808a708107e rpc: add entry time to getorphantxs 56bf3027144 refactor: rename rpc_getorphantxs to rpc_orphans 7824f6b0770 test: check that getorphantxs is hidden ac68fcca701 rpc: disallow undefined verbosity in getorphantxs 25dacae9c7f Merge bitcoin/bitcoin#31040: test: Assert that when we add the max orphan amount that we cannot add anymore and that a random orphan gets dropped 40e5f26a3ff mapport: remove dead code in DispatchMapPort 38fdf7c1fb1 mapport: drop outdated comments 915640e191b depends: zeromq: don't install .pc files and remove patches for them 6b8a74463b5 cmake: Add `FindZeroMQ` module 9a7206a34e3 Merge bitcoin/bitcoin#29536: fuzz: fuzz connman with non-empty addrman + ASMap d4abaf8c9d9 Merge bitcoin/bitcoin#29608: optimization: Preallocate addresses in GetAddr based on nNodes b7b24352906 doc: add release note for #31130 1b6dec98da3 depends: drop miniupnpc 953533d0214 doc: remove mentions of UPnP 94ad614482f ci: remove UPnP options f32c34d0c3d functional test: Additional package evaluation coverage 87532fe5585 netinfo: allow setting an outbound-only peer list 9f243cd7fa6 Introduce `g_fuzzing` global for fuzzing checks b95adf057a4 Merge bitcoin/bitcoin#31150: util: Treat Assume as Assert when evaluating at compile-time 8f24e492e20 Merge bitcoin/bitcoin#29991: depends: sqlite 3.46.1 2ef5004f78c Merge bitcoin/bitcoin#31146: ci: Temporary workaround for old CCACHE_DIR cirrus env 8c12fe828de Merge bitcoin/bitcoin#29936: fuzz: wallet: add target for `CreateTransaction` 5c299ecafe6 test: Assert that when we add the max orphan amount that we cannot add anymore and that a random orphan gets dropped 0f4bc635854 [fuzz] txdownloadman and txdownload_impl 699643f23a1 [unit test] MempoolRejectedTx fa584cbe727 [p2p] add TxDownloadOptions bool to make TxRequestTracker deterministic f803c8ce8dd [p2p] filter 1p1c for child txid in recent rejects 5269d57e6d7 [p2p] don't process orphan if in recent rejects 2266eba43a9 [p2p] don't find 1p1cs for reconsiderable txns that are AlreadyHaveTx fa7027d0fc1 [refactor] add CheckIsEmpty and GetOrphanTransactions, remove access to TxDownloadMan internals 969b07237b9 [refactor] wrap {Have,Get}TxToReconsider in txdownload f150fb94e7d [refactor] make AlreadyHaveTx and Find1P1CPackage private to TxDownloadImpl 1e08195135b [refactor] move new tx logic to txdownload 257568eab5b [refactor] move invalid package processing to TxDownload c4ce0c1218d [refactor] move invalid tx processing to TxDownload c6b21749ca0 [refactor] move valid tx processing to TxDownload a8cf3b6e845 [refactor] move Find1P1CPackage to txdownload f497414ce76 [refactor] put peerman tasks at the end of ProcessInvalidTx 6797bc42a76 [p2p] restrict RecursiveDynamicUsage of orphans added to vExtraTxnForCompact 798cc8f5aac [refactor] move Find1P1CPackage into ProcessInvalidTx 416fbc952b2 [refactor] move new orphan handling to ProcessInvalidTx c8e67b9169b [refactor] move ProcessInvalidTx and ProcessValidTx definitions down 3a41926d1b5 [refactor] move notfound processing to txdownload 042a97ce7fc [refactor] move tx inv/getdata handling to txdownload 58e09f244b4 [p2p] don't log tx invs when in IBD 288865338f5 [refactor] rename maybe_add_extra_compact_tx to first_time_failure f48d36cd97e [refactor] move peer (dis)connection logic to TxDownload f61d9e4b4b8 [refactor] move AlreadyHaveTx to TxDownload 84e4ef843db [txdownload] add read-only reference to mempool af918349de5 [refactor] move ValidationInterface functions to TxDownloadManager f6c860efb12 [doc] fix typo in m_lazy_recent_confirmed_transactions doc 5f9004e1550 [refactor] add TxDownloadManager wrapping TxOrphanage, TxRequestTracker, and bloom filters 947f2925d55 Merge bitcoin/bitcoin#31124: util: Remove RandAddSeedPerfmon 7640cfdd624 Merge bitcoin/bitcoin#31118: doc: replace `-?` with `-h` and `-help` 74fb19317ae Merge bitcoin/bitcoin#30849: refactor: migrate `bool GetCoin` to return `optional<Coin>` c16e909b3e2 Merge bitcoin/bitcoin#28574: wallet: optimize migration process, batch db transactions a9598e5eaab build: drop miniupnpc dependency a5fcfb7385c interfaces: remove now unused 'use_upnp' arg from 'mapPort' 038bbe7b200 daemon: remove UPnP support 844770b05eb qt: remove UPnP settings dd92911732d Merge bitcoin/bitcoin#31148: ci: display logs of failed unit tests automatically fa69a5f4b76 util: Treat Assume as Assert when evaluating at compile-time 0c79c343a9f Merge bitcoin/bitcoin#31147: cmake, qt, test: Remove problematic code 8523d8c0fc8 ci: display logs of failed tests automatically 2f40e453ccd Merge bitcoin/bitcoin#29450: build: replace custom `MAC_OSX` macro with existing `__APPLE__` cb7c5ca824e Add gdb and lldb links to debugging troubleshooting 6c6b2442eda build: Replace MAC_OSX macro with existing __APPLE__ fb46d57d4e7 cmake, qt, test: Remove problematic code fa9747a8961 ci: Temporary workaround for old CCACHE_DIR cirrus env 6c9fe7b73ea test: Prevent connection attempts to random IPs in p2p_seednodes.py bb97b1ffa9f test: fix intermittent timeout in p2p_seednodes.py 57529ac4dbb test: set P2PConnection.p2p_connected_to_node in peer_connect_helper() 22cd0e888c7 test: support WTX INVs from P2PDataStore and fix a comment ebe42c00aa4 test: extend the SOCKS5 Python proxy to actually connect to a destination 9bb92c0e7ff util: Remove RandAddSeedPerfmon c98fc36d094 wallet: migration, consolidate external wallets db writes 7c9076a2d2e wallet: migration, consolidate main wallet db writes 9ef20e86d7f wallet: provide WalletBatch to 'SetupDescriptorScriptPubKeyMans' 34bf0795fc0 wallet: refactor ApplyMigrationData to return util::Result<void> aacaaaa0d3a wallet: provide WalletBatch to 'RemoveTxs' 57249ff6697 wallet: introduce active db txn listeners 91e065ec175 wallet: remove post-migration signals connection 055c0532fc8 wallet: provide WalletBatch to 'DeleteRecords' 122d103ca22 wallet: introduce 'SetWalletFlagWithDB' 6052c7891dc wallet: decouple default descriptors creation from external signer setup f2541d09e13 wallet: batch MigrateToDescriptor() db transactions 66c9936455f bench: add coverage for wallet migration process 33a28e252a7 Change default help arg to `-help` and mention `-h` and `-?` as alternatives f0130ab1a1e doc: replace `-?` with `-h` for bench_bitcoin help 681ebcceca7 netinfo: rename and hoist max level constant to use in top-level help e7d307ce8cf netinfo: clarify relaytxes and addr_relay_enabled help docs eef2a9d4062 netinfo: add peer services column 3a4a788ee0d init: Correct coins db cache size setting 2957ca96119 build: have "make test" depend on "make all" bbbbaa0d9ac Fix unsigned integer overflows in interpreter c4dc81f9c69 test: Remove dead code from interface_zmq c495731a316 fuzz: wallet: add target for `CreateTransaction` 3db68e29ec6 wallet: move `ImportDescriptors`/`FuzzedWallet` to util 552cae243a1 fuzz: cover `ASMapHealthCheck` in connman target 33b0f3ae966 fuzz: use `ConsumeNetGroupManager` in connman target 18c8a0945bd fuzz: move `ConsumeNetGroupManager` to util fe624631aeb fuzz: fuzz `connman` with a non-empty addrman 0a12cff2a8e fuzz: move `AddrManDeterministic` to util 4feaa287284 refactor: Rely on returned value of GetCoin instead of parameter 46dfbf169b4 refactor: Return optional of Coin in GetCoin e31bfb26c21 refactor: Remove unrealistic simulation state ba621ffb9cb test: improve debug log message from P2PConnection::connection_made() def6dd0c597 depends: sqlite 3.46.1 66082ca3488 Preallocate addresses in GetAddr based on nNodes REVERT: 1047757ea3b kernel: Add pure kernel bitcoin-chainstate REVERT: c568fdf75fd kernel: Add block index utility functions to C header REVERT: 0f1da1dcba5 kernel: Add function to read block undo data from disk to C header REVERT: 45af559c9f6 kernel: Add functions to read block from disk to C header REVERT: 2a7f8a8240c kernel: Add function for copying block data to C header REVERT: b19f5336c03 kernel: Add functions for the block validation state to C header REVERT: 9c0ffa913f4 kernel: Add validation interface to C header REVERT: a93318c6152 kernel: Add interrupt function to C header REVERT: 51053f33720 kernel: Add import blocks function to C header REVERT: 6b0ada2af42 kernel: Add chainstate load options for in-memory dbs in C header REVERT: 34427bfa9c7 kernel: Add options for reindexing in C header REVERT: ca57311c969 kernel: Add block validation to C header REVERT: 44156d84838 Kernel: Add chainstate loading to kernel C header REVERT: 2cee46cdcc1 kernel: Add chainstate manager object to C header REVERT: 7102c7ae45e kernel: Add notifications context option to C header REVERT: ed628a2a3c4 kerenl: Add chain params context option to C header REVERT: 27643297ff7 kernel: Add kernel library context object REVERT: 2ba22cf3f90 kernel: Add logging to kernel library C header REVERT: 873874c03e9 kernel: Introduce initial kernel C header API git-subtree-dir: libbitcoinkernel-sys/bitcoin git-subtree-split: 48158303fe276cb2f8fbc53ff31a4162d8f55c84

faaaf59 test: Make g_rng_temp_path rand, not dependent on SeedRandomForTest (MarcoFalke) fa80b08 test: Revert to random path element (MarcoFalke) Pull request description: The randomness in the path element is required to allow a single fuzz test to run in parallel. Previous releases used a uint256 random value, but 10 random bytes should be sufficient as well, while avoiding a `MAX_PATH` violation on Windows. The issue was introduced by myself, by suggesting to use the current time in #31000 (comment). ACKs for top commit: kevkevinpal: reACK faaaf59 hodlinator: ACK faaaf59 tdb3: re ACK faaaf59 dergoegge: ACK faaaf59 Tree-SHA512: f12256c8b353618291030f71bf36eab97a25ffeaa28e36a5f2c6718dfc1fbbc8548c71475edec53d59026f2a779a05778db83f0530dd3e1d1faf6e4fc0ee7d70

DrahtBot added Tests CI failed labels Sep 29, 2024

furszy force-pushed the 2024_bench_custom_datadir branch 2 times, most recently from fb5be46 to 1ba225c Compare September 30, 2024 02:29

DrahtBot removed the CI failed label Sep 30, 2024

EricMujjona approved these changes Sep 30, 2024

View reviewed changes

furszy mentioned this pull request Sep 30, 2024

wallet: optimize migration process, batch db transactions #28574

Merged

maflcko mentioned this pull request Oct 2, 2024

test: Fix RANDOM_CTX_SEED use with parallel tests #30737

Closed

hodlinator reviewed Oct 3, 2024

View reviewed changes

furszy force-pushed the 2024_bench_custom_datadir branch from 1ba225c to ded1a6c Compare October 3, 2024 19:18

hodlinator reviewed Oct 5, 2024

View reviewed changes

pablomartin4btc reviewed Oct 9, 2024

View reviewed changes

DrahtBot requested a review from hodlinator October 9, 2024 02:59

tdb3 reviewed Oct 12, 2024

View reviewed changes

furszy force-pushed the 2024_bench_custom_datadir branch from ded1a6c to 15cfeeb Compare October 13, 2024 16:11

l0rinc reviewed Oct 15, 2024

View reviewed changes

hodlinator approved these changes Oct 15, 2024

View reviewed changes

DrahtBot requested review from tdb3 and pablomartin4btc October 15, 2024 09:52

pablomartin4btc approved these changes Oct 16, 2024

View reviewed changes

DrahtBot requested a review from tdb3 November 9, 2024 12:26

furszy force-pushed the 2024_bench_custom_datadir branch from 4bbcc70 to b842ad6 Compare November 9, 2024 16:30

tdb3 approved these changes Nov 9, 2024

View reviewed changes

src/test/util/setup_common.cpp Outdated Show resolved Hide resolved

DrahtBot requested a review from maflcko November 9, 2024 17:26

furszy force-pushed the 2024_bench_custom_datadir branch from b842ad6 to 5adb4a4 Compare November 9, 2024 17:31

tdb3 approved these changes Nov 9, 2024

View reviewed changes

maflcko reviewed Nov 11, 2024

View reviewed changes

DrahtBot requested a review from maflcko November 11, 2024 10:21

furszy force-pushed the 2024_bench_custom_datadir branch from 5adb4a4 to 6c7b3bf Compare November 11, 2024 15:25

DrahtBot requested a review from tdb3 November 11, 2024 15:33

tdb3 approved these changes Nov 11, 2024

View reviewed changes

hodlinator suggested changes Nov 11, 2024

View reviewed changes

src/test/util/setup_common.cpp Show resolved Hide resolved

src/test/util/setup_common.cpp Show resolved Hide resolved

DrahtBot requested a review from hodlinator November 11, 2024 16:21

furszy added 2 commits November 11, 2024 11:31

bench: add support for custom data directory

fa66e08

Expands the benchmark framework with the existing '-testdatadir' arg, enabling the ability to change the benchmark data directory. This is useful for running benchmarks on different storage devices, and not just under the OS /tmp/ directory.

furszy force-pushed the 2024_bench_custom_datadir branch from 6c7b3bf to fa66e08 Compare November 11, 2024 16:32

tdb3 approved these changes Nov 11, 2024

View reviewed changes

hodlinator approved these changes Nov 11, 2024

View reviewed changes

DrahtBot mentioned this pull request Nov 13, 2024

scripted-diff: Type-safe settings retrieval #31260

Open

pablomartin4btc approved these changes Nov 13, 2024

View reviewed changes

achow101 merged commit 4228259 into bitcoin:master Nov 14, 2024
16 checks passed

furszy deleted the 2024_bench_custom_datadir branch November 14, 2024 02:22

furszy mentioned this pull request Nov 14, 2024

test: group executed tests within the same directory #31291

Closed

maflcko mentioned this pull request Nov 18, 2024

test: Revert to random path element #31317

Merged

	static std::vector<std::string> parseTestSetupArgs(const ArgsManager& argsman) {
	static std::vector<std::string> ParseTestSetupArgs(const ArgsManager& argsman)
	{

		const auto rand_str{util::ToString(GetTime<std::chrono::nanoseconds>().count())};
		m_path_root = fs::temp_directory_path() / TEST_DIR_PATH_ELEMENT / test_name / rand_str;

bench: add support for custom data directory #31000

bench: add support for custom data directory #31000

Conversation

furszy commented Sep 29, 2024

DrahtBot commented Sep 29, 2024 • edited Loading

Code Coverage & Benchmarks

Reviews

Conflicts

DrahtBot commented Sep 29, 2024

maflcko commented Sep 30, 2024

EricMujjona left a comment

Choose a reason for hiding this comment

furszy commented Sep 30, 2024 • edited Loading

maflcko commented Oct 1, 2024

furszy commented Oct 1, 2024 • edited Loading

hodlinator left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

furszy commented Oct 3, 2024

hodlinator left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pablomartin4btc left a comment

Choose a reason for hiding this comment

tdb3 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

furszy Oct 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hodlinator left a comment

Choose a reason for hiding this comment

Concept

Tested

maflcko commented Oct 16, 2024

pablomartin4btc left a comment

Choose a reason for hiding this comment

achow101 commented Oct 23, 2024

furszy commented Oct 23, 2024

tdb3 left a comment

Choose a reason for hiding this comment

tdb3 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

furszy Nov 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maflcko commented Nov 11, 2024

tdb3 left a comment • edited Loading

Choose a reason for hiding this comment

furszy commented Nov 11, 2024

tdb3 left a comment

Choose a reason for hiding this comment

hodlinator left a comment

Choose a reason for hiding this comment

Decimal time since epoch replacing random 256-bit value

Determinism

TEST_DIR_PATH_ELEMENT

Testing

maflcko commented Nov 13, 2024

pablomartin4btc left a comment

Choose a reason for hiding this comment

achow101 commented Nov 13, 2024

DrahtBot commented Sep 29, 2024 •

edited

Loading

furszy commented Sep 30, 2024 •

edited

Loading

furszy commented Oct 1, 2024 •

edited

Loading

furszy Oct 15, 2024 •

edited

Loading

furszy Nov 11, 2024 •

edited

Loading

tdb3 left a comment •

edited

Loading

`TEST_DIR_PATH_ELEMENT`