Expose additional builder booster related flags in the vc (sigp#5086)

* expose builder booster flags in vc, enable options in validator endpoints, update tests

* resolve failing test

* fix issues related to CreateConfig and MoveConfig

* remove unneeded val, change how boost factor flag logic in the vc, add some additional documentation

* fix typos

* fix typos

* assume builder-proosals flag if one of other two vc builder flags are present

* fmt

* typo

* typo

* Fix CLI help text

* Prioritise per validator builder boost configurations over CLI flags.

* Add http test for builder boost factor with process defaults.

* Fix issue with PATCH request

* Add prefer builder proposals

* Add more builder boost factor tests.

---------

Co-authored-by: Mac L <mjladson@pm.me>
Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
Co-authored-by: Paul Hauner <paul@paulhauner.com>

account_manager/src/validator/import.rs

@@ -284,6 +284,8 @@ pub fn cli_run(matches: &ArgMatches, validator_dir: PathBuf) -> Result<(), Strin
             suggested_fee_recipient,
             None,
             None,
+            None,
+            None,
         )
         .map_err(|e| format!("Unable to create new validator definition: {:?}", e))?;
 

book/src/api-vc-endpoints.md

@@ -427,7 +427,7 @@ Example Response Body
 
 ## `PATCH /lighthouse/validators/:voting_pubkey`
 
-Update some values for the validator with `voting_pubkey`. Possible fields: `enabled`, `gas_limit`, `builder_proposals`, 
+Update some values for the validator with `voting_pubkey`. Possible fields: `enabled`, `gas_limit`, `builder_proposals`, `builder_boost_factor`, `prefer_builder_proposals`
 and `graffiti`.  The following example updates a validator from `enabled: true` to `enabled: false`.
 
 ### HTTP Specification

book/src/builders.md

@@ -31,6 +31,18 @@ blinded blocks, you should use the following flag:
 lighthouse vc --builder-proposals
 ```
 With the `--builder-proposals` flag, the validator client will ask for blinded blocks for all validators it manages.
+
+```
+lighthouse vc --prefer-builder-proposals
+```
+With the `--prefer-builder-proposals` flag, the validator client will always prefer blinded blocks, regardless of the payload value, for all validators it manages.
+
+```
+lighthouse vc --builder-boost-factor <INTEGER>
+```
+With the `--builder-boost-factor` flag, a percentage multiplier is applied to the builder's payload value when choosing between a
+builder payload header and payload from the paired execution node.
+
 In order to configure whether a validator queries for blinded blocks check out [this section.](#validator-client-configuration)
 
 ## Multiple builders
@@ -46,9 +58,9 @@ relays, run one of the following services and configure lighthouse to use it wit
 In the validator client you can configure gas limit and fee recipient on a per-validator basis. If no gas limit is
 configured, Lighthouse will use a default gas limit of 30,000,000, which is the current default value used in execution
 engines.  You can also enable or disable use of external builders on a per-validator basis rather than using
-`--builder-proposals`, which enables external builders for all validators. In order to manage these configurations
-per-validator, you can either make updates to the `validator_definitions.yml` file or you can use the HTTP requests
-described below.
+`--builder-proposals`, `--builder-boost-factor` or `--prefer-builder-proposals`, which apply builder related preferences for all validators.
+In order to manage these configurations per-validator, you can either make updates to the `validator_definitions.yml` file
+or you can use the HTTP requests described below.
 
 Both the gas limit and fee recipient will be passed along as suggestions to connected builders. If there is a discrepancy
 in either, it will *not* keep you from proposing a block with the builder. This is because the bounds on gas limit are

book/src/help_vc.md

@@ -56,6 +56,9 @@ FLAGS:
             machine. Note that logs can often contain sensitive information about your validator and so this flag should
             be used with caution. For Windows users, the log file permissions will be inherited from the parent folder.
         --metrics                                Enable the Prometheus metrics HTTP server. Disabled by default.
+        --prefer-builder-proposals
+            If this flag is set, Lighthouse will always prefer blocks constructed by builders, regardless of payload
+            value.
         --produce-block-v3
             Enable block production via the block v3 endpoint for this validator client. This should only be enabled
             when paired with a beacon node that has this endpoint implemented. This flag will be enabled by default in
@@ -80,6 +83,9 @@ OPTIONS:
             Comma-separated list of beacon API topics to broadcast to all beacon nodes. Possible values are: none,
             attestations, blocks, subscriptions, sync-committee. Default (when flag is omitted) is to broadcast
             subscriptions only.
+        --builder-boost-factor <UINT64>
+            Defines the boost factor, a percentage multiplier to apply to the builder's payload value when choosing
+            between a builder payload header and payload from the local execution node.
         --builder-registration-timestamp-override <builder-registration-timestamp-override>
             This flag takes a unix timestamp value that will be used to override the timestamp used in the builder api
             registration

book/src/help_vm_create.md

@@ -42,6 +42,9 @@ OPTIONS:
             A HTTP(S) address of a beacon node using the beacon-API. If this value is provided, an error will be raised
             if any validator key here is already known as a validator by that beacon node. This helps prevent the same
             validator being created twice and therefore slashable conditions.
+        --builder-boost-factor <UINT64>
+            Defines the boost factor, a percentage multiplier to apply to the builder's payload value when choosing
+            between a builder payload header and payload from the local execution node.
         --builder-proposals <builder-proposals>
             When provided, all created validators will attempt to create blocks via builder rather than the local EL.
             [possible values: true, false]
@@ -93,13 +96,18 @@ OPTIONS:
         --logfile-max-size <SIZE>
             The maximum size (in MB) each log file can grow to before rotating. If set to 0, background file logging is
             disabled. [default: 200]
-        --mnemonic-path <MNEMONIC_PATH>                         If present, the mnemonic will be read in from this file.
+        --mnemonic-path <MNEMONIC_PATH>
+            If present, the mnemonic will be read in from this file.
+
         --network <network>
             Name of the Eth2 chain Lighthouse will sync and follow. [possible values: mainnet, prater, goerli, gnosis,
             chiado, sepolia, holesky]
         --output-path <DIRECTORY>
             The path to a directory where the validator and (optionally) deposits files will be created. The directory
             will be created if it does not exist.
+        --prefer-builder-proposals <prefer-builder-proposals>
+            If this flag is set, Lighthouse will always prefer blocks constructed by builders, regardless of payload
+            value. [possible values: true, false]
         --safe-slots-to-import-optimistically <INTEGER>
             Used to coordinate manual overrides of the SAFE_SLOTS_TO_IMPORT_OPTIMISTICALLY parameter. This flag should
             only be used if the user has a clear understanding that the broad Ethereum community has elected to override

book/src/help_vm_move.md

@@ -26,10 +26,13 @@ FLAGS:
     -V, --version                        Prints version information
 
 OPTIONS:
+        --builder-boost-factor <UINT64>
+            Defines the boost factor, a percentage multiplier to apply to the builder's payload value when choosing
+            between a builder payload header and payload from the local execution node.
         --builder-proposals <builder-proposals>
             When provided, all created validators will attempt to create blocks via builder rather than the local EL.
             [possible values: true, false]
-        --count <VALIDATOR_COUNT>                               The number of validators to move.
+        --count <VALIDATOR_COUNT>                                The number of validators to move.
     -d, --datadir <DIR>
             Used to specify a custom root data directory for lighthouse keys and databases. Defaults to
             $HOME/.lighthouse/{network} where network is the value of the `network` flag Note: Users should specify
@@ -75,6 +78,9 @@ OPTIONS:
         --network <network>
             Name of the Eth2 chain Lighthouse will sync and follow. [possible values: mainnet, prater, goerli, gnosis,
             chiado, sepolia, holesky]
+        --prefer-builder-proposals <prefer-builder-proposals>
+            If this flag is set, Lighthouse will always prefer blocks constructed by builders, regardless of payload
+            value. [possible values: true, false]
         --safe-slots-to-import-optimistically <INTEGER>
             Used to coordinate manual overrides of the SAFE_SLOTS_TO_IMPORT_OPTIMISTICALLY parameter. This flag should
             only be used if the user has a clear understanding that the broad Ethereum community has elected to override

common/account_utils/src/validator_definitions.rs

@@ -157,6 +157,12 @@ pub struct ValidatorDefinition {
     #[serde(skip_serializing_if = "Option::is_none")]
     pub builder_proposals: Option<bool>,
     #[serde(default)]
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub builder_boost_factor: Option<u64>,
+    #[serde(default)]
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub prefer_builder_proposals: Option<bool>,
+    #[serde(default)]
     pub description: String,
     #[serde(flatten)]
     pub signing_definition: SigningDefinition,
@@ -169,13 +175,16 @@ impl ValidatorDefinition {
     /// ## Notes
     ///
     /// This function does not check the password against the keystore.
+    #[allow(clippy::too_many_arguments)]
     pub fn new_keystore_with_password<P: AsRef<Path>>(
         voting_keystore_path: P,
         voting_keystore_password_storage: PasswordStorage,
         graffiti: Option<GraffitiString>,
         suggested_fee_recipient: Option<Address>,
         gas_limit: Option<u64>,
         builder_proposals: Option<bool>,
+        builder_boost_factor: Option<u64>,
+        prefer_builder_proposals: Option<bool>,
     ) -> Result<Self, Error> {
         let voting_keystore_path = voting_keystore_path.as_ref().into();
         let keystore =
@@ -196,6 +205,8 @@ impl ValidatorDefinition {
             suggested_fee_recipient,
             gas_limit,
             builder_proposals,
+            builder_boost_factor,
+            prefer_builder_proposals,
             signing_definition: SigningDefinition::LocalKeystore {
                 voting_keystore_path,
                 voting_keystore_password_path,
@@ -344,6 +355,8 @@ impl ValidatorDefinitions {
                     suggested_fee_recipient: None,
                     gas_limit: None,
                     builder_proposals: None,
+                    builder_boost_factor: None,
+                    prefer_builder_proposals: None,
                     signing_definition: SigningDefinition::LocalKeystore {
                         voting_keystore_path,
                         voting_keystore_password_path,

common/eth2/src/lighthouse_vc/http_client.rs

@@ -483,12 +483,15 @@ impl ValidatorClientHttpClient {
     }
 
     /// `PATCH lighthouse/validators/{validator_pubkey}`
+    #[allow(clippy::too_many_arguments)]
     pub async fn patch_lighthouse_validators(
         &self,
         voting_pubkey: &PublicKeyBytes,
         enabled: Option<bool>,
         gas_limit: Option<u64>,
         builder_proposals: Option<bool>,
+        builder_boost_factor: Option<u64>,
+        prefer_builder_proposals: Option<bool>,
         graffiti: Option<GraffitiString>,
     ) -> Result<(), Error> {
         let mut path = self.server.full.clone();
@@ -505,6 +508,8 @@ impl ValidatorClientHttpClient {
                 enabled,
                 gas_limit,
                 builder_proposals,
+                builder_boost_factor,
+                prefer_builder_proposals,
                 graffiti,
             },
         )

common/eth2/src/lighthouse_vc/types.rs

@@ -32,6 +32,12 @@ pub struct ValidatorRequest {
     #[serde(default)]
     #[serde(skip_serializing_if = "Option::is_none")]
     pub builder_proposals: Option<bool>,
+    #[serde(default)]
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub builder_boost_factor: Option<u64>,
+    #[serde(default)]
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub prefer_builder_proposals: Option<bool>,
     #[serde(with = "serde_utils::quoted_u64")]
     pub deposit_gwei: u64,
 }
@@ -86,6 +92,12 @@ pub struct ValidatorPatchRequest {
     #[serde(default)]
     #[serde(skip_serializing_if = "Option::is_none")]
     pub graffiti: Option<GraffitiString>,
+    #[serde(default)]
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub builder_boost_factor: Option<u64>,
+    #[serde(default)]
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub prefer_builder_proposals: Option<bool>,
 }
 
 #[derive(Clone, PartialEq, Serialize, Deserialize)]
@@ -105,6 +117,12 @@ pub struct KeystoreValidatorsPostRequest {
     #[serde(default)]
     #[serde(skip_serializing_if = "Option::is_none")]
     pub builder_proposals: Option<bool>,
+    #[serde(default)]
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub builder_boost_factor: Option<u64>,
+    #[serde(default)]
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub prefer_builder_proposals: Option<bool>,
 }
 
 #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
@@ -135,6 +153,12 @@ pub struct Web3SignerValidatorRequest {
     pub client_identity_path: Option<PathBuf>,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub client_identity_password: Option<String>,
+    #[serde(default)]
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub builder_boost_factor: Option<u64>,
+    #[serde(default)]
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub prefer_builder_proposals: Option<bool>,
 }
 
 #[derive(Debug, Clone, PartialEq, Deserialize, Serialize)]

lighthouse/tests/account_manager.rs

@@ -492,6 +492,8 @@ fn validator_import_launchpad() {
         suggested_fee_recipient: None,
         gas_limit: None,
         builder_proposals: None,
+        builder_boost_factor: None,
+        prefer_builder_proposals: None,
         voting_public_key: keystore.public_key().unwrap(),
         signing_definition: SigningDefinition::LocalKeystore {
             voting_keystore_path,
@@ -614,6 +616,8 @@ fn validator_import_launchpad_no_password_then_add_password() {
         suggested_fee_recipient: None,
         gas_limit: None,
         builder_proposals: None,
+        builder_boost_factor: None,
+        prefer_builder_proposals: None,
         voting_public_key: keystore.public_key().unwrap(),
         signing_definition: SigningDefinition::LocalKeystore {
             voting_keystore_path,
@@ -640,6 +644,8 @@ fn validator_import_launchpad_no_password_then_add_password() {
         suggested_fee_recipient: None,
         gas_limit: None,
         builder_proposals: None,
+        builder_boost_factor: None,
+        prefer_builder_proposals: None,
         voting_public_key: keystore.public_key().unwrap(),
         signing_definition: SigningDefinition::LocalKeystore {
             voting_keystore_path: dst_keystore_dir.join(KEYSTORE_NAME),
@@ -742,6 +748,8 @@ fn validator_import_launchpad_password_file() {
         suggested_fee_recipient: None,
         gas_limit: None,
         builder_proposals: None,
+        builder_boost_factor: None,
+        prefer_builder_proposals: None,
         signing_definition: SigningDefinition::LocalKeystore {
             voting_keystore_path,
             voting_keystore_password_path: None,

lighthouse/tests/validator_client.rs

@@ -464,6 +464,32 @@ fn builder_proposals_flag() {
         .with_config(|config| assert!(config.builder_proposals));
 }
 #[test]
+fn builder_boost_factor_flag() {
+    CommandLineTest::new()
+        .flag("builder-boost-factor", Some("150"))
+        .run()
+        .with_config(|config| assert_eq!(config.builder_boost_factor, Some(150)));
+}
+#[test]
+fn no_builder_boost_factor_flag() {
+    CommandLineTest::new()
+        .run()
+        .with_config(|config| assert_eq!(config.builder_boost_factor, None));
+}
+#[test]
+fn prefer_builder_proposals_flag() {
+    CommandLineTest::new()
+        .flag("prefer-builder-proposals", None)
+        .run()
+        .with_config(|config| assert!(config.prefer_builder_proposals));
+}
+#[test]
+fn no_prefer_builder_proposals_flag() {
+    CommandLineTest::new()
+        .run()
+        .with_config(|config| assert!(!config.prefer_builder_proposals));
+}
+#[test]
 fn no_builder_registration_timestamp_override_flag() {
     CommandLineTest::new()
         .run()