Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Log validator index, status and pubkey each epoch #5161

Merged
Merged
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
01b32cb
Fix log validator index, status and pubkey each epoch (#4785)
maschad Feb 17, 2023
a7a778d
Update validator to display correct status and pubkey each epoch (#4785)
maschad Feb 17, 2023
fe09b0d
Fix update log info formatting for validator status (#4785)
maschad Feb 17, 2023
b4eac94
Update logs to include total validator availability count (#4785)
maschad Feb 22, 2023
2a9b72f
Merge branch 'unstable' into fix/log-validator-status-and-pubkey
maschad Feb 22, 2023
1014495
Extracted logging to a function and included all validator statuses (…
maschad Feb 22, 2023
f1f0e36
Refactor to print logs more efficiently (#4785)
maschad Feb 22, 2023
aa79941
Fix imports and updated readability (#4785)
maschad Feb 22, 2023
26e35d0
Updated imports + adjusted coding preferences and log formatting
maschad Feb 23, 2023
ba679cc
Refactored logs to only report the totals by status + removed logging…
maschad Feb 24, 2023
9250c53
fix linting issue
maschad Feb 24, 2023
f1f54aa
Merge branch 'unstable' into fix/log-validator-status-and-pubkey
maschad Feb 27, 2023
31a3879
fix: added mapping for simple validator statuses
maschad Feb 27, 2023
cd71135
updated mapping for validator statuses
maschad Feb 28, 2023
6e27d85
fix potential NaN error in pending statuses + some minor refactors
maschad Mar 2, 2023
50825d1
fix issue with getting validator pending statuses
maschad Mar 2, 2023
07e9d07
Removed debug log for validator exists in beacon chain
maschad Mar 2, 2023
7a2110a
Merge branch 'unstable' into fix/log-validator-status-and-pubkey
maschad Mar 2, 2023
d8094a4
updated import in prepareBeaconProposer
maschad Mar 2, 2023
700b062
reverted changes in prepareBeaconProposer and added TODO comments
maschad Mar 2, 2023
5eb7541
update lint error
maschad Mar 3, 2023
335ebbc
updated log params
maschad Mar 3, 2023
b01b460
removed TODOs
maschad Mar 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 52 additions & 4 deletions packages/validator/src/services/indices.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import {ValidatorIndex} from "@lodestar/types";
import {Logger} from "@lodestar/utils";
import {Logger, MapDef} from "@lodestar/utils";
import {toHexString} from "@chainsafe/ssz";
import {Api, ApiError} from "@lodestar/api";
import {Api, ApiError, routes} from "@lodestar/api";
import {batchItems} from "../util/index.js";
import {Metrics} from "../metrics.js";

Expand All @@ -14,6 +14,31 @@ const PUBKEYS_PER_REQUEST = 10;
// To assist with readability
type PubkeyHex = string;

// To assist with logging statuses, we only log the statuses that are not active_exiting or withdrawal_possible
type SimpleValidatorStatus = "pending" | "active" | "exited" | "withdrawn";

const statusToSimpleStatusMapping = (status: routes.beacon.ValidatorStatus): SimpleValidatorStatus => {
switch (status) {
case "active":
case "active_exiting":
case "active_slashed":
case "active_ongoing":
return "active";

case "withdrawal_possible":
case "exited_slashed":
case "exited_unslashed":
return "exited";

case "pending_initialized":
case "pending_queued":
return "pending";

case "withdrawal_done":
return "withdrawn";
}
};

export class IndicesService {
readonly index2pubkey = new Map<ValidatorIndex, PubkeyHex>();
/** Indexed by pubkey in hex 0x prefixed */
Expand Down Expand Up @@ -90,7 +115,6 @@ export class IndicesService {
newIndices.push(...validatorIndicesArr);
}

this.logger.info("Discovered new validators", {count: newIndices.length});
this.metrics?.discoveredIndices.inc(newIndices.length);

return newIndices;
Expand All @@ -101,15 +125,39 @@ export class IndicesService {
ApiError.assert(res, "Can not fetch state validators from beacon node");

const newIndices = [];

const allValidatorStatuses = new MapDef<SimpleValidatorStatus, number>(() => 0);

for (const validatorState of res.response.data) {
// Group all validators by status
const status = statusToSimpleStatusMapping(validatorState.status);
allValidatorStatuses.set(status, allValidatorStatuses.getOrDefault(status) + 1);

const pubkeyHex = toHexString(validatorState.validator.pubkey);
if (!this.pubkey2index.has(pubkeyHex)) {
this.logger.debug("Discovered validator", {pubkey: pubkeyHex, index: validatorState.index});
this.logger.info("Validator exists in beacon chain", {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maschad running this on goerli with quite a lot of validators and this inital log is quite noisy in my opinion, we are printing out Validator exists in beacon chain for each validator which seems bit redundant because these logs all happen after each other.

I think we should reconsider this and maybe only print out Validator exists in beacon chain once and then just list all the validators, with index, pubkey and ideally also fee recipient (if possible)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I've included that change in be5a7e4

validatorIndex: validatorState.index,
pubKey: pubkeyHex,
});
this.pubkey2index.set(pubkeyHex, validatorState.index);
this.index2pubkey.set(validatorState.index, pubkeyHex);
newIndices.push(validatorState.index);
}
}

// The number of validators that are not in the beacon chain
const pendingCount = pubkeysHex.length - res.response.data.length;

allValidatorStatuses.set("pending", allValidatorStatuses.getOrDefault("pending") + pendingCount);

// Retrieve the number of validators for each status
const statuses = Object.fromEntries(Array.from(allValidatorStatuses.entries()).filter((entry) => entry[1] > 0));

// The total number of validators
const total = pubkeysHex.length;

this.logger.info("Validator statuses", {...statuses, total});
Copy link
Member

@nflaig nflaig Apr 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maschad wondering here, wasn't this intended to be logged every epoch?

Once we discovered all pubkey we are not calling this due to this check

const pubkeysHexToDiscover = pubkeysHex.filter((pubkey) => !this.pubkey2index.has(pubkey));
if (pubkeysHexToDiscover.length === 0) {
return [];
}

I guess this should be fixed by the other PR #5239


return newIndices;
}
}