Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate tracing #6339

Open
wants to merge 123 commits into
base: unstable
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 60 commits
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
f59246e
add tracing to beacon_node
ThreeHrSleep Sep 2, 2024
c5ab874
add tracing to common
ThreeHrSleep Sep 2, 2024
2b2577b
add tracing to consensus
ThreeHrSleep Sep 2, 2024
131e939
add tracing to database_manager
ThreeHrSleep Sep 2, 2024
a3c4a4d
add tracing to lcli
ThreeHrSleep Sep 2, 2024
9749a5e
add tracing to lighthouse
ThreeHrSleep Sep 2, 2024
18b7a6a
add tracing to slasher
ThreeHrSleep Sep 2, 2024
80ee046
add tracing to validator_client
ThreeHrSleep Sep 2, 2024
e5c107e
add tracing to validator_client
ThreeHrSleep Sep 2, 2024
f558244
add tracing to beacon_node
ThreeHrSleep Sep 2, 2024
52e0471
add tracing to beacon_node/lighthouse_network
ThreeHrSleep Sep 7, 2024
c361e0e
add tracing to beacon_node/network
ThreeHrSleep Sep 7, 2024
eaf3174
add tracing to boot_node
ThreeHrSleep Sep 7, 2024
497e04a
nit
ThreeHrSleep Sep 7, 2024
df09d4e
suggested changes from last review
ThreeHrSleep Sep 17, 2024
8d7a009
remove redundant loggers (partial)
ThreeHrSleep Sep 21, 2024
49745c0
remove more loggers
ThreeHrSleep Sep 21, 2024
4aebccb
minor fixes + cargo fmt
ThreeHrSleep Sep 22, 2024
4ca9e28
remove junk
ThreeHrSleep Sep 22, 2024
73743a5
Tracing executor and metrics rename
AgeManning Sep 23, 2024
b2c058e
Apply suggestions from code review
ThreeHrSleep Oct 2, 2024
57df87e
fix errors(partial)
ThreeHrSleep Oct 5, 2024
d616b8d
display formatting related fixes
ThreeHrSleep Oct 6, 2024
d235cf6
nit
ThreeHrSleep Oct 7, 2024
d137df1
nit
ThreeHrSleep Oct 7, 2024
d222bb7
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Oct 7, 2024
99ddc8d
resolve merge conflicts
ThreeHrSleep Oct 7, 2024
5147d05
comment out sse logging portion + remove more slog stuff
ThreeHrSleep Oct 8, 2024
602b16d
replace sloggers with tracing-appender
ThreeHrSleep Oct 9, 2024
1c65b76
fix beacon node crash
ThreeHrSleep Oct 9, 2024
9ada2c5
rip out sloggers totally
ThreeHrSleep Oct 9, 2024
acd500d
fix file logging for discv5 & libp2p
ThreeHrSleep Oct 10, 2024
60335a4
nit
ThreeHrSleep Oct 10, 2024
9744b27
use RollingFileAppender::builder() instead of RollingFileAppender:new()
ThreeHrSleep Oct 10, 2024
c069004
resolve conflict
ThreeHrSleep Oct 10, 2024
0a52c2d
add custom formatting layer
ThreeHrSleep Oct 11, 2024
a0828a7
adjust formatter
ThreeHrSleep Oct 11, 2024
991be3b
format crit
ThreeHrSleep Oct 11, 2024
9f0da68
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Oct 12, 2024
3314eba
nit
ThreeHrSleep Oct 12, 2024
63002ff
add sse logging
ThreeHrSleep Oct 14, 2024
a2a4c97
format crit (sse)
ThreeHrSleep Oct 14, 2024
666f572
clean up
ThreeHrSleep Oct 14, 2024
46c5fd4
remove slog totally
ThreeHrSleep Oct 15, 2024
6cc4f18
add spans
ThreeHrSleep Oct 15, 2024
d2cf3d6
plug in MetricsLayer
ThreeHrSleep Oct 15, 2024
9249fdc
support disable-log-timestamp
ThreeHrSleep Oct 15, 2024
394ea1b
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Oct 15, 2024
8b58428
nit
ThreeHrSleep Oct 16, 2024
c7b3680
remove unwrap from SSE_LOGGING_COMPONENTS
ThreeHrSleep Oct 16, 2024
129bf06
nit
ThreeHrSleep Oct 17, 2024
c6a00d9
add SSE logging support for vc
ThreeHrSleep Oct 17, 2024
4f9bd29
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Oct 17, 2024
1cf9373
make file logging optional
ThreeHrSleep Oct 18, 2024
c16e505
add support for logfile-debug-level,debug-level
ThreeHrSleep Oct 19, 2024
b078f75
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Oct 19, 2024
3f41aee
fix errors
ThreeHrSleep Oct 19, 2024
8b35d37
fix spans
ThreeHrSleep Oct 20, 2024
4d1681f
add spans for service
ThreeHrSleep Oct 21, 2024
fb79b99
nit
ThreeHrSleep Oct 21, 2024
d84ba77
fix simulator tests
ThreeHrSleep Oct 22, 2024
56852b0
fix compilation errors in tests
ThreeHrSleep Oct 24, 2024
dc75b41
nit
ThreeHrSleep Oct 24, 2024
b2757b0
remove unnecessary layers from basic_sim
ThreeHrSleep Oct 25, 2024
069240c
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Oct 25, 2024
ac83633
Remove static SSE
macladson Oct 25, 2024
b01aaff
Merge branch 'unstable' into tracing-integration
macladson Oct 25, 2024
938f173
add default location for bn,vc logfile
ThreeHrSleep Oct 26, 2024
02848a4
add tracing spans to the slasher
eserilev Oct 28, 2024
b8d6c73
info level
eserilev Oct 28, 2024
2cbc2c2
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Oct 28, 2024
7b4fa4d
rename places where r#type is used
ThreeHrSleep Oct 28, 2024
0370a2c
Merge pull request #1 from eserilev/tracing-integration-slasher-span
ThreeHrSleep Oct 30, 2024
89b2847
de-duplicate logger construction code
ThreeHrSleep Oct 31, 2024
86a18a2
rename --logfile to --logfile-dir
ThreeHrSleep Oct 31, 2024
1b50ac6
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Nov 4, 2024
f2918f5
move LevelFilters into LoggerConfig
ThreeHrSleep Nov 5, 2024
349baf3
remaining changes
ThreeHrSleep Nov 5, 2024
c9c9463
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Nov 15, 2024
58dbff9
resolve merge conflicts
ThreeHrSleep Nov 17, 2024
2bba3ab
resolve more conflicts
ThreeHrSleep Nov 17, 2024
59fa07c
moreee conflicts
ThreeHrSleep Nov 21, 2024
17c1d98
clean-up
ThreeHrSleep Nov 22, 2024
eb0cef9
Merge latest unstable
AgeManning Nov 26, 2024
bf6bd4f
merge conflicts and lints
eserilev Nov 27, 2024
83a6297
merge conflicts
eserilev Nov 27, 2024
0281b7d
data availability checker tracing span
eserilev Nov 27, 2024
45c8ad8
add validator_monitor span instrumentation
eserilev Nov 27, 2024
2d926ef
fmt
eserilev Nov 27, 2024
27c4fdd
add spans to libp2p_rpc, libp2p, sender, receiver, router, sync, network
eserilev Nov 27, 2024
c57cc31
add range_sync span
eserilev Nov 27, 2024
de7b04b
backfill_sync span
eserilev Nov 27, 2024
77fc2d4
lookup sync span
eserilev Nov 27, 2024
35532a2
tracing spans
eserilev Nov 27, 2024
c02f0e5
fix tests
eserilev Nov 27, 2024
b140180
resolve merge conflicts
eserilev Nov 28, 2024
7cf46fc
add logging to rpc tests
eserilev Nov 28, 2024
874e82c
add more logging to tests
eserilev Nov 28, 2024
3428496
init testing logging
eserilev Dec 1, 2024
2d2e108
resolve merge conflicts
eserilev Dec 1, 2024
9b936b8
Remove unneeded dep
eserilev Dec 1, 2024
b9e5f6c
udeps and lints
eserilev Dec 1, 2024
9feb37a
slasher guard'
eserilev Dec 1, 2024
f7016f2
suggested changes
sayan0111 Dec 2, 2024
066a9e9
conflicts
ThreeHrSleep Dec 2, 2024
8ff6dc4
nit
ThreeHrSleep Dec 2, 2024
16a1a60
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Dec 16, 2024
75150aa
merge conflicts
ThreeHrSleep Dec 16, 2024
ddf80ee
resolve more merge conflicts
ThreeHrSleep Dec 17, 2024
54aafb8
remove slog from some tests
ThreeHrSleep Dec 17, 2024
5d10bd9
moreeee conflicts
ThreeHrSleep Dec 20, 2024
7dbde61
clean up
ThreeHrSleep Dec 23, 2024
bc6bdf8
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Jan 2, 2025
9f4772f
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Jan 6, 2025
3208430
Merge pull request #6768 from sigp/unstable
jimmygchen Jan 8, 2025
b9aa8fb
Merge branch 'anchor' into tracing-integration
jimmygchen Jan 8, 2025
843c675
Merge branch 'unstable' into tracing-integration
ThreeHrSleep Jan 10, 2025
103c944
fix compilation errors
ThreeHrSleep Jan 10, 2025
ac51378
cargo sort
ThreeHrSleep Jan 12, 2025
e87898d
Minor formatting
macladson Jan 12, 2025
b536036
Fix test compliation errors
macladson Jan 13, 2025
7efe331
make cli
macladson Jan 13, 2025
84fc3ac
Remove unused dependencies
macladson Jan 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
312 changes: 52 additions & 260 deletions Cargo.lock

Large diffs are not rendered by default.

5 changes: 0 additions & 5 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,6 @@ members = [
"testing/execution_engine_integration",
"testing/node_test_rig",
"testing/simulator",
"testing/test-test_logger",
"testing/state_transition_vectors",
"testing/web3signer_tests",

Expand Down Expand Up @@ -157,10 +156,6 @@ serde_json = "1"
serde_repr = "0.1"
serde_yaml = "0.9"
sha2 = "0.9"
slog = { version = "2", features = ["max_level_debug", "release_max_level_debug", "nested-values"] }
slog-async = "2"
slog-term = "2"
sloggers = { version = "2", features = ["json"] }
smallvec = { version = "1.11.2", features = ["arbitrary"] }
snap = "1"
ssz_types = "0.8"
Expand Down
2 changes: 1 addition & 1 deletion beacon_node/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ types = { workspace = true }
store = { workspace = true }
client = { path = "client" }
clap = { workspace = true }
slog = { workspace = true }
tracing = { workspace = true }
dirs = { workspace = true }
directory = { workspace = true }
environment = { workspace = true }
Expand Down
5 changes: 1 addition & 4 deletions beacon_node/beacon_chain/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -58,10 +58,6 @@ sensitive_url = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
slasher = { workspace = true }
slog = { workspace = true }
slog-async = { workspace = true }
slog-term = { workspace = true }
sloggers = { workspace = true }
slot_clock = { workspace = true }
smallvec = { workspace = true }
ssz_types = { workspace = true }
Expand All @@ -76,6 +72,7 @@ tokio-stream = { workspace = true }
tree_hash = { workspace = true }
tree_hash_derive = { workspace = true }
types = { workspace = true }
tracing = { workspace = true }

[[test]]
name = "beacon_chain_tests"
Expand Down
15 changes: 9 additions & 6 deletions beacon_node/beacon_chain/src/attestation_rewards.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ use eth2::lighthouse::StandardAttestationRewards;
use eth2::types::ValidatorId;
use safe_arith::SafeArith;
use serde_utils::quoted_u64::Quoted;
use slog::debug;
use state_processing::common::base::{self, SqrtTotalActiveBalance};
use state_processing::per_epoch_processing::altair::{
process_inactivity_updates_slow, process_justification_and_finalization,
Expand All @@ -29,6 +28,7 @@ use store::consts::altair::{
PARTICIPATION_FLAG_WEIGHTS, TIMELY_HEAD_FLAG_INDEX, TIMELY_SOURCE_FLAG_INDEX,
TIMELY_TARGET_FLAG_INDEX,
};
use tracing::debug;
use types::consts::altair::WEIGHT_DENOMINATOR;
use types::{BeaconState, Epoch, EthSpec, RelativeEpoch};

Expand All @@ -38,7 +38,11 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
epoch: Epoch,
validators: Vec<ValidatorId>,
) -> Result<StandardAttestationRewards, BeaconChainError> {
debug!(self.log, "computing attestation rewards"; "epoch" => epoch, "validator_count" => validators.len());
debug!(
%epoch,
validator_count = validators.len(),
"computing attestation rewards"
);

// Get state
let state_slot = (epoch + 1).end_slot(T::EthSpec::slots_per_epoch());
Expand Down Expand Up @@ -215,10 +219,9 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
// Return 0s for unknown/inactive validator indices.
let Ok(validator) = state.get_validator(validator_index) else {
debug!(
self.log,
"No rewards for inactive/unknown validator";
"index" => validator_index,
"epoch" => previous_epoch
index = validator_index,
epoch = %previous_epoch,
"No rewards for inactive/unknown validator"
);
total_rewards.push(TotalAttestationRewards {
validator_index: validator_index as u64,
Expand Down
21 changes: 8 additions & 13 deletions beacon_node/beacon_chain/src/attestation_simulator.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
use crate::{BeaconChain, BeaconChainTypes};
use slog::{debug, error};
use slot_clock::SlotClock;
use std::sync::Arc;
use task_executor::TaskExecutor;
use tokio::time::sleep;
use tracing::{debug, error};
use types::{EthSpec, Slot};

/// Don't run the attestation simulator if the head slot is this many epochs
Expand Down Expand Up @@ -36,10 +36,7 @@ async fn attestation_simulator_service<T: BeaconChainTypes>(
Some(duration) => {
sleep(duration + additional_delay).await;

debug!(
chain.log,
"Simulating unagg. attestation production";
);
debug!("Simulating unagg. attestation production");

// Run the task in the executor
let inner_chain = chain.clone();
Expand All @@ -53,7 +50,7 @@ async fn attestation_simulator_service<T: BeaconChainTypes>(
);
}
None => {
error!(chain.log, "Failed to read slot clock");
error!("Failed to read slot clock");
// If we can't read the slot clock, just wait another slot.
sleep(slot_duration).await;
}
Expand Down Expand Up @@ -85,10 +82,9 @@ pub fn produce_unaggregated_attestation<T: BeaconChainTypes>(
let data = unaggregated_attestation.data();

debug!(
chain.log,
"Produce unagg. attestation";
"attestation_source" => data.source.root.to_string(),
"attestation_target" => data.target.root.to_string(),
attestation_source = data.source.root.to_string(),
attestation_target = data.target.root.to_string(),
"Produce unagg. attestation"
);

chain
Expand All @@ -98,9 +94,8 @@ pub fn produce_unaggregated_attestation<T: BeaconChainTypes>(
}
Err(e) => {
debug!(
chain.log,
"Failed to simulate attestation";
"error" => ?e
error = ?e,
"Failed to simulate attestation"
);
}
}
Expand Down
14 changes: 6 additions & 8 deletions beacon_node/beacon_chain/src/attestation_verification.rs
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@ use crate::{
use bls::verify_signature_sets;
use itertools::Itertools;
use proto_array::Block as ProtoBlock;
use slog::debug;
use slot_clock::SlotClock;
use state_processing::{
common::{
Expand All @@ -58,6 +57,7 @@ use state_processing::{
};
use std::borrow::Cow;
use strum::AsRefStr;
use tracing::debug;
use tree_hash::TreeHash;
use types::{
Attestation, AttestationRef, BeaconCommittee, BeaconStateError::NoCommitteeFound, ChainSpec,
Expand Down Expand Up @@ -409,10 +409,9 @@ fn process_slash_info<T: BeaconChainTypes>(
Ok((indexed, _)) => (indexed, true, err),
Err(e) => {
debug!(
chain.log,
"Unable to obtain indexed form of attestation for slasher";
"attestation_root" => format!("{:?}", attestation.tree_hash_root()),
"error" => format!("{:?}", e)
attestation_root = ?attestation.tree_hash_root(),
error = ?e,
"Unable to obtain indexed form of attestation for slasher"
);
return err;
}
Expand All @@ -426,9 +425,8 @@ fn process_slash_info<T: BeaconChainTypes>(
if check_signature {
if let Err(e) = verify_attestation_signature(chain, &indexed_attestation) {
debug!(
chain.log,
"Signature verification for slasher failed";
"error" => format!("{:?}", e),
error = ?e,
"Signature verification for slasher failed"
);
return err;
}
Expand Down
22 changes: 9 additions & 13 deletions beacon_node/beacon_chain/src/beacon_block_reward.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ use crate::{BeaconChain, BeaconChainError, BeaconChainTypes, StateSkipConfig};
use attesting_indices_base::get_attesting_indices;
use eth2::lighthouse::StandardBlockReward;
use safe_arith::SafeArith;
use slog::error;
use state_processing::common::attesting_indices_base;
use state_processing::{
common::{
Expand All @@ -19,6 +18,7 @@ use store::{
consts::altair::{PARTICIPATION_FLAG_WEIGHTS, PROPOSER_WEIGHT, WEIGHT_DENOMINATOR},
RelativeEpoch,
};
use tracing::error;
use types::{AbstractExecPayload, BeaconBlockRef, BeaconState, BeaconStateError, EthSpec};

type BeaconBlockSubRewardValue = u64;
Expand Down Expand Up @@ -56,9 +56,8 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
.compute_beacon_block_proposer_slashing_reward(block, state)
.map_err(|e| {
error!(
self.log,
"Error calculating proposer slashing reward";
"error" => ?e
error = ?e,
"Error calculating proposer slashing reward"
);
BeaconChainError::BlockRewardError
})?;
Expand All @@ -67,9 +66,8 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
.compute_beacon_block_attester_slashing_reward(block, state)
.map_err(|e| {
error!(
self.log,
"Error calculating attester slashing reward";
"error" => ?e
error = ?e,
"Error calculating attester slashing reward"
);
BeaconChainError::BlockRewardError
})?;
Expand All @@ -78,19 +76,17 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
self.compute_beacon_block_attestation_reward_base(block, state)
.map_err(|e| {
error!(
self.log,
"Error calculating base block attestation reward";
"error" => ?e
error = ?e,
"Error calculating base block attestation reward"
);
BeaconChainError::BlockRewardAttestationError
})?
} else {
self.compute_beacon_block_attestation_reward_altair_deneb(block, state)
.map_err(|e| {
error!(
self.log,
"Error calculating altair block attestation reward";
"error" => ?e
error = ?e,
"Error calculating altair block attestation reward"
);
BeaconChainError::BlockRewardAttestationError
})?
Expand Down
Loading