Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add hints to port conflict and lock file panics #1535

Merged
merged 61 commits into from
Jan 29, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
0500cae
add hint for port error
oxarbitrage Dec 15, 2020
c11203f
change error to panic
oxarbitrage Dec 17, 2020
eb378b5
add issue filter for port panic
oxarbitrage Dec 17, 2020
a894261
add lock file hint
oxarbitrage Dec 17, 2020
6f71772
remove non needed brackets
oxarbitrage Dec 17, 2020
ef7744b
add metrics endpoint port conflict hint
oxarbitrage Dec 17, 2020
8756236
add hint for tracing endpoint port conflict
oxarbitrage Dec 17, 2020
88e31eb
apply all suggestions from code review
oxarbitrage Dec 18, 2020
02ed386
add acceptance test for resource conflics
oxarbitrage Dec 18, 2020
770ffb5
Use the correct regex for the port conflict acceptance test
teor2345 Dec 18, 2020
09493ef
Simplify the test using helper functions
teor2345 Dec 19, 2020
29cd0cd
Split out common config conflict code into a function
teor2345 Dec 19, 2020
a2f97ab
Add state, metrics, and tracing conflict tests
teor2345 Dec 19, 2020
4326b67
Add a full set of stderr acceptance test functions
teor2345 Dec 19, 2020
a4b83fa
print stderr
oxarbitrage Jan 19, 2021
fa15874
fix the build
oxarbitrage Jan 19, 2021
7921547
add a longer delay for port conflict
oxarbitrage Jan 19, 2021
0b20a69
change port conflict regex for windows
oxarbitrage Jan 20, 2021
abe5902
rustfmt
oxarbitrage Jan 20, 2021
8036941
disable check
oxarbitrage Jan 20, 2021
0fa3781
skip conflict tests on macos
oxarbitrage Jan 20, 2021
2f4728d
add timeout to listener bind
yaahc Jan 21, 2021
16e61c2
test if output is being lost
yaahc Jan 22, 2021
b19d8b3
add more tracing
yaahc Jan 22, 2021
6b4134a
add more tracing
yaahc Jan 22, 2021
8817a2c
more printf debugging, MOREEE
yaahc Jan 22, 2021
767b94b
move zcash listener opening
oxarbitrage Jan 23, 2021
6aa6abb
remove listener bind timeout
oxarbitrage Jan 23, 2021
2e3cd15
remove big delay
oxarbitrage Jan 23, 2021
7fc0983
update platform conditionals docs
oxarbitrage Jan 23, 2021
09aab1d
test timeout in macOS zcash listener
oxarbitrage Jan 23, 2021
f11a3b6
remove commented out code
oxarbitrage Jan 25, 2021
6ca008c
add and use network contants
oxarbitrage Jan 25, 2021
e99063b
use timeout constant in listener bind timeout
oxarbitrage Jan 25, 2021
a0a3a61
remove unused import
oxarbitrage Jan 25, 2021
9198f18
cleanup
oxarbitrage Jan 26, 2021
e11b976
add todo about hint for disk full
oxarbitrage Jan 26, 2021
925a4fb
avoid the 2 constant names by os
oxarbitrage Jan 26, 2021
df0e38d
remove one kill
oxarbitrage Jan 26, 2021
d349d21
add constant for lock file
oxarbitrage Jan 26, 2021
7797ef6
remove whitespace
oxarbitrage Jan 26, 2021
7ee9149
early exit ignored macos test
oxarbitrage Jan 27, 2021
1a8f724
make LOCK_FILE_ERROR a regex
oxarbitrage Jan 28, 2021
6b9f2f6
match path in state cache
oxarbitrage Jan 28, 2021
1459503
changes to PORT_IN_USE_ERROR
oxarbitrage Jan 28, 2021
a0dbca9
rustfmt
oxarbitrage Jan 28, 2021
2b32a2b
try a pathbuf
oxarbitrage Jan 28, 2021
11d3a61
try escape backslash for windows
oxarbitrage Jan 28, 2021
ca20f22
try string literal
oxarbitrage Jan 28, 2021
1d0741c
another windows match try
oxarbitrage Jan 28, 2021
d599ba8
remove the string literal
oxarbitrage Jan 28, 2021
6c87525
change regex
oxarbitrage Jan 28, 2021
cbdcddf
escape
oxarbitrage Jan 28, 2021
8f3c0f4
dont match windows cache path
oxarbitrage Jan 29, 2021
a69d061
Use Display for state path logs
teor2345 Jan 29, 2021
b792c7d
Add Windows conflict error messages
teor2345 Jan 29, 2021
5d0c8da
Turn PORT_IN_USE_ERROR into a regex
teor2345 Jan 29, 2021
0e9b38a
Use the new PORT_IN_USE_ERROR regex
teor2345 Jan 29, 2021
5274bad
Syntax fixes
teor2345 Jan 29, 2021
4c25225
More syntax
teor2345 Jan 29, 2021
825db47
Add regex and lazy_static to zebra_network
teor2345 Jan 29, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions zebra-network/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,10 @@ hex = "0.4"
# indexmap has rayon support for parallel iteration,
# which we don't use, so disable it to drop the dependencies.
indexmap = { version = "1.6", default-features = false }
lazy_static = "1.4.0"
pin-project = "0.4"
rand = "0.7"
regex = "1"
serde = { version = "1", features = ["serde_derive"] }
thiserror = "1"

Expand Down
13 changes: 13 additions & 0 deletions zebra-network/src/constants.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

use std::time::Duration;

use lazy_static::lazy_static;
use regex::Regex;

// XXX should these constants be split into protocol also?
use crate::protocol::external::types::*;

Expand Down Expand Up @@ -95,6 +98,16 @@ pub const EWMA_DEFAULT_RTT: Duration = Duration::from_secs(20 + 1);
/// better peers when we restart the sync.
pub const EWMA_DECAY_TIME: Duration = Duration::from_secs(200);

lazy_static! {
/// OS-specific error when the port attempting to be opened is already in use.
pub static ref PORT_IN_USE_ERROR: Regex = if cfg!(unix) {
#[allow(clippy::trivial_regex)]
Regex::new("already in use")
} else {
Regex::new("(access a socket in a way forbidden by its access permissions)|(Only one usage of each socket address)")
}.expect("regex is valid");
}

/// Magic numbers used to identify different Zcash networks.
pub mod magics {
use super::*;
Expand Down
2 changes: 1 addition & 1 deletion zebra-network/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ pub type BoxError = Box<dyn std::error::Error + Send + Sync + 'static>;

mod address_book;
mod config;
mod constants;
pub mod constants;
mod isolated;
mod meta_addr;
mod peer;
Expand Down
36 changes: 25 additions & 11 deletions zebra-network/src/peer_set/initialize.rs
Original file line number Diff line number Diff line change
Expand Up @@ -115,16 +115,7 @@ where
);
let peer_set = Buffer::new(BoxService::new(peer_set), constants::PEERSET_BUFFER_SIZE);

// Connect the tx end to the 3 peer sources:

// 1. Initial peers, specified in the config.
let add_guard = tokio::spawn(add_initial_peers(
config.initial_peers(),
connector.clone(),
peerset_tx.clone(),
));

// 2. Incoming peer connections, via a listener.
// 1. Incoming peer connections, via a listener.

// Warn if we're configured using the wrong network port.
// TODO: use the right port if the port is unspecified
Expand All @@ -144,6 +135,18 @@ where

let listen_guard = tokio::spawn(listen(config.listen_addr, listener, peerset_tx.clone()));

let initial_peers_fut = {
let initial_peers = config.initial_peers();
let connector = connector.clone();
let tx = peerset_tx.clone();

// Connect the tx end to the 3 peer sources:
add_initial_peers(initial_peers, connector, tx)
};

// 2. Initial peers, specified in the config.
let add_guard = tokio::spawn(initial_peers_fut);

// 3. Outgoing peers we connect to in response to load.
let mut candidates = CandidateSet::new(address_book.clone(), peer_set.clone());

Expand Down Expand Up @@ -211,7 +214,18 @@ where
S: Service<(TcpStream, SocketAddr), Response = peer::Client, Error = BoxError> + Clone,
S::Future: Send + 'static,
{
let listener = TcpListener::bind(addr).await?;
let listener_result = TcpListener::bind(addr).await;

let listener = match listener_result {
Ok(l) => l,
Err(e) => panic!(
"Opening Zcash network protocol listener {:?} failed: {:?}. \
Hint: Check if another zebrad or zcashd process is running. \
Try changing the network listen_addr in the Zebra config.",
addr, e,
),
};

let local_addr = listener.local_addr()?;
info!("Opened Zcash protocol endpoint at {}", local_addr);
loop {
Expand Down
1 change: 1 addition & 0 deletions zebra-state/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ zebra-chain = { path = "../zebra-chain" }
dirs = "3.0.1"
hex = "0.4.2"
lazy_static = "1.4.0"
regex = "1"
serde = { version = "1", features = ["serde_derive"] }

futures = "0.3.12"
Expand Down
10 changes: 10 additions & 0 deletions zebra-state/src/constants.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
//! Definitions of constants.

/// The maturity threshold for transparent coinbase outputs.
///
/// A transaction MUST NOT spend a transparent output of a coinbase transaction
Expand All @@ -13,3 +15,11 @@ pub const MAX_BLOCK_REORG_HEIGHT: u32 = MIN_TRANSPARENT_COINBASE_MATURITY - 1;

/// The database format version, incremented each time the database format changes.
pub const DATABASE_FORMAT_VERSION: u32 = 4;

use lazy_static::lazy_static;
use regex::Regex;

lazy_static! {
/// Regex that matches the RocksDB error when its lock file is already open.
pub static ref LOCK_FILE_ERROR: Regex = Regex::new("(lock file).*(temporarily unavailable)|(in use)|(being used by another process)").expect("regex is valid");
}
2 changes: 1 addition & 1 deletion zebra-state/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#![allow(clippy::unnecessary_wraps)]

mod config;
mod constants;
pub mod constants;
mod error;
mod request;
mod response;
Expand Down
17 changes: 15 additions & 2 deletions zebra-state/src/service/finalized_state.rs
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,21 @@ impl FinalizedState {
rocksdb::ColumnFamilyDescriptor::new("sprout_nullifiers", db_options.clone()),
rocksdb::ColumnFamilyDescriptor::new("sapling_nullifiers", db_options.clone()),
];
let db = rocksdb::DB::open_cf_descriptors(&db_options, path, column_families)
.expect("database path and options are valid");
let db_result = rocksdb::DB::open_cf_descriptors(&db_options, &path, column_families);

let db = match db_result {
Ok(d) => {
tracing::info!("Opened Zebra state cache at {}", path.display());
d
}
// TODO: provide a different hint if the disk is full, see #1623
Err(e) => panic!(
oxarbitrage marked this conversation as resolved.
Show resolved Hide resolved
"Opening database {:?} failed: {:?}. \
Hint: Check if another zebrad process is running. \
Try changing the state cache_dir in the Zebra config.",
path, e,
),
};

let new_state = Self {
queued_by_prev_hash: HashMap::new(),
Expand Down
Loading