This repository has been archived by the owner on Nov 6, 2020. It is now read-only.

Implement hardcoded sync in the light client #8075

Merged

tomaka merged 8 commits into openethereum:master from tomaka:hardcoded-sync

Mar 27, 2018

Contributor

tomaka commented Mar 8, 2018 •

edited

Loading

Adds a hardcodedSync field to the chain specifications.
When set, contains the RLP of a header, a total difficulty, and a list of CHT root hashes. If there is no light client database, the light client will immediately jump to the given block.

After this change, the light client synchronizes in 10 to 15 seconds even at the first launch on the main network.

For now only the specs of the main network have been updated.

parity-cla-bot commented Mar 8, 2018

It looks like @tomaka signed our Contributor License Agreement. 👍

Many thanks,

Parity Technologies CLA Bot

tomaka force-pushed the hardcoded-sync branch from 52b690b to a3fc5b4 Compare

March 8, 2018 15:34

rphmeier added the A0-pleasereview 🤓 label


          Implement hardcoded sync

9e32988

tomaka force-pushed the hardcoded-sync branch from eb9d6a3 to 9e32988 Compare

March 8, 2018 16:07

tomaka added the M4-core ⛓ label

rphmeier reviewed

View reviewed changes

ethcore/light/src/client/header_chain.rs Outdated

+              					let cht_num = cht::block_to_cht_number(decoded_header.number() - 1)
+              						.expect("specs provided a hardcoded block with height 0");
+              					if cht_num >= hardcoded_sync.chts.len() as u64 {
+              						panic!("specs didn't provide enough CHT roots for its hardcoded block");

Contributor

rphmeier Mar 8, 2018

rule for panics of any kind is "prove or remove" -- We may rather want to print a warning and fall back to hardcoded-less sync. Or move these checks to the deserialization phase where we can exit with an error more gracefully.

rphmeier reviewed

View reviewed changes

ethcore/light/src/client/header_chain.rs Outdated

@@ @@ -299,11 +332,36 @@ impl HeaderChain { @@
               	///
               	/// If the block is an epoch transition, provide the transition along with
               	/// the header.
+              	#[inline]

Contributor

rphmeier Mar 8, 2018

does that actually help? generally I figure LLVM will figure it out.

rphmeier reviewed

View reviewed changes

ethcore/light/src/client/header_chain.rs Outdated

+              					let header = if let Some(header) = self.block_header(BlockId::Number(h_num)) {
+              						header
+              					} else {
+              						panic!("Header of block #{} not found in DB", h_num);

Contributor

rphmeier Mar 8, 2018

same: do not panic.

rphmeier reviewed

View reviewed changes

ethcore/light/src/client/header_chain.rs Outdated


		let decoded = header.decode();

		let entry: Entry = ::rlp::decode(&self.db.get(self.col, era_key(h_num).as_bytes()).unwrap().unwrap());

Contributor

rphmeier Mar 8, 2018

and we really never use unwrap.

rphmeier reviewed

View reviewed changes

ethcore/light/src/client/header_chain.rs Outdated

+              					let entry: Entry = ::rlp::decode(&self.db.get(self.col, era_key(h_num).as_bytes()).unwrap().unwrap());
+              					let total_difficulty = entry.candidates.iter()
+              						.find(|c| c.hash == decoded.hash())
+              						.expect("no candidate matching block found in DB")

Contributor

rphmeier Mar 8, 2018

should rather prove based on the fact that an inconsistent but correctly-loaded database shouldn't be possible.

rphmeier reviewed

View reviewed changes

ethcore/light/src/client/header_chain.rs Outdated

+              			chts.push(cht);
+              		}
+              		unreachable!("we are after an infinite loop")

Contributor

rphmeier Mar 8, 2018 •

edited

Loading

could be a loop with a break-value, either way it could follow the codebase style of proposition; proof; qed

rphmeier reviewed

View reviewed changes

parity/run.rs Outdated

               	let service = light_client::Service::start(config, &spec, fetch, db, cache.clone())
               		.map_err(|e| format!("Error starting light client: {}", e))?;
               	let client = service.client();
+              	// Note: uncomment this code if you want to regenerate the JSON of the hardcoded sync feature
+              	// in the chain specifications.
+              	/*if let Some(hs) = client.read_hardcoded_sync() {

Contributor

rphmeier Mar 8, 2018

we could make this a subcommand of some kind

Contributor

rphmeier Mar 8, 2018

generally not a fan of the commented-out code. the CLI is a PITA to work with though :\

rphmeier added A4-gotissues 💥 and removed A0-pleasereview 🤓 labels

Contributor

rphmeier commented Mar 8, 2018

needs an opt-out flag also (off by default is fine, just that it should be there).

Contributor

rphmeier commented Mar 8, 2018

looks good other than grumbles

Contributor Author

tomaka commented Mar 8, 2018 •

edited

Loading

Should I make read_hardcoded_sync() return None in case of error? Or rather Result<Option<...>, ...>? I admit that I didn't really know what to do with it, it's an internal-ish method (which is also why I totally overlooked its quality :-/).

Contributor

rphmeier commented Mar 8, 2018

Result<Option<_>> would be better IMO

tomaka added 3 commits

March 8, 2018 18:33


          Fix concerns


          Remove artifact

a3d6830


          Fix cli tests

c4c4a61

Contributor Author

tomaka commented Mar 9, 2018

Sorry for the bad PR quality. I think it should be good now!

svyatonik added A0-pleasereview 🤓 and removed A4-gotissues 💥 labels

rphmeier mentioned this pull request

Get the light client to compile for iOS #7955

Closed

Contributor

rphmeier commented Mar 20, 2018 •

edited

Loading

needs merge, can you ping me for review? I'm not looking that closely at the Parity repo these days.


          Merge remote-tracking branch 'upstream/master' into hardcoded-sync

ff483c0

Contributor Author

tomaka commented Mar 20, 2018

@rphmeier Conflicts solved!

Tbaut mentioned this pull request

--light client stuck at #2088992 #8148

Closed


          Fix compilation

06b84dc

tomaka force-pushed the hardcoded-sync branch from a550069 to 06b84dc Compare

March 20, 2018 10:51


          Update hardcoded sync block

679eae3

5chdn added this to the 1.11 milestone

rphmeier reviewed

View reviewed changes

parity/export_hardcoded_sync.rs Outdated

+              	let on_demand = Arc::new(::light::on_demand::OnDemand::new(cache.clone()));
+              	let sync_handle = Arc::new(RwLock::new(Weak::new()));
+              	let fetch = ::light_helpers::EpochFetch {

Contributor

rphmeier Mar 21, 2018

on_demand, sync, network etc. are not necessary to start here -- a NoOpEpochFetch (I think it already exists for tests) will be enough.

Contributor

rphmeier commented Mar 21, 2018

LGTM except for grumble


          Don't use any data fetch for the light service

bc7fc65

Contributor Author

tomaka commented Mar 21, 2018

r?

tomaka mentioned this pull request

Allow user to reset light client to a trusted block hash #7263

Closed

5chdn mentioned this pull request

Parity warp sync is no longer very warpy. #6372

Closed

5chdn reviewed

View reviewed changes

ethcore/res/ethereum/foundation.json

    
              		"header": "f90219a061d694007fbaca6e23e73e29c8c6a60099abc740ab7e27ae3cd1b9c6a47efef7a01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347945a0b54d5dc17e0aadc383d2db43b0a0d3e029c4ca0a2f1bdabc1a72737de1c22a76cacc8fc162366904f759a99db7d6d19efee3090a0ac5f5b236e8977928a2ce43c7569ea5a74919643cb0b06d7540407b1ea1298f0a04356ddc5d77c83923a6541260308be167386e0969a608a017770c9e38091cfcab90100a00010002001009080011010141088000004000080081100000a002023000002204204801204084000c000010008000000000880080020c0000440200460000290005010c01c80800080004800100406003380000400402040000028084002a80087000008090a00200100544020019580022000000306100a0080100084020006809000e80000010000254810002000000a240050014200002002c10809202030006422022000203012000241089300080400000009001021020200012410348500028290230408100302000000058c0000020c08c20480081040020260004008481000080000800010010060020000e00020002140100a8988000004400201870b9af4a66df8038350a8018379d54483799eba845ab0426d984554482e45544846414e532e4f52472d3231323134313232a05eeccc80302d8fecca48a47be03654b5a41b5e5f296f271f910ebae631124f518890074810024c6c2b",

              		"totalDifficulty": "3144406426008470895785",

              		"CHTs": [

              			"0x0eb474b7721727204978e92e27d31cddff56471911e424a4c8271c35f9c982cc",

Contributor

5chdn Mar 26, 2018 •

edited

Loading

Is this anyhow possible without adding 2500 CHTs to the chain spec?

Contributor Author

tomaka Mar 26, 2018

Hmmm, what would the alternative be? We're hardcoding headers in the client after all, we have to put them somewhere.

Contributor

rphmeier Mar 27, 2018

the alternative is to allow the client to be deployed w/o history or CHT roots and have the history fetched afterwards

rphmeier approved these changes

View reviewed changes

tomaka merged commit 0493161 into openethereum:master

tomaka deleted the hardcoded-sync branch

March 27, 2018 11:57

5chdn mentioned this pull request

Light clients broken on chains with genesis validator-set contract #8468

Closed

5chdn mentioned this pull request

Update hardcoded headers #8925

Merged

5chdn mentioned this pull request

Update light client hardcoded headers #9098

Merged

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

A0-pleasereview 🤓 M4-core ⛓