Bad blocks in ropsten with geth 1.8.10 #16884

LefterisJP · 2018-06-04T12:54:19Z

System information

Geth version: 1.8.10
OS & Version: Linux
Commit hash : eae63c5

Expected behaviour

Ropsten should be syncing fine

Actual behaviour

Getting many bad blocks while syncing ropsten

Steps to reproduce the behaviour

Just sync ropsten with latest geth

Backtrace

########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: <nil> DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: <nil> Engine: ethash}                                                                                                                     

Number: 3371497
Hash: 0x1ef79ec54470f7f06782c4dacce290b1a4c298275d9ce11dc157a3ab06e8ee50


Error: invalid chain id for signer
##############################

ERROR[06-04|14:47:40]
########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: <nil> DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: <nil> Engine: ethash}                                                                                                                     

Number: 3371497
Hash: 0x1ef79ec54470f7f06782c4dacce290b1a4c298275d9ce11dc157a3ab06e8ee50


Error: invalid chain id for signer
##############################

INFO [06-04|14:47:42] Imported new chain segment               blocks=1   txs=186  mgas=11.565  elapsed=89.151ms  mgasps=129.726  number=3371497 hash=aba68d…110593 cache=14.13mB                                                                                               
ERROR[06-04|14:47:45]
########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: <nil> DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: <nil> Engine: ethash}                                                                                                                     

Number: 3371497
Hash: 0x1ef79ec54470f7f06782c4dacce290b1a4c298275d9ce11dc157a3ab06e8ee50


Error: invalid chain id for signer
##############################

INFO [06-04|14:47:50] Imported new chain segment               blocks=1   txs=74   mgas=2.604   elapsed=37.660ms  mgasps=69.134   number=3371498 hash=8024a4…89d922 cache=14.18mB                                                                                               
INFO [06-04|14:48:04] Imported new chain segment               blocks=1   txs=133  mgas=4.712   elapsed=56.762ms  mgasps=83.006   number=3371499 hash=8e95a2…b84a19 cache=14.23mB                                                                                               
INFO [06-04|14:48:10] Imported new chain segment               blocks=1   txs=120  mgas=5.254   elapsed=61.447ms  mgasps=85.500   number=3371500 hash=7e09f2…683f5e cache=14.20mB                                                                                               
INFO [06-04|14:48:14] Imported new chain segment               blocks=1   txs=72   mgas=3.795   elapsed=44.078ms  mgasps=86.099   number=3371501 hash=1bfd40…23fbde cache=14.19mB                                                                                               
INFO [06-04|14:48:33] Imported new chain segment               blocks=1   txs=101  mgas=3.566   elapsed=46.819ms  mgasps=76.174   number=3371502 hash=1acae5…516cbf cache=14.17mB                                                                                               
INFO [06-04|14:48:37] Imported new chain segment               blocks=1   txs=156  mgas=7.893   elapsed=65.139ms  mgasps=121.178  number=3371503 hash=64fb66…8655b0 cache=14.28mB                                                                                               
INFO [06-04|14:48:58] Imported new chain segment               blocks=1   txs=74   mgas=2.719   elapsed=35.730ms  mgasps=76.088   number=3371504 hash=ede996…8586af cache=14.16mB                                                                                               
INFO [06-04|14:49:13] Imported new chain segment               blocks=1   txs=218  mgas=9.054   elapsed=94.840ms  mgasps=95.461   number=3371505 hash=1c1d27…13527b cache=14.27mB                                                                                               
INFO [06-04|14:49:20] Imported new chain segment               blocks=1   txs=161  mgas=6.775   elapsed=89.188ms  mgasps=75.959   number=3371506 hash=a426ca…20b6a5 cache=14.38mB                                                                                               
> INFO [06-04|14:50:01] Imported new chain segment               blocks=1   txs=149  mgas=9.764   elapsed=68.043ms  mgasps=143.501  number=3371507 hash=9cd121…cd2182 cache=14.34mB                                                                                             
INFO [06-04|14:50:02] Imported new chain segment               blocks=1   txs=314  mgas=12.543  elapsed=173.178ms mgasps=72.428   number=3371508 hash=decd65…c10dcc cache=14.52mB                                                                                               
INFO [06-04|14:50:12] Imported new chain segment               blocks=1   txs=36   mgas=1.346   elapsed=23.758ms  mgasps=56.651   number=3371509 hash=d71812…7404f4 cache=14.43mB                                                                                               
INFO [06-04|14:50:23] Imported new chain segment               blocks=1   txs=266  mgas=10.137  elapsed=109.795ms mgasps=92.325   number=3371510 hash=6b82d6…f3ca8b cache=14.58mB                                                                                               
INFO [06-04|14:50:49] Imported new chain segment               blocks=1   txs=57   mgas=2.173   elapsed=32.477ms  mgasps=66.912   number=3371511 hash=b39aa2…1ee846 cache=14.55mB                                                                                               
INFO [06-04|14:51:01] Imported new chain segment               blocks=1   txs=329  mgas=11.758  elapsed=139.093ms mgasps=84.535   number=3371512 hash=e5e5f2…2b3f27 cache=14.72mB                                                                                               
INFO [06-04|14:51:03] Imported new chain segment               blocks=1   txs=80   mgas=4.000   elapsed=42.996ms  mgasps=93.029   number=3371513 hash=950146…941f24 cache=14.73mB                                                                                               
INFO [06-04|14:51:04] Imported new chain segment               blocks=1   txs=80   mgas=4.000   elapsed=39.677ms  mgasps=100.812  number=3371513 hash=1c0f14…d9a8c6 cache=14.73mB                                                                                               
INFO [06-04|14:51:33] Imported new chain segment               blocks=1   txs=11   mgas=0.371   elapsed=9.758ms   mgasps=38.024   number=3371514 hash=690428…2e9e19 cache=14.69mB                                                                                               
INFO [06-04|14:51:39] Imported new chain segment               blocks=1   txs=367  mgas=14.771  elapsed=153.638ms mgasps=96.140   number=3371515 hash=6897de…7b17db cache=14.92mB                                                                                               
INFO [06-04|14:51:51] Imported new chain segment               blocks=1   txs=34   mgas=1.260   elapsed=20.999ms  mgasps=59.986   number=3371516 hash=d4ce2e…76790b cache=14.93mB                                                                                               
ERROR[06-04|14:52:03]
########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: <nil> DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: <nil> Engine: ethash}                                                                                                                     

Number: 3371517
Hash: 0x177d8d483ad8cb4572f044e651947f73b0b1b7e9f36486fe5781ecbf9ec61de3


Error: invalid chain id for signer
##############################

ERROR[06-04|14:52:03]
########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: <nil> DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: <nil> Engine: ethash}                                                                                                                     

Number: 3371517
Hash: 0x177d8d483ad8cb4572f044e651947f73b0b1b7e9f36486fe5781ecbf9ec61de3


Error: invalid chain id for signer
##############################

INFO [06-04|14:52:30] Imported new chain segment               blocks=1   txs=264  mgas=11.392  elapsed=107.382ms mgasps=106.091  number=3371517 hash=2af6f5…c0c989 cache=15.01mB                                                                                               
INFO [06-04|14:53:31] Imported new chain segment               blocks=1   txs=185  mgas=18.719  elapsed=90.793ms  mgasps=206.168  number=3371518 hash=8849b7…f96a9d cache=15.22mB

The text was updated successfully, but these errors were encountered:

ldeffenb · 2018-06-04T14:42:36Z

Here's another pair of instances that I received. This seems to have started sometime yesterday.
########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: Engine: ethash}

Number: 3372000
Hash: 0x05f3954d79af17cebeb78e752986560bf3a9c41bbade247646d0a96c84541e0e

Error: invalid chain id for signer
##############################

ERROR[06-04|10:41:10]
########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: Engine: ethash}

Number: 3372000
Hash: 0x05f3954d79af17cebeb78e752986560bf3a9c41bbade247646d0a96c84541e0e

Error: invalid chain id for signer
##############################

bradlucas · 2018-06-05T02:19:07Z

I am seeing similar blocks.
geth version

Geth
Version: 1.8.11-unstable
Git Commit: c8dcb95
Architecture: amd64
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.9.2
Operating System: linux
GOPATH=
GOROOT=/usr/lib/golang

Backtrace

ERROR[06-05|02:15:19]
########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: Engine: ethash}

Number: 3375171
Hash: 0x276eb87f179c1b138952959ae3dac73f4e350ce89296ccc7ea3d6161e8f39102

Error: invalid chain id for signer
##############################

ERROR[06-05|02:15:20]
########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: Engine: ethash}

Number: 3375169
Hash: 0x14028a022e65c63e6e06cfc6d8aa877cb08066be63ac5fb51db4b1ddc59a1ec3

Error: invalid chain id for signer
##############################

doooyang · 2018-06-05T04:02:29Z

me too
these days i received so many bad blocks

########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: Engine: ethash}

Number: 3375634
Hash: 0xe6995541b2d0aeea64017a586cd50a5304c273252a059db86fdbfbc5b178b5b3

Error: invalid chain id for signer
##############################

ERROR[06-05|03:57:44]
########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: Engine: ethash}

Number: 3375634
Hash: 0xe6995541b2d0aeea64017a586cd50a5304c273252a059db86fdbfbc5b178b5b3

Error: invalid chain id for signer
##############################

ERROR[06-05|03:57:44]
########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: Engine: ethash}

Number: 3375634
Hash: 0xe6995541b2d0aeea64017a586cd50a5304c273252a059db86fdbfbc5b178b5b3

Error: invalid chain id for signer
##############################

holiman · 2018-06-05T07:44:16Z

It would be interesting if you could use the rpc to invoke debug_getBadBlocks (no arguments).

curl -X POST --data '{"jsonrpc":"2.0","method":"debug_getBadBlocks","params":[],"id":1}'

That would make it easier to investigate the actual blocks.

karalabe · 2018-06-05T10:15:21Z

Note that none of the blocks mentioned here end up on the canonical chain according to https://ropsten.etherscan.io. Could these be actual attack blocks trying to inject invalid data into the network?

LefterisJP · 2018-06-05T11:47:03Z

That would make it easier to investigate the actual blocks.

Let me fire up the ropsten node again and will provide an answer here soon.

LefterisJP · 2018-06-05T12:40:00Z

@holiman Here is the first bad block I just got:

curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"debug_getBadBlocks","params":[],"id":1}' http://localhost:8545
{"jsonrpc":"2.0","id":1,"result":[{"hash":"0x4d208db0540bb9b3e15736e69f152badf725a43e26bf0792669020cd689795cd","header":{"parentHash":"0x94333501131696d8d4e9bac4214cb5698e70dc1f944adffebcb89510fcf1dbf2","sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","miner":"0x1585936b53834b021f68cc13eeefdec2efc8e724","stateRoot":"0x2ca5a38f80e4191b583519eefff12e8a0533ff0486d78196f3cbe8c74aa645d6","transactionsRoot":"0xc3cb79aa67642581c2da37e1247a72c684ef570baf72d5bf8f9e99f77c050530","receiptsRoot":"0x1c061d94b4a3fba534415c0ea5a3d0eb250ea2b4f428ae4509f8279d9353e41d","logsBloom":"0x00000400004000000000000080500002100000000001020004000000000000004000010080100000000000200010010000000002400000010000020800000000804000400000020000000808000000000200000001800000000001001000800000000001000200004000000000000020000000000004040400000010000000000020002002000000000022000000001000000001080000000000000000000000000000000000000202000000000201000800000200000000000000000000000000000002000000000000000080000000001000000000080000000020000200000000210200080000000000004000000000080408000080000000004000020000","difficulty":"0x54c25d0a","number":"0x33892c","gasLimit":"0x4f0e504","gasUsed":"0x437a51","timestamp":"0x5b168322","extraData":"0x414952412f7630","mixHash":"0xcae410f6f38da3e138030de06f7091a008e777936d718e11a259120a7458c342","nonce":"0x70a695c0180a2071","hash":"0x4d208db0540bb9b3e15736e69f152badf725a43e26bf0792669020cd689795cd"}}]}

nikitaeverywhere · 2018-06-05T13:17:51Z

The same happened with all our testnet geth nodes yesterday:

ERROR[06-05|13:13:43] 
########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: &lt;nil&gt; DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Engine: ethash}
 Number: 3377452
Hash: 0x4d208db0540bb9b3e15736e69f152badf725a43e26bf0792669020cd689795cd
 Error: invalid chain id for signer
##############################

veox · 2018-06-05T14:10:54Z

I first started getting these around block 3364573:

INFO [06-03|12:22:56] Imported new chain segment               blocks=1   txs=13   mgas=0.323   elapsed=31.825ms  mgasps=10.142   number=3364571 hash=84870f…b0d3bb cache=20.02mB
INFO [06-03|12:23:05] Imported new chain segment               blocks=1   txs=20   mgas=2.317   elapsed=31.459ms  mgasps=73.665   number=3364572 hash=8eae6d…095dc6 cache=20.02mB
INFO [06-03|12:23:19] Imported new chain segment               blocks=1   txs=17   mgas=0.574   elapsed=31.991ms  mgasps=17.934   number=3364573 hash=216570…001aa8 cache=20.05mB
INFO [06-03|12:23:20] Imported new chain segment               blocks=1   txs=9    mgas=0.406   elapsed=30.252ms  mgasps=13.412   number=3364573 hash=74428d…f4819a cache=20.06mB
ERROR[06-03|12:23:29]
########## BAD BLOCK #########
Chain config: {ChainID: 3 Homestead: 0 DAO: <nil> DAOSupport: true EIP150: 0 EIP155: 10 EIP158: 10 Byzantium: 1700000 Constantinople: <nil> Engine: ethash}
Number: 3364574
Hash: 0x83be00825695ef37a71eb43b24c584f9f20191b2611f2b35cd363df76a77438c
Error: invalid chain id for signer
##############################

(Timezone is UTC+3.)

AFAIU Ropsten is now seeing network segmentation - a geth node of mine is refusing blocks from another parity node (also mine), with err="retrieved hash chain is invalid".

veox · 2018-06-05T14:19:01Z

JIC, here's the "bad blocks", according to the geth node (two of them).

karalabe · 2018-06-05T14:23:08Z

@veox Could you dump the transactions from the parity node for the offending blocks?

5chdn · 2018-06-05T14:30:27Z

Looks like we forked https://ropsten-stats.parity.io/

Etherscan seems to be running on the Parity chain.

karalabe · 2018-06-05T14:34:07Z

Can someone please post an offending transaction here?

veox · 2018-06-05T14:39:12Z

@karalabe Working on it...

veox · 2018-06-05T14:48:53Z

web3.py ~~dump of~~ output for block 0x4d208db0540bb9b3e15736e69f152badf725a43e26bf0792669020cd689795cd:

https://gist.github.com/veox/17d8abe9d103de181c5d43a508015719

karalabe · 2018-06-05T14:51:09Z

The transactions are not part of the header. Could you please create a dump of the transaction RLPs?

tomusdrw · 2018-06-05T14:53:59Z

@karalabe does geth allow non-replay protected (without chain_id) transactions on ropsten?

karalabe · 2018-06-05T14:55:32Z

@tomusdrw @veox @5chdn I'm fairly sure Parity is broken https://ropsten.etherscan.io/tx/0x595584e60f42bf02978856989553946dd73004789fa6df2bc3e3fa0367ca630f

The transaction originates from the 0xff address

5chdn · 2018-06-05T15:03:34Z

we are on it

veox · 2018-06-05T15:08:27Z

@karalabe Ah, I misunderstood.

So far, I got (JSON + the exception-raising unformattable that it was created from):

https://gist.github.com/veox/b430aa0597926e2b5f4734817d8cbfa5

karalabe · 2018-06-05T15:17:04Z

Closing as it's an issue in Parity.

ajunge · 2018-06-05T17:53:34Z

Any link to the parity issue?

jeremyschlatter · 2018-06-06T01:09:03Z

@ajunge I think this was it: openethereum/parity-ethereum#8802

ajunge · 2018-06-06T01:39:27Z

Thanx @jeremyschlatter

karalabe · 2018-06-06T12:51:40Z

Sorry for being unspecific yesterday, didn't want to expose the consensus issue ahead of time.

rstormsf · 2018-06-08T23:01:42Z

Explanation of what happened:
https://medium.com/@poanetwork/exploiting-consensus-vulnerability-in-a-parity-client-to-hard-fork-an-ethereum-based-network-d2c368bf0bac

karalabe closed this as completed Jun 5, 2018

ec2 mentioned this issue Jun 5, 2018

Ropsten out of sync? INFURA/infura#112

Closed

veox mentioned this issue Jun 7, 2018

middleware/pythonic: only apply formatter to 'publicKey' if not null. ethereum/web3.py#890

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bad blocks in ropsten with geth 1.8.10 #16884

Bad blocks in ropsten with geth 1.8.10 #16884

LefterisJP commented Jun 4, 2018

ldeffenb commented Jun 4, 2018

bradlucas commented Jun 5, 2018

doooyang commented Jun 5, 2018

holiman commented Jun 5, 2018

karalabe commented Jun 5, 2018

LefterisJP commented Jun 5, 2018

LefterisJP commented Jun 5, 2018

nikitaeverywhere commented Jun 5, 2018

veox commented Jun 5, 2018 •

edited

Loading

veox commented Jun 5, 2018

karalabe commented Jun 5, 2018

5chdn commented Jun 5, 2018

karalabe commented Jun 5, 2018

veox commented Jun 5, 2018

veox commented Jun 5, 2018 •

edited

Loading

karalabe commented Jun 5, 2018

tomusdrw commented Jun 5, 2018

karalabe commented Jun 5, 2018

5chdn commented Jun 5, 2018

veox commented Jun 5, 2018 •

edited

Loading

karalabe commented Jun 5, 2018

ajunge commented Jun 5, 2018

jeremyschlatter commented Jun 6, 2018

ajunge commented Jun 6, 2018

karalabe commented Jun 6, 2018

rstormsf commented Jun 8, 2018

Bad blocks in ropsten with geth 1.8.10 #16884

Bad blocks in ropsten with geth 1.8.10 #16884

Comments

LefterisJP commented Jun 4, 2018

System information

Expected behaviour

Actual behaviour

Steps to reproduce the behaviour

Backtrace

ldeffenb commented Jun 4, 2018

bradlucas commented Jun 5, 2018

doooyang commented Jun 5, 2018

holiman commented Jun 5, 2018

karalabe commented Jun 5, 2018

LefterisJP commented Jun 5, 2018

LefterisJP commented Jun 5, 2018

nikitaeverywhere commented Jun 5, 2018

veox commented Jun 5, 2018 • edited Loading

veox commented Jun 5, 2018

karalabe commented Jun 5, 2018

5chdn commented Jun 5, 2018

karalabe commented Jun 5, 2018

veox commented Jun 5, 2018

veox commented Jun 5, 2018 • edited Loading

karalabe commented Jun 5, 2018

tomusdrw commented Jun 5, 2018

karalabe commented Jun 5, 2018

5chdn commented Jun 5, 2018

veox commented Jun 5, 2018 • edited Loading

karalabe commented Jun 5, 2018

ajunge commented Jun 5, 2018

jeremyschlatter commented Jun 6, 2018

ajunge commented Jun 6, 2018

karalabe commented Jun 6, 2018

rstormsf commented Jun 8, 2018

veox commented Jun 5, 2018 •

edited

Loading

veox commented Jun 5, 2018 •

edited

Loading

veox commented Jun 5, 2018 •

edited

Loading