Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chifra init all not working #3531

Closed
cevatbostancioglu opened this issue Mar 5, 2024 · 24 comments
Closed

chifra init all not working #3531

cevatbostancioglu opened this issue Mar 5, 2024 · 24 comments
Labels
bug TC-apps:init

Comments

@cevatbostancioglu
Copy link

$ chifra init --all
WARN[05-03|07:40:35.983] the --call value provided (manifestHashMap(0x0, "mainnet-ts")) was not found: abi not found for manifestHashMap(0x0, "mainnet-ts"): abi not found
Error: the --call value provided (manifestHashMap(0x0, "mainnet")) was not found: abi not found for manifestHashMap(0x0, "mainnet"): abi not found
Usage:
chifra init [flags]

Flags:
-a, --all in addition to Bloom filters, download full index chunks (recommended)
-d, --dry_run display the results of the download without actually downloading
-F, --first_block uint do not download any chunks earlier than this block
-s, --sleep float seconds to sleep between downloads
-v, --verbose enable verbose output
-h, --help display this help screen

Notes:

  • If run with no options, this tool will download or freshen only the Bloom filters.
  • The --first_block option will fall back to the start of the containing chunk.
  • You may re-run the tool as often as you wish. It will repair or freshen the index.
    #ubuntu22.04
    everything configured.
    i am able to run chifra blocks 12
@tjayrush
Copy link
Member

tjayrush commented Mar 5, 2024

You must have a valid rpcProvider for the Ethereum mainnet chain for chifra init to work properly. The error message is admittedly poor. When you try to run chifra status --chain mainnet, what result do you get?

@cevatbostancioglu
Copy link
Author

/opt/etl$ chifra status --chain mainnet
WARN[05-03|22:05:29.995] the --call value provided (manifestHashMap(0x0, "mainnet-ts")) was not found: abi not found for manifestHashMap(0x0, "mainnet-ts"): abi not found
05-03|22:05:30.050 Client: erigon/2.49.1/linux-amd64/go1.19.13 (archive, tracing)
05-03|22:05:30.050 TrueBlocks: GHC-TrueBlocks//2.5.8-release (eskey, no pinkey)
05-03|22:05:30.050 RPC Provider: http://localhost:8545 - mainnet (1/1)
05-03|22:05:30.050 Root Config Path: /home/cevat/.local/share/trueblocks/
05-03|22:05:30.050 Chain Config Path: /home/cevat/.local/share/trueblocks/config/mainnet/
05-03|22:05:30.050 Cache Path: /home/cevat/.local/share/trueblocks/cache/mainnet/
05-03|22:05:30.050 Index Path: /home/cevat/.local/share/trueblocks/unchained/mainnet/
05-03|22:05:30.050 Progress: 16999999, 0, 0, 0 ts: 0

@cevatbostancioglu
Copy link
Author

$erigon --datadir=./ethereum_erigon --chain=mainnet --http.api=engine,eth,erigon,web3,trace,net,debug

@tjayrush
Copy link
Member

tjayrush commented Mar 6, 2024

So, it looks like you're starting your node right, but it's not visible to chifra. To eliminate chifra from the equation, try querying the node direct from the same machine that you ran chifra status from.

Try this:

curl -X POST -H "Content-Type: application/json" --data '{ "jsonrpc": "2.0", "method": "eth_getBlockByNumber", "params": [ "0xc", true ], "id": 1 }' "http://localhost:8545"

I'm not certain, of course, but I don't think this will work either. If it does not, then neither will chifra (since chifra is using that RPC provider).

If it does work, then we may have a different problem.

@cevatbostancioglu
Copy link
Author

/opt$ curl -X POST -H "Content-Type: application/json" --data '{ "jsonrpc": "2.0", "method": "eth_getBlockByNumber", "params": [ "0xc", true ], "id": 1 }' "http://localhost:8545"
{"jsonrpc":"2.0","id":1,"result":{"difficulty":"0x3ffffa000","extraData":"0x476574682f76312e302e302f6c696e75782f676f312e342e32","gasLimit":"0x1388","gasUsed":"0x0","hash":"0xc63f666315fa1eae17e354fab532aeeecf549be93e358737d0648f50d57083a0","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","miner":"0x0193d941b50d91be6567c7ee1c0fe7af498b4137","mixHash":"0xbe4ba21fe1ecb061e44f178428c772d2a0f59a7aafb5ed4e198eba4df3656e52","nonce":"0x5f6a5cc5c36e6627","number":"0xc","parentHash":"0x3f5e756c3efcb93099361b7ddd0dabfeaa592439437c1c836e443ccb81e93242","receiptsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","size":"0x219","stateRoot":"0x821c41f30a2fd9580605363784a8a2a6575b255ec37cacf87fe52715b8828d8e","timestamp":"0x55ba42c0","totalDifficulty":"0x33f2ffe033","transactions":[],"transactionsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","uncles":[]}}

i am not completly sure it worked or not.

@tjayrush
Copy link
Member

tjayrush commented Mar 7, 2024

I get this:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "difficulty": "0x3ffffa000",
    "extraData": "0x476574682f76312e302e302f6c696e75782f676f312e342e32",
    "gasLimit": "0x1388",
    "gasUsed": "0x0",
    "hash": "0xc63f666315fa1eae17e354fab532aeeecf549be93e358737d0648f50d57083a0",
    "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
    "miner": "0x0193d941b50d91be6567c7ee1c0fe7af498b4137",
    "mixHash": "0xbe4ba21fe1ecb061e44f178428c772d2a0f59a7aafb5ed4e198eba4df3656e52",
    "nonce": "0x5f6a5cc5c36e6627",
    "number": "0xc",
    "parentHash": "0x3f5e756c3efcb93099361b7ddd0dabfeaa592439437c1c836e443ccb81e93242",
    "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
    "sha3Uncles": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
    "size": "0x219",
    "stateRoot": "0x821c41f30a2fd9580605363784a8a2a6575b255ec37cacf87fe52715b8828d8e",
    "timestamp": "0x55ba42c0",
    "totalDifficulty": "0x33f2ffe033",
    "transactions": [],
    "transactionsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
    "uncles": []
  }
}

So your node is working, but now I'm curious if you're running chifra and that curl command you just ran on the same exact physical machine.

@tjayrush
Copy link
Member

tjayrush commented Mar 7, 2024

But....and this is a BIG but....

I'm just now noticing that your node is only synced to block 16999999. This is a problem because the unchained index is now being published by a different publisher...which means that it may not find the record it's looking for.

Try this:

  1. Either wait until your node syncs to the head of the chain (you can tell how far along it is using a curl comment with eth_blocknumber), or
  2. Add --publisher 0xf503017d7baf7fbc0fff7492b751025c6a78179b to your query.

I would suggest (1), but (2) may work. I'd be interested to learn if it does.

@tjayrush
Copy link
Member

tjayrush commented Mar 7, 2024 •
edited
Loading

When I do:

curl http://localhost:23456 -X POST -H "Content-Type: application/json" --data '{"method":"eth_blockNumber","params":[],"id":1,"jsonrpc":"2.0"}'

I get:

{
  "jsonrpc":"2.0",
  "id":1,
  "result":"0x127b71c"
}

which is block 19379996. The chifra status run above return 16999999 as latest block. I think your node is stuck.

@cevatbostancioglu
Copy link
Author

Okay, it seems after eth merge, i also need to run consensus layer client as well. i am trying it with erigon internal consensus layer client. i am seeing different logs this time.
INFO[03-07|19:27:55.482] [1/12 Snapshots] download progress="65.09% 321.9GB/494.5GB" time-left=8hrs:4m total-time=2m20s download=6.1MB/s upload=0B/s peers=21 files=138 metadata=138/138 connections=26 alloc=6.9GB sys=7.3GB
i will let you know when its done. thanks for your help.

@tjayrush
Copy link
Member

tjayrush commented Mar 7, 2024

Okay, it seems after eth merge, i also need to run consensus layer client as well.

I thought this might be the problem, but I wasn't sure if block 6999999 was the right block, so I didn't want to confuse things. Yes. After the merge you must run two clients. CL (like Prysm) and EL (like Erigon). Sorry it took so long to get to this point...

Happy to help. If you have any more questions, please let me know.

@cevatbostancioglu
Copy link
Author

/opt/etl$ curl -X POST -H "Content-Type: application/json" --data '{ "jsonrpc": "2.0", "method": "eth_getBlockByNumber", "params": [ "0xc", true ], "id": 1 }' "http://localhost:8545"
{"jsonrpc":"2.0","id":1,"result":{"difficulty":"0x3ffffa000","extraData":"0x476574682f76312e302e302f6c696e75782f676f312e342e32","gasLimit":"0x1388","gasUsed":"0x0","hash":"0xc63f666315fa1eae17e354fab532aeeecf549be93e358737d0648f50d57083a0","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","miner":"0x0193d941b50d91be6567c7ee1c0fe7af498b4137","mixHash":"0xbe4ba21fe1ecb061e44f178428c772d2a0f59a7aafb5ed4e198eba4df3656e52","nonce":"0x5f6a5cc5c36e6627","number":"0xc","parentHash":"0x3f5e756c3efcb93099361b7ddd0dabfeaa592439437c1c836e443ccb81e93242","receiptsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","size":"0x219","stateRoot":"0x821c41f30a2fd9580605363784a8a2a6575b255ec37cacf87fe52715b8828d8e","timestamp":"0x55ba42c0","totalDifficulty":"0x33f2ffe033","transactions":[],"transactionsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","uncles":[]}}

opt/etl$ chifra status --chain mainnet
WARN[08-03|16:00:55.353] the --call value provided (manifestHashMap(0x0, "mainnet-ts")) was not found: abi not found for manifestHashMap(0x0, "mainnet-ts"): abi not found
08-03|16:00:55.405 Client: erigon/2.58.1/linux-amd64/go1.22.0 (archive, tracing)
08-03|16:00:55.405 TrueBlocks: GHC-TrueBlocks//2.5.8-release (eskey, no pinkey)
08-03|16:00:55.405 RPC Provider: http://localhost:8545 - mainnet (1/1)
08-03|16:00:55.405 Root Config Path: /home/cevat/.local/share/trueblocks/
08-03|16:00:55.405 Chain Config Path: /home/cevat/.local/share/trueblocks/config/mainnet/
08-03|16:00:55.405 Cache Path: /home/cevat/.local/share/trueblocks/cache/mainnet/
08-03|16:00:55.405 Index Path: /home/cevat/.local/share/trueblocks/unchained/mainnet/
08-03|16:00:55.405 Progress: 17819982, 0, 0, 0 ts: 0

/opt/etl$ chifra init --all
WARN[08-03|16:02:02.748] the --call value provided (manifestHashMap(0x0, "mainnet-ts")) was not found: abi not found for manifestHashMap(0x0, "mainnet-ts"): abi not found
Error: the --call value provided (manifestHashMap(0x0, "mainnet")) was not found: abi not found for manifestHashMap(0x0, "mainnet"): abi not found
Usage:
chifra init [flags]

Flags:
-a, --all in addition to Bloom filters, download full index chunks (recommended)
-d, --dry_run display the results of the download without actually downloading
-F, --first_block uint do not download any chunks earlier than this block
-s, --sleep float seconds to sleep between downloads
-v, --verbose enable verbose output
-h, --help display this help screen

Notes:

  • If run with no options, this tool will download or freshen only the Bloom filters.
  • The --first_block option will fall back to the start of the containing chunk.
  • You may re-run the tool as often as you wish. It will repair or freshen the index.
    still similar issues. do i need to wait until erigon is done with up to latest block ?

@tjayrush
Copy link
Member

tjayrush commented Mar 8, 2024

Okay. So you're making progress syncing...I now see block 17,819,982. That's good.

Now we've encountered a real bug in our code...the "publisher" of the index (which is what chifra init is looking for) didn't exist on-chain until block 18,538,072.

Once your chain is caught up that far, I would hope chifra init will start working...

If not, then we have a more serious bug...

I did write this issue (#3533) but since it only happens if the end user's node is not synced to block 18,538,072 it won't get attention right away...but it will get fixed...

Upshot: wait just a little bit more until your node is further synced and it should work...

@cevatbostancioglu
Copy link
Author

/opt/etl$ chifra init --all
INFO[09-03|11:05:55.803] Chain: mainnet
INFO[09-03|11:05:55.803] Database: mainnet-ts
INFO[09-03|11:05:55.803] Gateway: https://ipfs.unchainedindex.io/ipfs/
INFO[09-03|11:05:55.803] URL: https://ipfs.unchainedindex.io/ipfs/QmVXxiYH7P2qJjcPXQtUVWPrP2d1jJWiRjwVq2DNtBGTtu
INFO[09-03|11:05:55.803] CID: QmVXxiYH7P2qJjcPXQtUVWPrP2d1jJWiRjwVq2DNtBGTtu
INFO[09-03|11:05:55.803] Downloading database mainnet-ts (QmVXxiYH7P2qJjcPXQtUVWPrP2d1jJWiRjwVq2DNtBGTtu). This may take a moment...
INFO[09-03|11:05:57.536] Downloading complete mainnet-ts (QmVXxiYH7P2qJjcPXQtUVWPrP2d1jJWiRjwVq2DNtBGTtu). Writing file...
PROG[09-03|11:06:06.023] Downloading timestamps. Please wait... 30%

it seems started to work.

@cevatbostancioglu
Copy link
Author

cevatbostancioglu commented Mar 9, 2024 •
edited
Loading

EROR[09-03|11:08:56.988] write to disc error [error copying 018395287-018398277 file in writeBytesToDisc: [stream error: stream ID 153; INTERNAL_ERROR; received from peer]]

and

EROR[09-03|13:34:23.416] write to disc error [error copying 015851886-015854901 file in writeBytesToDisc: [stream error: stream ID 9; PROTOCOL_ERROR; received from peer]]

is this normal

@tjayrush
Copy link
Member

tjayrush commented Mar 9, 2024

No. Try adding --sleep 0.5.

@cevatbostancioglu
Copy link
Author

i tried 0.5, 1, 2. it seems bloom filters cannot be downloaded from ipfs server. is scrape only option left?
INFO[10-03|20:51:14.082] Finished download of bloom 017234588-017237505 ( 3790 of 7487 50.6% [sleep: 1157.41ms])
INFO[10-03|20:51:15.239] Finished download of bloom 017231681-017234587 ( 3791 of 7487 50.6% [sleep: 1157.41ms])
INFO[10-03|20:51:16.397] Completed initializing bloom files.
EROR[10-03|20:51:19.393] write to disc error [error copying 016840377-016843387 file in writeBytesToDisc: [stream error: stream ID 31; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:51:19.402] Failed download 016600001-016602866 (will retry)
EROR[10-03|20:51:27.395] write to disc error [error copying 016837326-016840376 file in writeBytesToDisc: [stream error: stream ID 1; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:51:27.405] Failed download 016597169-016599853 (will retry)
EROR[10-03|20:51:35.396] write to disc error [error copying 016834332-016837325 file in writeBytesToDisc: [stream error: stream ID 5; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:51:35.402] Failed download 016594276-016597168 (will retry)
EROR[10-03|20:51:43.396] write to disc error [error copying 016831401-016834331 file in writeBytesToDisc: [stream error: stream ID 9; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:51:43.410] Failed download 016591552-016594275 (will retry)
EROR[10-03|20:51:51.398] write to disc error [error copying 016828414-016831400 file in writeBytesToDisc: [stream error: stream ID 13; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:51:51.411] Failed download 016588645-016591551 (will retry)
EROR[10-03|20:51:59.399] write to disc error [error copying 016825606-016828413 file in writeBytesToDisc: [stream error: stream ID 17; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:51:59.413] Failed download 016585754-016588644 (will retry)
EROR[10-03|20:52:07.401] write to disc error [error copying 016822542-016825605 file in writeBytesToDisc: [stream error: stream ID 23; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:52:07.415] Failed download 016583046-016585753 (will retry)
EROR[10-03|20:52:15.402] write to disc error [error copying 016819722-016822541 file in writeBytesToDisc: [stream error: stream ID 27; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:52:15.417] Failed download 016580220-016583045 (will retry)
EROR[10-03|20:52:23.404] write to disc error [error copying 016817025-016819721 file in writeBytesToDisc: [stream error: stream ID 31; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:52:23.419] Failed download 016577453-016580219 (will retry)
EROR[10-03|20:52:31.406] write to disc error [error copying 016814142-016817024 file in writeBytesToDisc: [stream error: stream ID 35; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:52:31.413] Failed download 016574764-016577452 (will retry)
EROR[10-03|20:52:39.406] write to disc error [error copying 016811188-016814141 file in writeBytesToDisc: [stream error: stream ID 3; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:52:39.420] Failed download 016571968-016574763 (will retry)
EROR[10-03|20:52:47.408] write to disc error [error copying 016808019-016811187 file in writeBytesToDisc: [stream error: stream ID 7; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:52:47.422] Failed download 016569186-016571967 (will retry)
EROR[10-03|20:52:55.408] write to disc error [error copying 016805244-016808018 file in writeBytesToDisc: [stream error: stream ID 13; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:52:55.422] Failed download 016566142-016569185 (will retry)
EROR[10-03|20:53:03.409] write to disc error [error copying 016802687-016805243 file in writeBytesToDisc: [stream error: stream ID 17; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:53:03.421] Failed download 016563242-016566141 (will retry)
EROR[10-03|20:53:11.410] write to disc error [error copying 016800001-016802686 file in writeBytesToDisc: [stream error: stream ID 21; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:53:11.417] Failed download 016560201-016563241 (will retry)
EROR[10-03|20:53:19.411] write to disc error [error copying 016797685-016800000 file in writeBytesToDisc: [stream error: stream ID 25; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:53:19.421] Failed download 016557205-016560200 (will retry)
EROR[10-03|20:53:27.413] write to disc error [error copying 016794733-016797684 file in writeBytesToDisc: [stream error: stream ID 29; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:53:27.427] Failed download 016554304-016557204 (will retry)
EROR[10-03|20:53:35.413] write to disc error [error copying 016791878-016794732 file in writeBytesToDisc: [stream error: stream ID 35; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:53:35.426] Failed download 016551179-016554303 (will retry)
EROR[10-03|20:53:43.415] write to disc error [error copying 016788867-016791877 file in writeBytesToDisc: [stream error: stream ID 41; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:53:43.422] Failed download 016548288-016551178 (will retry)
EROR[10-03|20:53:51.416] write to disc error [error copying 016785835-016788866 file in writeBytesToDisc: [stream error: stream ID 45; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:53:51.424] Failed download 016545297-016548287 (will retry)
EROR[10-03|20:53:59.417] write to disc error [error copying 016782892-016785834 file in writeBytesToDisc: [stream error: stream ID 5; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:53:59.431] Failed download 016542435-016545296 (will retry)
EROR[10-03|20:54:07.418] write to disc error [error copying 016779710-016782891 file in writeBytesToDisc: [stream error: stream ID 11; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:54:07.427] Failed download 016539584-016542434 (will retry)
EROR[10-03|20:54:15.419] write to disc error [error copying 016776765-016779709 file in writeBytesToDisc: [stream error: stream ID 17; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:54:15.426] Failed download 016536670-016539583 (will retry)
EROR[10-03|20:54:23.420] write to disc error [error copying 016773635-016776764 file in writeBytesToDisc: [stream error: stream ID 23; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:54:23.434] Failed download 016533838-016536669 (will retry)
EROR[10-03|20:54:31.422] write to disc error [error copying 016770620-016773634 file in writeBytesToDisc: [stream error: stream ID 29; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:54:31.434] Failed download 016530711-016533837 (will retry)
EROR[10-03|20:54:39.423] write to disc error [error copying 016767607-016770619 file in writeBytesToDisc: [stream error: stream ID 35; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:54:39.431] Failed download 016527792-016530710 (will retry)
EROR[10-03|20:54:47.424] write to disc error [error copying 016764460-016767606 file in writeBytesToDisc: [stream error: stream ID 41; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:54:47.430] Failed download 016524793-016527791 (will retry)
EROR[10-03|20:54:55.424] write to disc error [error copying 016761286-016764459 file in writeBytesToDisc: [stream error: stream ID 47; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:54:55.431] Failed download 016521773-016524792 (will retry)
EROR[10-03|20:55:03.426] write to disc error [error copying 016757987-016761285 file in writeBytesToDisc: [stream error: stream ID 3; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:55:03.433] Failed download 016518928-016521772 (will retry)
EROR[10-03|20:55:11.428] write to disc error [error copying 016754942-016757986 file in writeBytesToDisc: [stream error: stream ID 9; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:55:11.441] Failed download 016515804-016518927 (will retry)
EROR[10-03|20:55:19.428] write to disc error [error copying 016751643-016754941 file in writeBytesToDisc: [stream error: stream ID 17; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:55:19.442] Failed download 016512916-016515803 (will retry)
EROR[10-03|20:55:27.429] write to disc error [error copying 016748649-016751642 file in writeBytesToDisc: [stream error: stream ID 25; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:55:27.446] Failed download 016509570-016512915 (will retry)
EROR[10-03|20:55:35.431] write to disc error [error copying 016745701-016748648 file in writeBytesToDisc: [stream error: stream ID 31; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:55:35.444] Failed download 016506462-016509569 (will retry)
EROR[10-03|20:55:43.431] write to disc error [error copying 016742773-016745700 file in writeBytesToDisc: [stream error: stream ID 39; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:55:43.438] Failed download 016503129-016506461 (will retry)
EROR[10-03|20:55:51.431] write to disc error [error copying 016739901-016742772 file in writeBytesToDisc: [stream error: stream ID 49; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:55:51.439] Failed download 016500001-016503128 (will retry)
EROR[10-03|20:55:59.432] write to disc error [error copying 016736863-016739900 file in writeBytesToDisc: [stream error: stream ID 57; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:55:59.440] Failed download 016499108-016500000 (will retry)
EROR[10-03|20:56:07.433] write to disc error [error copying 016733943-016736862 file in writeBytesToDisc: [stream error: stream ID 3; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:56:07.448] Failed download 016496045-016499107 (will retry)
EROR[10-03|20:56:15.434] write to disc error [error copying 016730887-016733942 file in writeBytesToDisc: [stream error: stream ID 13; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:56:15.449] Failed download 016493010-016496044 (will retry)
EROR[10-03|20:56:23.436] write to disc error [error copying 016727874-016730886 file in writeBytesToDisc: [stream error: stream ID 23; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:56:23.450] Failed download 016489997-016493009 (will retry)
EROR[10-03|20:56:31.437] write to disc error [error copying 016724878-016727873 file in writeBytesToDisc: [stream error: stream ID 33; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:56:31.451] Failed download 016486866-016489996 (will retry)
EROR[10-03|20:56:39.438] write to disc error [error copying 016721713-016724877 file in writeBytesToDisc: [stream error: stream ID 45; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:56:39.453] Failed download 016483990-016486865 (will retry)
EROR[10-03|20:56:47.440] write to disc error [error copying 016718768-016721712 file in writeBytesToDisc: [stream error: stream ID 59; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:56:47.454] Failed download 016480821-016483989 (will retry)
EROR[10-03|20:56:55.440] write to disc error [error copying 016715515-016718767 file in writeBytesToDisc: [stream error: stream ID 73; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:56:55.456] Failed download 016477911-016480820 (will retry)
EROR[10-03|20:57:03.441] write to disc error [error copying 016712508-016715514 file in writeBytesToDisc: [stream error: stream ID 89; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:57:03.452] Failed download 016474811-016477910 (will retry)
EROR[10-03|20:57:11.442] write to disc error [error copying 016709244-016712507 file in writeBytesToDisc: [stream error: stream ID 105; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:57:11.456] Failed download 016471606-016474810 (will retry)
EROR[10-03|20:57:19.442] write to disc error [error copying 016706187-016709243 file in writeBytesToDisc: [stream error: stream ID 7; INTERNAL_ERROR; received from peer]]
WARN[10-03|20:57:19.457] Failed download 016468487-016471605 (will retry)
EROR[10-03|20:57:27.444] write to disc error [error copying 016703037-016706186 file in writeBytesToDisc: [stream error: stream ID 31; INTERNAL_ERROR; received from peer]]
WARN[10-03|20:57:27.461] Failed download 016464940-016468486 (will retry)
EROR[10-03|20:57:35.446] write to disc error [error copying 016700001-016703036 file in writeBytesToDisc: [stream error: stream ID 59; INTERNAL_ERROR; received from peer]]
WARN[10-03|20:57:35.460] Failed download 016461744-016464939 (will retry)
EROR[10-03|20:57:43.448] write to disc error [error copying 016697847-016700000 file in writeBytesToDisc: [stream error: stream ID 97; INTERNAL_ERROR; received from peer]]
WARN[10-03|20:57:43.462] Failed download 016457990-016461743 (will retry)
EROR[10-03|20:57:51.449] write to disc error [error copying 016694905-016697846 file in writeBytesToDisc: [stream error: stream ID 151; INTERNAL_ERROR; received from peer]]
WARN[10-03|20:57:51.455] Failed download 016454941-016457989 (will retry)
EROR[10-03|20:57:59.450] write to disc error [error copying 016692091-016694904 file in writeBytesToDisc: [stream error: stream ID 257; INTERNAL_ERROR; received from peer]]
WARN[10-03|20:57:59.464] Failed download 016451745-016454940 (will retry)
EROR[10-03|20:58:07.452] write to disc error [error copying 016689488-016692090 file in writeBytesToDisc: [stream error: stream ID 7; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:58:07.460] Failed download 016448874-016451744 (will retry)
EROR[10-03|20:58:15.453] write to disc error [error copying 016686871-016689487 file in writeBytesToDisc: [stream error: stream ID 677; INTERNAL_ERROR; received from peer]]
WARN[10-03|20:58:15.467] Failed download 016445471-016448873 (will retry)
EROR[10-03|20:58:23.455] write to disc error [error copying 016684076-016686870 file in writeBytesToDisc: [stream error: stream ID 27; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:58:23.469] Failed download 016442369-016445470 (will retry)
EROR[10-03|20:58:31.456] write to disc error [error copying 016681262-016684075 file in writeBytesToDisc: [stream error: stream ID 949; INTERNAL_ERROR; received from peer]]
WARN[10-03|20:58:31.470] Failed download 016439177-016442368 (will retry)
EROR[10-03|20:58:39.457] write to disc error [error copying 016678406-016681261 file in writeBytesToDisc: [stream error: stream ID 1119; INTERNAL_ERROR; received from peer]]
WARN[10-03|20:58:39.465] Failed download 016436157-016439176 (will retry)
EROR[10-03|20:58:47.459] write to disc error [error copying 016675633-016678405 file in writeBytesToDisc: [stream error: stream ID 73; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:58:47.473] Failed download 016433403-016436156 (will retry)
EROR[10-03|20:58:55.460] write to disc error [error copying 016672656-016675632 file in writeBytesToDisc: [stream error: stream ID 83; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:58:55.475] Failed download 016430527-016433402 (will retry)
EROR[10-03|20:59:03.461] write to disc error [error copying 016669838-016672655 file in writeBytesToDisc: [stream error: stream ID 97; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:59:03.471] Failed download 016427580-016430526 (will retry)
EROR[10-03|20:59:11.463] write to disc error [error copying 016666837-016669837 file in writeBytesToDisc: [stream error: stream ID 7; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:59:11.478] Failed download 016424427-016427579 (will retry)
EROR[10-03|20:59:19.464] write to disc error [error copying 016663861-016666836 file in writeBytesToDisc: [stream error: stream ID 19; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:59:19.478] Failed download 016421363-016424426 (will retry)
EROR[10-03|20:59:27.466] write to disc error [error copying 016660824-016663860 file in writeBytesToDisc: [stream error: stream ID 1863; INTERNAL_ERROR; received from peer]]
WARN[10-03|20:59:27.474] Failed download 016418388-016421362 (will retry)
EROR[10-03|20:59:35.467] write to disc error [error copying 016657751-016660823 file in writeBytesToDisc: [stream error: stream ID 57; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:59:35.481] Failed download 016415313-016418387 (will retry)
EROR[10-03|20:59:43.468] write to disc error [error copying 016654951-016657750 file in writeBytesToDisc: [stream error: stream ID 73; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:59:43.482] Failed download 016412414-016415312 (will retry)
EROR[10-03|20:59:51.470] write to disc error [error copying 016652332-016654950 file in writeBytesToDisc: [stream error: stream ID 89; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:59:51.484] Failed download 016408898-016412413 (will retry)
EROR[10-03|20:59:59.471] write to disc error [error copying 016649594-016652331 file in writeBytesToDisc: [stream error: stream ID 107; PROTOCOL_ERROR; received from peer]]
WARN[10-03|20:59:59.480] Failed download 016405814-016408897 (will retry)
EROR[10-03|21:00:07.472] write to disc error [error copying 016646700-016649593 file in writeBytesToDisc: [stream error: stream ID 125; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:00:07.479] Failed download 016402876-016405813 (will retry)
EROR[10-03|21:00:15.474] write to disc error [error copying 016643760-016646699 file in writeBytesToDisc: [stream error: stream ID 5; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:00:15.481] Failed download 016400001-016402875 (will retry)
EROR[10-03|21:00:23.474] write to disc error [error copying 016641064-016643759 file in writeBytesToDisc: [stream error: stream ID 29; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:00:23.482] Failed download 016397552-016400000 (will retry)
EROR[10-03|21:00:31.475] write to disc error [error copying 016638212-016641063 file in writeBytesToDisc: [stream error: stream ID 43; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:00:31.490] Failed download 016394423-016397551 (will retry)
EROR[10-03|21:00:39.477] write to disc error [error copying 016635446-016638211 file in writeBytesToDisc: [stream error: stream ID 59; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:00:39.484] Failed download 016391479-016394422 (will retry)
EROR[10-03|21:00:47.478] write to disc error [error copying 016632730-016635445 file in writeBytesToDisc: [stream error: stream ID 79; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:00:47.493] Failed download 016388383-016391478 (will retry)
EROR[10-03|21:00:55.479] write to disc error [error copying 016629900-016632729 file in writeBytesToDisc: [stream error: stream ID 3351; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:00:55.493] Failed download 016385429-016388382 (will retry)
EROR[10-03|21:01:03.481] write to disc error [error copying 016627238-016629899 file in writeBytesToDisc: [stream error: stream ID 123; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:01:03.488] Failed download 016382410-016385428 (will retry)
EROR[10-03|21:01:11.483] write to disc error [error copying 016624178-016627237 file in writeBytesToDisc: [stream error: stream ID 3647; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:01:11.490] Failed download 016379301-016382409 (will retry)
EROR[10-03|21:01:19.485] write to disc error [error copying 016621184-016624177 file in writeBytesToDisc: [stream error: stream ID 9; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:01:19.502] Failed download 016376364-016379300 (will retry)
EROR[10-03|21:01:27.485] write to disc error [error copying 016618437-016621183 file in writeBytesToDisc: [stream error: stream ID 33; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:01:27.492] Failed download 016373168-016376363 (will retry)
EROR[10-03|21:01:35.485] write to disc error [error copying 016615310-016618436 file in writeBytesToDisc: [stream error: stream ID 59; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:01:35.500] Failed download 016370239-016373167 (will retry)
EROR[10-03|21:01:43.487] write to disc error [error copying 016612475-016615309 file in writeBytesToDisc: [stream error: stream ID 81; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:01:43.502] Failed download 016367118-016370238 (will retry)
EROR[10-03|21:01:51.488] write to disc error [error copying 016609084-016612474 file in writeBytesToDisc: [stream error: stream ID 4285; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:01:51.501] Failed download 016364227-016367117 (will retry)
EROR[10-03|21:01:59.489] write to disc error [error copying 016606082-016609083 file in writeBytesToDisc: [stream error: stream ID 117; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:01:59.503] Failed download 016361441-016364226 (will retry)
EROR[10-03|21:02:07.491] write to disc error [error copying 016602867-016606081 file in writeBytesToDisc: [stream error: stream ID 133; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:02:07.499] Failed download 016357763-016361440 (will retry)
EROR[10-03|21:02:15.491] write to disc error [error copying 016600001-016602866 file in writeBytesToDisc: [stream error: stream ID 147; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:02:15.505] Failed download 016354610-016357762 (will retry)
EROR[10-03|21:02:23.492] write to disc error [error copying 016597169-016599853 file in writeBytesToDisc: [stream error: stream ID 1; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:02:23.500] Failed download 016350984-016354609 (will retry)
EROR[10-03|21:02:31.493] write to disc error [error copying 016594276-016597168 file in writeBytesToDisc: [stream error: stream ID 4883; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:02:31.502] Failed download 016348106-016350983 (will retry)
EROR[10-03|21:02:39.494] write to disc error [error copying 016591552-016594275 file in writeBytesToDisc: [stream error: stream ID 25; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:02:39.501] Failed download 016344673-016348105 (will retry)
EROR[10-03|21:02:47.495] write to disc error [error copying 016588645-016591551 file in writeBytesToDisc: [stream error: stream ID 5119; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:02:47.502] Failed download 016341611-016344672 (will retry)
EROR[10-03|21:02:55.496] write to disc error [error copying 016585754-016588644 file in writeBytesToDisc: [stream error: stream ID 5251; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:02:55.507] Failed download 016338656-016341610 (will retry)
EROR[10-03|21:03:03.498] write to disc error [error copying 016583046-016585753 file in writeBytesToDisc: [stream error: stream ID 5341; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:03:03.506] Failed download 016335491-016338655 (will retry)
EROR[10-03|21:03:11.499] write to disc error [error copying 016580220-016583045 file in writeBytesToDisc: [stream error: stream ID 69; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:03:11.513] Failed download 016332385-016335490 (will retry)
EROR[10-03|21:03:19.500] write to disc error [error copying 016577453-016580219 file in writeBytesToDisc: [stream error: stream ID 81; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:03:19.514] Failed download 016329104-016332384 (will retry)
EROR[10-03|21:03:27.500] write to disc error [error copying 016574764-016577452 file in writeBytesToDisc: [stream error: stream ID 3; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:03:27.510] Failed download 016325899-016329103 (will retry)
EROR[10-03|21:03:35.501] write to disc error [error copying 016571968-016574763 file in writeBytesToDisc: [stream error: stream ID 5735; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:03:35.515] Failed download 016322251-016325898 (will retry)
EROR[10-03|21:03:43.502] write to disc error [error copying 016569186-016571967 file in writeBytesToDisc: [stream error: stream ID 23; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:03:43.516] Failed download 016319119-016322250 (will retry)
EROR[10-03|21:03:51.503] write to disc error [error copying 016566142-016569185 file in writeBytesToDisc: [stream error: stream ID 35; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:03:51.517] Failed download 016315334-016319118 (will retry)
EROR[10-03|21:03:59.504] write to disc error [error copying 016563242-016566141 file in writeBytesToDisc: [stream error: stream ID 47; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:03:59.512] Failed download 016311512-016315333 (will retry)
EROR[10-03|21:04:07.505] write to disc error [error copying 016560201-016563241 file in writeBytesToDisc: [stream error: stream ID 59; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:04:07.520] Failed download 016307226-016311511 (will retry)
EROR[10-03|21:04:15.507] write to disc error [error copying 016557205-016560200 file in writeBytesToDisc: [stream error: stream ID 6203; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:04:15.521] Failed download 016303622-016307225 (will retry)
EROR[10-03|21:04:23.509] write to disc error [error copying 016554304-016557204 file in writeBytesToDisc: [stream error: stream ID 6283; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:04:23.522] Failed download 016300001-016303621 (will retry)
EROR[10-03|21:04:31.509] write to disc error [error copying 016551179-016554303 file in writeBytesToDisc: [stream error: stream ID 6285; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:04:31.523] Failed download 016298609-016300000 (will retry)
EROR[10-03|21:04:39.510] write to disc error [error copying 016548288-016551178 file in writeBytesToDisc: [stream error: stream ID 65; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:04:39.524] Failed download 016295114-016298608 (will retry)
EROR[10-03|21:04:47.511] write to disc error [error copying 016545297-016548287 file in writeBytesToDisc: [stream error: stream ID 1; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:04:47.525] Failed download 016291786-016295113 (will retry)
EROR[10-03|21:04:55.512] write to disc error [error copying 016542435-016545296 file in writeBytesToDisc: [stream error: stream ID 3; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:04:55.526] Failed download 016288293-016291785 (will retry)
EROR[10-03|21:05:03.513] write to disc error [error copying 016539584-016542434 file in writeBytesToDisc: [stream error: stream ID 5; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:05:03.522] Failed download 016284909-016288292 (will retry)
EROR[10-03|21:05:11.515] write to disc error [error copying 016536670-016539583 file in writeBytesToDisc: [stream error: stream ID 7; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:05:11.524] Failed download 016281446-016284908 (will retry)
EROR[10-03|21:05:19.515] write to disc error [error copying 016533838-016536669 file in writeBytesToDisc: [stream error: stream ID 9; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:05:19.530] Failed download 016278116-016281445 (will retry)
EROR[10-03|21:05:27.517] write to disc error [error copying 016530711-016533837 file in writeBytesToDisc: [stream error: stream ID 11; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:05:27.531] Failed download 016274803-016278115 (will retry)
EROR[10-03|21:05:35.519] write to disc error [error copying 016527792-016530710 file in writeBytesToDisc: [stream error: stream ID 13; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:05:35.526] Failed download 016271440-016274802 (will retry)
EROR[10-03|21:05:43.520] write to disc error [error copying 016524793-016527791 file in writeBytesToDisc: [stream error: stream ID 15; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:05:43.535] Failed download 016268388-016271439 (will retry)
EROR[10-03|21:05:51.521] write to disc error [error copying 016521773-016524792 file in writeBytesToDisc: [stream error: stream ID 17; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:05:51.530] Failed download 016264632-016268387 (will retry)
EROR[10-03|21:05:59.523] write to disc error [error copying 016518928-016521772 file in writeBytesToDisc: [stream error: stream ID 1; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:05:59.530] Failed download 016261051-016264631 (will retry)
EROR[10-03|21:06:07.523] write to disc error [error copying 016515804-016518927 file in writeBytesToDisc: [stream error: stream ID 3; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:06:07.538] Failed download 016256961-016261050 (will retry)
EROR[10-03|21:06:15.524] write to disc error [error copying 016512916-016515803 file in writeBytesToDisc: [stream error: stream ID 5; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:06:15.536] Failed download 016253393-016256960 (will retry)
EROR[10-03|21:06:23.526] write to disc error [error copying 016509570-016512915 file in writeBytesToDisc: [stream error: stream ID 7; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:06:23.540] Failed download 016249742-016253392 (will retry)
EROR[10-03|21:06:31.527] write to disc error [error copying 016506462-016509569 file in writeBytesToDisc: [stream error: stream ID 9; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:06:31.536] Failed download 016246607-016249741 (will retry)
EROR[10-03|21:06:39.529] write to disc error [error copying 016503129-016506461 file in writeBytesToDisc: [stream error: stream ID 11; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:06:39.538] Failed download 016243257-016246606 (will retry)
EROR[10-03|21:06:47.530] write to disc error [error copying 016500001-016503128 file in writeBytesToDisc: [stream error: stream ID 13; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:06:47.538] Failed download 016240269-016243256 (will retry)
EROR[10-03|21:06:55.531] write to disc error [error copying 016499108-016500000 file in writeBytesToDisc: [stream error: stream ID 15; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:06:55.542] Failed download 016236598-016240268 (will retry)
EROR[10-03|21:07:03.532] write to disc error [error copying 016496045-016499107 file in writeBytesToDisc: [stream error: stream ID 17; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:07:03.546] Failed download 016233474-016236597 (will retry)
EROR[10-03|21:07:11.534] write to disc error [error copying 016493010-016496044 file in writeBytesToDisc: [stream error: stream ID 1; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:07:11.541] Failed download 016230126-016233473 (will retry)
EROR[10-03|21:07:19.536] write to disc error [error copying 016489997-016493009 file in writeBytesToDisc: [stream error: stream ID 3; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:07:19.544] Failed download 016226889-016230125 (will retry)
EROR[10-03|21:07:27.537] write to disc error [error copying 016486866-016489996 file in writeBytesToDisc: [stream error: stream ID 5; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:07:27.545] Failed download 016223437-016226888 (will retry)
EROR[10-03|21:07:35.539] write to disc error [error copying 016483990-016486865 file in writeBytesToDisc: [stream error: stream ID 7; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:07:35.554] Failed download 016220257-016223436 (will retry)
EROR[10-03|21:07:43.540] write to disc error [error copying 016480821-016483989 file in writeBytesToDisc: [stream error: stream ID 9; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:07:43.549] Failed download 016216832-016220256 (will retry)
EROR[10-03|21:07:51.540] write to disc error [error copying 016477911-016480820 file in writeBytesToDisc: [stream error: stream ID 11; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:07:51.555] Failed download 016213283-016216831 (will retry)
EROR[10-03|21:07:59.541] write to disc error [error copying 016474811-016477910 file in writeBytesToDisc: [stream error: stream ID 13; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:07:59.555] Failed download 016209691-016213282 (will retry)
EROR[10-03|21:08:07.542] write to disc error [error copying 016471606-016474810 file in writeBytesToDisc: [stream error: stream ID 15; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:08:07.556] Failed download 016206346-016209690 (will retry)
EROR[10-03|21:08:15.544] write to disc error [error copying 016468487-016471605 file in writeBytesToDisc: [stream error: stream ID 17; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:08:15.551] Failed download 016203052-016206345 (will retry)
EROR[10-03|21:08:23.546] write to disc error [error copying 016464940-016468486 file in writeBytesToDisc: [stream error: stream ID 1; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:08:23.560] Failed download 016200001-016203051 (will retry)
EROR[10-03|21:08:31.548] write to disc error [error copying 016461744-016464939 file in writeBytesToDisc: [stream error: stream ID 3; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:08:31.555] Failed download 016199408-016200000 (will retry)
EROR[10-03|21:08:39.549] write to disc error [error copying 016457990-016461743 file in writeBytesToDisc: [stream error: stream ID 5; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:08:39.563] Failed download 016196541-016199407 (will retry)
EROR[10-03|21:08:47.550] write to disc error [error copying 016454941-016457989 file in writeBytesToDisc: [stream error: stream ID 7; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:08:47.558] Failed download 016193316-016196540 (will retry)
EROR[10-03|21:08:55.551] write to disc error [error copying 016451745-016454940 file in writeBytesToDisc: [stream error: stream ID 9; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:08:55.560] Failed download 016190392-016193315 (will retry)
EROR[10-03|21:09:03.553] write to disc error [error copying 016448874-016451744 file in writeBytesToDisc: [stream error: stream ID 11; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:09:03.560] Failed download 016187180-016190391 (will retry)
EROR[10-03|21:09:11.553] write to disc error [error copying 016445471-016448873 file in writeBytesToDisc: [stream error: stream ID 13; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:09:11.568] Failed download 016184123-016187179 (will retry)
EROR[10-03|21:09:19.554] write to disc error [error copying 016442369-016445470 file in writeBytesToDisc: [stream error: stream ID 15; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:09:19.570] Failed download 016180912-016184122 (will retry)
EROR[10-03|21:09:27.555] write to disc error [error copying 016439177-016442368 file in writeBytesToDisc: [stream error: stream ID 17; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:09:27.569] Failed download 016177849-016180911 (will retry)
EROR[10-03|21:09:35.557] write to disc error [error copying 016436157-016439176 file in writeBytesToDisc: [stream error: stream ID 1; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:09:35.565] Failed download 016175011-016177848 (will retry)
EROR[10-03|21:09:43.558] write to disc error [error copying 016433403-016436156 file in writeBytesToDisc: [stream error: stream ID 3; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:09:43.567] Failed download 016171828-016175010 (will retry)
EROR[10-03|21:09:51.559] write to disc error [error copying 016430527-016433402 file in writeBytesToDisc: [stream error: stream ID 5; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:09:51.575] Failed download 016168826-016171827 (will retry)
EROR[10-03|21:09:59.561] write to disc error [error copying 016427580-016430526 file in writeBytesToDisc: [stream error: stream ID 7; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:09:59.568] Failed download 016165762-016168825 (will retry)
EROR[10-03|21:10:07.561] write to disc error [error copying 016424427-016427579 file in writeBytesToDisc: [stream error: stream ID 9; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:10:07.577] Failed download 016162436-016165761 (will retry)
EROR[10-03|21:10:15.563] write to disc error [error copying 016421363-016424426 file in writeBytesToDisc: [stream error: stream ID 11; PROTOCOL_ERROR; received from peer]]
WARN[10-03|21:10:15.577] Failed download 016159038-016162435 (will retry)

@tjayrush
Copy link
Member

Are you out of disc space or something?

What does df -h say?

@cevatbostancioglu
Copy link
Author

/opt$ df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 13G 5,9M 13G 1% /run
/dev/nvme0n1p2 92G 20G 67G 23% /
tmpfs 63G 159M 63G 1% /dev/shm
tmpfs 5,0M 4,0K 5,0M 1% /run/lock
efivarfs 128K 43K 81K 35% /sys/firmware/efi/efivars
/dev/nvme0n1p1 952M 6,1M 946M 1% /boot/efi
/dev/nvme0n1p3 7,2T 2,9T 3,9T 43% /opt
tmpfs 13G 108K 13G 1% /run/user/1000

i think some index files are not on the ipfs server for some reason. i will try local indexing today.

@cevatbostancioglu
Copy link
Author

/opt$ chifra scrape --block_cnt 50000 --sleep 1
INFO[11-03|20:46:46.106] Scraping:
INFO[11-03|20:46:46.106] Path: /home/cevat/.local/share/trueblocks/unchained/mainnet/
INFO[11-03|20:46:46.106] Settings: {2000000 100000 2300000 28 false 20}
INFO[11-03|20:47:07.295] mainnet #18999999, found 574292 apps and 410129 addrs (1.4 apps/addr). created 0 chunks, staged 574292 apps (28.7%), need 1425708.
INFO[11-03|20:47:08.578] no ripe files to consolidate
INFO[11-03|20:47:09.843] no ripe files to consolidate
INFO[11-03|20:47:11.123] no ripe files to consolidate
INFO[11-03|20:47:12.414] no ripe files to consolidate
INFO[11-03|20:47:13.698] no ripe files to consolidate
INFO[11-03|20:47:14.984] no ripe files to consolidate
INFO[11-03|20:47:16.253] no ripe files to consolidate
INFO[11-03|20:47:17.787] no ripe files to consolidate
INFO[11-03|20:47:19.069] no ripe files to consolidate
INFO[11-03|20:47:20.348] no ripe files to consolidate
INFO[11-03|20:47:21.631] no ripe files to consolidate
INFO[11-03|20:47:22.921] no ripe files to consolidate
INFO[11-03|20:47:24.200] no ripe files to consolidate

is this normal

@tjayrush
Copy link
Member

/opt$ df -h Filesystem Size Used Avail Use% Mounted on tmpfs 13G 5,9M 13G 1% /run /dev/nvme0n1p2 92G 20G 67G 23% / tmpfs 63G 159M 63G 1% /dev/shm tmpfs 5,0M 4,0K 5,0M 1% /run/lock efivarfs 128K 43K 81K 35% /sys/firmware/efi/efivars /dev/nvme0n1p1 952M 6,1M 946M 1% /boot/efi /dev/nvme0n1p3 7,2T 2,9T 3,9T 43% /opt tmpfs 13G 108K 13G 1% /run/user/1000

i think some index files are not on the ipfs server for some reason. i will try local indexing today.

This may be the problem. I see you have a 7TB hard drive at /opt. But, your Unchained Index is stored at /home/cevat/.local/share/trueblocks/unchained/mainnet/ which I don't think it on /opt unless you did something in particular.

Do you know if /home/cevat/.local/share/trueblocks/unchained is on a hard drive that has space?

You can modify that location to put the Unchained Index anywhere you like (for example, on /opt/trueblocks) by editing the indexPath that you'll find under the [settings] group when you type chifra config edit.

It doesn't look like any of the other discs are full, but you can do this if you suspect that you're writing to a smaller drive that may fill up. The Unchained Index requires about 150GB.

@tjayrush
Copy link
Member

tjayrush commented Mar 11, 2024 •
edited
Loading

/opt$ chifra scrape --block_cnt 50000 --sleep 1 INFO[11-03|20:46:46.106] Scraping: INFO[11-03|20:46:46.106] Path: /home/cevat/.local/share/trueblocks/unchained/mainnet/ INFO[11-03|20:46:46.106] Settings: {2000000 100000 2300000 28 false 20} INFO[11-03|20:47:07.295] mainnet #18999999, found 574292 apps and 410129 addrs (1.4 apps/addr). created 0 chunks, staged 574292 apps (28.7%), need 1425708. INFO[11-03|20:47:08.578] no ripe files to consolidate INFO[11-03|20:47:09.843] no ripe files to consolidate INFO[11-03|20:47:11.123] no ripe files to consolidate INFO[11-03|20:47:12.414] no ripe files to consolidate INFO[11-03|20:47:13.698] no ripe files to consolidate INFO[11-03|20:47:14.984] no ripe files to consolidate INFO[11-03|20:47:16.253] no ripe files to consolidate INFO[11-03|20:47:17.787] no ripe files to consolidate INFO[11-03|20:47:19.069] no ripe files to consolidate INFO[11-03|20:47:20.348] no ripe files to consolidate INFO[11-03|20:47:21.631] no ripe files to consolidate INFO[11-03|20:47:22.921] no ripe files to consolidate INFO[11-03|20:47:24.200] no ripe files to consolidate

is this normal

It's normal in the sense that chifra scrape seems to be working correctly. It may not be normal if your node is stuck (that is, not moving past block 18999999, which looks like a hugely suspicious number...) Once chifra scrape catches up to what (appears to be?) the head of the chain -- that is no new blocks have been produced since the last time it ran -- it will do exactly what it's doing...

@cevatbostancioglu
Copy link
Author

/opt$ df -h Filesystem Size Used Avail Use% Mounted on tmpfs 13G 5,9M 13G 1% /run /dev/nvme0n1p2 92G 20G 67G 23% / tmpfs 63G 159M 63G 1% /dev/shm tmpfs 5,0M 4,0K 5,0M 1% /run/lock efivarfs 128K 43K 81K 35% /sys/firmware/efi/efivars /dev/nvme0n1p1 952M 6,1M 946M 1% /boot/efi /dev/nvme0n1p3 7,2T 2,9T 3,9T 43% /opt tmpfs 13G 108K 13G 1% /run/user/1000
i think some index files are not on the ipfs server for some reason. i will try local indexing today.

This may be the problem. I see you have a 7TB hard drive at /opt. But, your Unchained Index is stored at /home/cevat/.local/share/trueblocks/unchained/mainnet/ which I don't think it on /opt unless you did something in particular.

Do you know if /home/cevat/.local/share/trueblocks/unchained is on a hard drive that has space?

You can modify that location to put the Unchained Index anywhere you like (for example, on /opt/trueblocks) by editing the indexPath that you'll find under the [settings] group when you type chifra config edit.

It doesn't look like any of the other discs are full, but you can do this if you suspect that you're writing to a smaller drive that may fill up. The Unchained Index requires about 150GB.

hello, i have linked ./local/share/trueblocks to /opt/trueblocks. it works fine ? before the folder link, chifra had this issue. i think the links have problem or is it possible that i need to configure some network related things ?

@tjayrush
Copy link
Member

Thanks so much for your continued patience. Maybe it would be easier if we got on a call. Please connect with me in discord. For now, I think I'll close this issue. Getting on a call will get you to a solution a lot sooner.

@tjayrush
Copy link
Member

This issue is related to #3696

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug TC-apps:init
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tjayrush @cevatbostancioglu