Problem with CRIT - Failed to truncate extra state histories

[kasrakhamesi]

I Get this error : CRIT - Failed to truncate extra state histories err="EOF, fileid: 0, start: 325221424, length: 3969745872"
when I trying to run my node.why ?

this is my Dockerfile command to run node :

CMD ["./geth", "--datadir","/bsc/node","--config","/bsc/config/config.toml","--db.engine","pebble","--state.scheme","path","--http","--http.api","eth,net,web3,admin","--http.rpcprefix","/","--http.corsdomain","","--http.addr","0.0.0.0","--ws.addr","0.0.0.0","--cache","8000","--rpc.allow-unprotected-txs","--history.transactions","0"]

I get https://github.com/bnb-chain/bsc-snapshots - Path-Base-State-Scheme(recommand) chain data

my geth version : v1.3.8

please Help me to fix that

[MatusKysel]

Hey @kasrakhamesi do you know how to build bsc from source code? I think we have a fix for your issue, but you need to build a custom branch to test it

[seeforschauer]

hi there,
I have a similar bug
t=2024-02-09T17:33:46+0200 lvl=crit msg="Failed to truncate extra state histories" err="out of range, tail: 0, head: 0, target: 35986828"

/home/bsc/geth/geth \
        --config /home/bsc/bin/config.toml \
        --datadir /var/storage/bsc/server/data-seed/geth/  \
        --http \
        --http.api "engine,db,eth,net,web3,txpool,debug" \
        --http.addr "0.0.0.0" \
        --http.vhosts "*" \
        --cache 8000 \
        --rpc.allow-unprotected-txs \
        --metrics \
        --metrics.expensive \
        --metrics.addr "0.0.0.0" \
        --metrics.port 9090 \
        --diffblock=5000 \
        --history.transactions=0 \
        --syncmode=full \
        --tries-verify-mode=local \
        --pruneancient=true \
        --db.engine=pebble \
        --state.scheme=path

version is 1.3.8

[MatusKysel]

@seeforschauer are you ok with testing my custom branch, do you know how to build it from source?

[STdevK]

@seeforschauer did you observe Disk db is pruned within the log description?

[seeforschauer]

@STdevK No.

t=2024-02-09T17:33:44+0200 lvl=info msg="Use CLI state scheme" CLI=path
t=2024-02-09T17:33:44+0200 lvl=info msg="Set global gas cap"   cap=50,000,000
t=2024-02-09T17:33:44+0200 lvl=info msg="Initializing the KZG library" backend=gokzg
t=2024-02-09T17:33:44+0200 lvl=warn msg="Using deprecated resource file, please move this file to the 'geth' subdirectory of datadir." file=/var/storage/bsc/server/data-seed/geth/chaindata
t=2024-02-09T17:33:44+0200 lvl=info msg="Using pebble as the backing database"
t=2024-02-09T17:33:44+0200 lvl=info msg="Allocated cache and file handles"     database=/var/storage/bsc/server/data-seed/geth/chaindata cache="3.12 GiB" handles=524,288 "memory table"="800.00 MiB"
t=2024-02-09T17:33:44+0200 lvl=info msg="Read ancientdb item counts"           items=0
t=2024-02-09T17:33:44+0200 lvl=info msg="Opened ancientdb with nodata mode"    database=/var/storage/bsc/server/data-seed/geth/chaindata/ancient/chain frozen=35,019,181
t=2024-02-09T17:33:44+0200 lvl=info msg="State scheme set by user"             scheme=path
t=2024-02-09T17:33:44+0200 lvl=info msg="Capped dirty cache size"              provided="1.95 GiB" adjusted="256.00 MiB"
t=2024-02-09T17:33:44+0200 lvl=info msg="Clean cache size"                     provided="1.17 GiB"
t=2024-02-09T17:33:44+0200 lvl=info msg="Allocated trie memory caches"         clean="1.17 GiB" dirty="256.00 MiB"
t=2024-02-09T17:33:44+0200 lvl=info msg="Unprotected transactions allowed"
t=2024-02-09T17:33:44+0200 lvl=info msg=Parlia                                 chainConfig="{ChainID: 56 Homestead: 0 DAO: <nil> DAOSupport: false EIP150: 0 EIP155: 0 EIP158: 0 Byzantium: 0 Constantinople: 0 Petersburg: 0 Istanbul: 0, Muir Glacier: 0, Ramanujan: 0, Niels: 0, MirrorSync: 5184000, Bruno: 13082000, Berlin: 31302048, YOLO v3: <nil>, CatalystBlock: <nil>, London: 31302048, ArrowGlacier: <nil>, MergeFork:<nil>, Euler: 18907621, Gibbs: 23846001, Nano: 21962149, Moran: 22107423, Planck: 27281024,Luban: 29020050, Plato: 30720096, Hertz: 31302048, Hertzfix: 34140700, ShanghaiTime: 1705996800, KeplerTime: 1705996800, Engine: parlia}"
t=2024-02-09T17:33:44+0200 lvl=info msg="Initialising Ethereum protocol"       network=56 dbversion=8
t=2024-02-09T17:33:45+0200 lvl=info msg="new async node buffer"                limit="256.00 MiB" layers=183
t=2024-02-09T17:33:46+0200 lvl=info msg="Opened ancient database"              database=/var/storage/bsc/server/data-seed/geth/chaindata/ancient/state readonly=false frozen=0
t=2024-02-09T17:33:46+0200 lvl=crit msg="Failed to truncate extra state histories" err="out of range, tail: 0, head: 0, target: 35986828"

[sysvm]

@seeforschauer which snapshot do you use?

[YuXiaoCoder]

1.3.9 Having the same problem

[MatusKysel]

1.3.9 Having the same problem

@YuXiaoCoder you need to resync, as 1.3.9 is only fixing issue not recovering from it

[seeforschauer]

@seeforschauer which snapshot do you use?

@sysvm the snapshot URl was https://snapshots.48.club/geth.pbss.35092542.tar.zst

[seeforschauer]

@seeforschauer are you ok with testing my custom branch, do you know how to build it from source?

@STdevK ok, no atm.

[MatusKysel]

This issue is fixed in https://github.com/bnb-chain/bsc/releases/tag/v1.3.9, this version doesn't recover from that state, just prevent from happing. I am closing this issue for now, but feel free to re-open in case is still happening for you

[sysvm]

@seeforschauer which snapshot do you use?

@sysvm the snapshot URl was https://snapshots.48.club/geth.pbss.35092542.tar.zst

The data in this snapshot is incorrect, you'd better use the latest version. The version you're using is a pruned snapshot, you should view this log Disk db is pruned.

[FerdyCai]

This issue is fixed in https://github.com/bnb-chain/bsc/releases/tag/v1.3.9, this version doesn't recover from that state, just prevent from happing. I am closing this issue for now, but feel free to re-open in case is still happening for you

do you mean if that state happened, wo should resync with v1.3.9? I want to update v1.3.7 to v.1.3.12, I should resync ,too?

[MatusKysel]

@FerdyCai if you want to update, you do not need to resync

[FerdyCai]

@FerdyCai if you want to update, you do not need to resync

so how can I resolve the problem "lvl=crit msg="Failed to truncate extra state histories" err="out of range, tail: 0, head: 0, target: 35258935"" when update v1.3.7 to v.1.3.12 ?

[MatusKysel]

@FerdyCai if you want to solve it, you need to resync as I mentioned new version does not recover from this only prevents it from happening