neutrino sync and shutdown problems

[chappjc]

There are a number of neutrino-related issues that only really pop out with testnet or mainnet where sync can take actual time.

1. Wallet sync progress hits 100% way too soon: client/asset/dcr: restart required after restoring native dcr wallet with funds #1816 (comment) It was hung during that too of course.
2. When the wallets appear to be hung, I always find that in the assetdb neutrino.log it is requesting cfilters, nice and slow, usually 1-by-1 instead of batch. That's something that can be improved with neutrino+query+rescan: improve rescan speed lightninglabs/neutrino#236, which I tested at one point.
3. Even with the above fix, neutrino sync is non-cancellable. We'd have to figure out why it doesn't respond to shutdown requests when in sync.

Originally posted by @chappjc in #1931 (comment)

[JoeGruffins]

I've looked into #3 a bit.

Currently bchwallet will just not let you quit while it is doing recovery from the wallet birthday. Specifically, it will not let you lock the wallet. btcwallet has a fix for this but unfortunately it causes the wallet to panic unless shutting down. So, any attempt to lock the wallet while recovery is happening blows up dexc. We can shut down, just not lock.

This is locking the wallet with the UI while recovery is happening with btc currently:

panic: unable to synchronize wallet to chain: unable to perform wallet recovery: recovery: forced shutdown

goroutine 658 [running]:
github.com/btcsuite/btcwallet/wallet.(*Wallet).handleChainNotifications(0xc00112c120)
        /home/joe/git/btcwallet/wallet/chainntfns.go:118 +0x10fe
created by github.com/btcsuite/btcwallet/wallet.(*Wallet).SynchronizeRPC
        /home/joe/git/btcwallet/wallet/wallet.go:213 +0x25c

Also, bchwallet seems to think we are in sync and starts the recovery before the backend is ready. This may be a testnet thing, will look into it more.

So, I guess I will push the panic inducing changes to bchwallet. Then we need to be careful as to not lock the wallet as it rescans for the wallet birthday. Or change btcwallet and bchwallet so that they can stop in the middle of the recovery and also not panic, which may or may not be a large diff.

The point of panic https://github.com/btcsuite/btcwallet/blob/27f244e8454076f5afcc5be6266c3762f902800f/wallet/chainntfns.go#L115-L119

The problematic recovery that we can not currently escape unless shutdown https://github.com/btcsuite/btcwallet/blob/27f244e8454076f5afcc5be6266c3762f902800f/wallet/wallet.go#L665-L792

[JoeGruffins]

I made an issue and will see if anyone has a comment about it. btcsuite/btcwallet#827

Again, bchwallet does not panic here, it just does not let you lock which is why client will hang on wallet lock until recovery is finished. I don't really want to open a pr with bchwallet that causes more problems. Maybe can get btcwallet fixed first then apply those changes to bchwallet.

[JoeGruffins]

The btcwallet is also #1690 with a panic in handleChainNotifications