Insufficient balance error at 0.0001 BTC, yet available balance says 0.0023 BTC

[314159265359879]

bc1qzl5xt45gazvj2j4su2xdm5tftqkckgst5hhald

On another wallet it shows

I have always assumed small UTXO's causing it to be non economical to include everything but if that is the case I don't understand why the "available balance" is displayed so much higher.

I am sure the current network fees also play a role

I would love to see this confirmed by a developer who can mock this address. What would be the transaction size, is it that high that the transfer can't be covered by the available balance?

[314159265359879]

We know that the size (and thus fee) of a transaction can increase a lot for every additional input it has. Leather is supposed to select UTXO's based on size so that the largest gets included first (because it is most cost effective).

Yet when they this user tries with Leather to send 0.0001 BTC the message is "insufficient balance" if Leather uses the biggest UTXO to cover this... it should be no more expensive to cover the fee than with the other wallet 0.00033390?

Using this API to check UTXO's on the address we do see a lot of small ones, likely not cost effective to transfer right now with 200-300 sats/vB fees. https://blockstream.info/api/address/bc1qzl5xt45gazvj2j4su2xdm5tftqkckgst5hhald/utxo but there is atleast one large UTXO in there too:

That UTXO alone there (blockstream API) that could cover it the transfer of 0.0001 BTC: that would be an estimated 150-200 kB transaction with a single input and output.
at 200 sat/vB still just 40000 sats total fee so ample to cover the transfer and fee.

[314159265359879]

Another case like the one above:
With this address: bc1qy0kfxd896lvujuq4xreuv6xc85t65ljxhs2san, similarly it has a lot of small UTXO's.

https://blockstream.info/api/address/bc1qy0kfxd896lvujuq4xreuv6xc85t65ljxhs2san/utxo

The available balance in the lower right corner looks sufficient. Average network fees for the last five blocks were 30-72 sats/vB.

The bigger UTXO's look like enough to cover fee and the transfer for a small amount like 0.00006 BTC, if a single UTXO would be used to cover the fee and transfer, right?

[kyranjamie]

I wonder if this might just be a bug with the validation as I've ran into this happening illegitimately before.

I'd recommend that we try testing this feature with the wallet set to slow 3G emulation, or perhaps it's a react rendering issue where the validator is running behind what the UI is showing.

[markmhendrickson]

@314159265359879 can you perform such a test?

[314159265359879]

@markmhendrickson I had a call about this last week with Kyran, I need some dev eyes on this.

I can't run these tests. I think it may also require mocking the addresses that were affected (address' balance state has changed since this was first reported, atleast for the first address).
bc1qzl5xt45gazvj2j4su2xdm5tftqkckgst5hhald
bc1qy0kfxd896lvujuq4xreuv6xc85t65ljxhs2san

[314159265359879]

Another case like this user has a wallet with several small UTXO's but also one big one.

This address
https://mempool.space/address/bc1qdmzh8st7lfe7tk5uehu4ff8nkkcf359eq5h96s

the largest UTXO is 56177 sats.

When the user tried "send max" only 16k was transferrable. A high total fee is expected because currently, send max will include non-economical amounts.

Because the wallet should select the biggest UTXO's first it should also be able to send out about 43000 sats without an issue. The transaction fee would be about 12000-13000 sats at a rate of 72 sats/vB. Which means 56177 sats UTXO should be able to cover it on its own without including any small utxo's in the 600 sats range.

But the user sees this:

Left is "sending max" amount.

On the right is trying to send 30,000 sats.

[kyranjamie]

I wonder if you can provide some test cases that'll help us test this logic @314159265359879?

Ideally we can have this covered by better unit tests that consider a set of UTXOs and how much the max transferable amount should be?

[markmhendrickson]

Path forward here is to write integration tests for send max

[314159265359879]

Test cases. Are we changing the current behavior's or keeping it as it is?

Right now it will empty the wallet at whatever cost.
Users prefer to exclude UTXO's when it costs a lot to send them out. There are still some options we can choose between:
aa. exclude when the UTXO's cost more than a certain percentage of what they're worth? (lets say 20%)
bb. include them as long as sending them out costs less then 100% of the UTXO?
cc. or perhaps do you want to exclude UTXO's with a size below 1201 sats regardless of network fees?

Goal of the test is: Are UTXO's excluded from send-max when it is not economic to send them out (based on predefined rule)

Lets assume a wallet has these UTXO's on their BTC address (native segwit, P2WPKH)

a. 0.00000600
b. 0.00000600
c. 0.00001200
d. 0.00001200
d. 0.00010000
e. 0.00010000
f. 0.00025000
g. 0.00040000
h. 0.50000000

According to this calculator adding an input increases the transaction size by 68 vB (transaction type P2WPKH).

at these fee rates .... including one more input will cost: -
A. 1 sat/vB - 68 sats per input
B. 5 sats/vB - 340 sats per input
C. 30 sats/vB - 2040 sats per input
D. 100 sats/vB - 6800 sats per input
E. 400 sats/vB - 27400 sats per input

If the only goal is to test that UTXO's are being excluded when not cost-effective to send and at least 80% of the value is sent (not spend as fee) (aa) the test should determine if at (E.) 400 sats/vB fees, the wallet does excludes UTXO a. through g. and only includes h. as cost-effective.

Further testing could help determine if the send-max also excludes part of the UTXO's with varying market fees. For example if aa. is adapted and the market fees average 30 sats/vB. Excluded UTXO should be a. through d. and included should be e. through h.

[markmhendrickson]

@314159265359879 your last comment here seems like a related but different UX question than this issue's original one? In that this issue deals with ensuring that users can send their entire available balance, whereas your last comment deals with providing users with warnings or optionality in regards to using non-cost-effective UTXOs?

If so, can we break the latter out into its own issue?

[markmhendrickson]

Also by "writing integrations tests" for this particular issue, we wouldn't aim to change the current UX, but rather fix it (along the lines of ensuring users can send their entire available balance). These automated tests would presumably help identify cases in which the bug arises, fix them, and prevent regressions in the future.