3

for business reason i need to use the bitcoin-core (v0.20.1) automatic coin selection feature and few days ago i saw a transaction come out that left me puzzled, it was like that:

input

0.00017[...]  bc1qaddr1
  0.01292[...]  bc1qaddr2
  0.00030[...]  bc1qaddr3
  0.00020[...]  bc1qaddr4
  0.00015[...]  bc1qaddr5
  0.00035[...]  bc1qaddr6
  0.02371[...]  bc1qaddr7


output


0.02783[...]  bc1qaddr8  > amount sent
  0.00965[...]  bc1qaddr9  > change

in this transaction 5 out of 7 utxos are unnecessary, is there some benefit in chaining toghether utxos in this way?
from my point of view this is a terrible way of loosing privacy and pay way more fee than needed, what i miss?
i read that bitcoin-core tries to find an exact match from available utxos and the amount to send, but if it fails then try to find a subset of utxos to reach a change of 0.01btc, is that right?
so, in my case, the unneeded utxos are added to get the change of my trasaction closer to 0.01?

leevancleef
  • 862
  • 7
  • 17
  • Did you have another UTXO or UTXOs(less than 7) to pay for 0.027 or more BTC? –  Mar 24 '21 at 22:59
  • If yes: https://github.com/bitcoin/bitcoin/issues/20598 –  Mar 24 '21 at 22:59
  • bc1qaddr2 and bc1qaddr7 summed are already more than 0.03663 – leevancleef Mar 24 '21 at 23:02
  • Known Issue from long time.Few other issues with wallet related to privacy. Workarounds: 1. Dont use Bitcoin Core wallet until those issues are fixed 2. Try to fix those issues. Another example: https://github.com/bitcoin/bitcoin/pull/21284#issuecomment-787538566 –  Mar 24 '21 at 23:06
  • the issue here is not that there are better (smaller) subset to choose, but that most utxos in this subset are completely useless to reach the amount needed – leevancleef Mar 24 '21 at 23:09
  • I am not sure how you define useless but coin selection algo is not working as expected IMO in some cases. –  Mar 24 '21 at 23:11
  • I say that they are useless because, if i understand well, they are not needed to reach the amount i have chosen to send, but they are needed to reach the amount chosen by bitcoin-core (my amount + 0.01 btc) – leevancleef Mar 24 '21 at 23:20
  • This is done by coin selection algorithm. Either you select inputs manually and manage things or follow what I wrote above. –  Mar 24 '21 at 23:25
  • Or moderators Murch and andrew-chow can help if they care about the issue –  Mar 24 '21 at 23:29

1 Answers1

3

This occurs because Bitcoin Core's coin selection algorithm tries to have a minimum change of 0.01 BTC. Fees are subtracted from the change output so the result is that the change output has a lower value, but before fees, it will try to for the change output to be at least 0.01 BTC.

As we can see in your transaction, the change before fees is approximately 0.01 BTC. Presumably it is actually over that because you have obfuscated the last several digits. Then the transaction fee was subtracted from the change output to get the final change output value that you have.

The reason that there is a minimum change is so to avoid creating very small change outputs. Small change outputs can be detrimental to the user's wallet and to the network as a whole. Those small change outputs result in the wallet having more UTXOs, which means that for future transactions, require more UTXOs to spend. Furthermore, small values can quickly become uneconomical outputs when the feerate rises. If lots of small change outputs were being created, a wallet could end up having a significant portion of its UTXOs be unspendable when fees are high.

Additionally, by having a minimum change target, the coin selection algorithm will tend to consolidate and sweep up smaller UTXOs. This behavior is, in general, good for the wallet and good for the network. It reduces the number of UTXOs globally, reduces the number of UTXOs that the wallet needs to handle and keep track of, and makes it less likely that UTXOs in the wallet will become uneconomical in high fee events.

In general, coin selection is a hard problem which needs to balance several factors. It is not just what the fees are for the transaction right now, nor is it necessarily just about the user of the wallet in that moment. Coin selection needs to consider the cost of spending outputs in the future and how its behavior will effect the selection of UTXOs in a future transaction. Coin selection needs to balance the health of the network and the performance of the user's wallet.

Ava Chow
  • 70,382
  • 5
  • 81
  • 161
  • tries to have a minimum change of 0.01 BTC this is not how it should work IMO. Not sure how these random values are defined. –  Mar 24 '21 at 23:40
  • 1
    @Prayank You're right, it's probably an outdated, hardcoded value that shouldn't be needed. But we also can't just get rid of it - Bitcoin Core's coin selection algorithm is the result of many tweaks over the years, and we know it works okayish in most cases. It is obviously suboptimal in some, but small and innocently looking changes can have very dramatic effects (in fact, seemingly good ideas have been reverted in the past because they resulted in very bad wallet states). We need improvements to coin selection, but they're not as simple as getting rid of things you don't understand. – Pieter Wuille Mar 24 '21 at 23:49
  • 1
    @Andrew Chow Perhaps explain why a rule like this - albeit outdated - is useful: without it, the wallet in some usage patterns quickly grinds down its UTXO set to dust. – Pieter Wuille Mar 24 '21 at 23:51
  • ok, i understand that the matter is not easy and there are many needs to be taken into account, but what scary me a bit is that this transaction alone, after a few days, allows to group dozens of other transactions and in a few months there will probably be hundreds ... does this loss of privacy really make sense to avoid further fragmenting the utxos? in my view, if there is a subset of utxos within the bitcoin-core chosen utxos group, which is sufficient to reach the amount to be sent, the utxos that are not in this subset are just an unsolicited consolidation. – leevancleef Mar 25 '21 at 00:04
  • I've updated my answer to specify why there is a minimum change target. – Ava Chow Mar 25 '21 at 00:05
  • @PieterWuille but they're not as simple as getting rid of things you don't understand If I understood everything involved and no pending PR author has issues, I would have created a PR to fix these issues. Sure I dont understand everything. But I know Bitcoin Core Coin Selection is not working as expected in many cases. You can either fix it or ignore it and also help users. –  Mar 25 '21 at 00:06
  • Related to this question is PR 4096 which does exactly as you suggest: "It will prune any extraneous inputs in the selected subset, in order to decrease the number of inputs and the resulting change." On the surface, this seems like a good idea, and many contributors agreed. However, in practice, it was found to create many small change UTXOs (as I explain in my answer) which was detrimental to wallets and caused a significant increase in the UTXO set size. This PR was later reverted in PR 8298. – Ava Chow Mar 25 '21 at 00:10
  • @AndrewChow we agree that prevent fragmentation is good and that to reduce utxos we need to consolidate, but why consolidate randomly (as happen in this way) is better than consolidate only when necessary? this behavior prevents me from consolidating the utxos by hand when I think it is more appropriate. – leevancleef Mar 25 '21 at 00:20
  • @leevancleef I never said that this approach that Bitcoin Core does is good or better. This is merely an explanation of what and why this behavior occurs. I agree that consolidating randomly is not good, and currently, Bitcoin Core's coin selection needs work. There is ongoing work (from myself) to improve the coin selection algorithm, but it has not garnered much review. However consolidation should not be left up to the user to do as many users are not aware of consolidation or knowledgable about UTXO management in general. – Ava Chow Mar 25 '21 at 00:25
  • A strategy that is in the works to improve this is to use a more consolidatory algorithm when fees are low, and use a less consolidatory one when fees are high. The current coin selection algorithm is always consolidatory and does not take into account the current feerates and this is something that I am trying to change. – Ava Chow Mar 25 '21 at 00:26
  • happy to know these things, for my need I will try to find a selection algorithm that takes privacy as a first requirement ... i wonder if there are other business that need to send automatic payment respecting privacy, but probably the most need the opposite :D – leevancleef Mar 25 '21 at 00:38
  • If your primary goal is non-linkage between addresses, the only real solution is having completely separate wallets (or very rigorous use of manual coin control). – Pieter Wuille Mar 25 '21 at 00:39