0

Currently, Every Bitcoin Core full node has to download the whole blockchain at least once. This situation is even worse if pruning is enabled - the user has to download the whole gigantic blockchain all over again if a rescan is required (typically after importing privkeys/addresses/wallet.dat).

I once heard that "backward syncing" might help. I also heard that "UTXO commitment" could eliminate the need for downloading old blocks. I wonder what exactly are them? Is there any relationship between them? Is there any technical hurdle that is very hard to overcome?

Chris Chen
  • 944
  • 4
  • 15

1 Answers1

1

In bitcoin we add unspent entries to the UTXO when created inside a block, and delete them when they are spent. Syncing “in reverse” would be a messy process that wouldn’t allow for the verification of authenticity of the blocks being imported until the sync was entirely complete, where going forwards we have complete confidence in blocks as they progress.

The software can reverse blocks to undo changes to the state during a block reorganization, but this is excruciating slow as it requires going back to the older blocks using a reference list and reinserting the UTXO entries that were previously removed.

"UTXO commitment" could eliminate the need for downloading old blocks.

UTXO commitments don’t remove the need for full validation, they introduce a new weaker level of node operation where they trust that miners have included valid commitments.

UTXO commitments also require an extremely large amount of hashing to achieve, which makes them implausible for use in the production network without other changes.

the user has to download the whole gigantic blockchain all over again if a rescan is required (typically after importing privkeys/addresses/wallet.dat).

BIP157 allows for wallet rescans without the block data being retained.

Claris
  • 15,408
  • 2
  • 26
  • 43
  • I see Bitcoin Core enables assumevalid by default, so by default it doesn't actually fully validate old blocks independently, right? In other words, the user trusts the developer who hardcoded the block hash if he never sets assumevalid to 0. – Chris Chen Jan 18 '19 at 04:00
  • This is to prevent a denial of service where you produce infinitely large numbers of difficulty 1 blocks. Only signature validation is skipped, and this peg hasn’t been updated in many years. – Claris Jan 18 '19 at 04:01
  • Checkpoints, instead of assumevalid, are intended to defend against DoS attacks, which is a legacy that hasn't been updated in many years as far as I know. – Chris Chen Jan 18 '19 at 04:04
  • As for BIP157, I heard it's for lightweight clients. I wonder if it's possible that a malicious full node feeds incorrect data to the client intentionally? – Chris Chen Jan 18 '19 at 04:04
  • You can make your own 157 filters and discard the blocks, giving you a compact data set to rescan from without trusting anybody else. – Claris Jan 18 '19 at 04:07
  • 1
    assumevalid only disables script verification as that is expensive. Everything else (PoW, UTXOs, merkle root, etc.) is still checked. Assumevalid and checkpoints are actually very similar, they even share the same code. That's because assumevalid was designed to have the performance benefits of checkpoints without the requirement of blocks needing to be in the blockchain. Neither really defend against DoS attacks; rather that's what minimumchainwork is for. It requires that the cumulative chainwork must be at least that value. – Ava Chow Jan 18 '19 at 04:53
  • 1
    When Bitcoin Core starts using BIP 157, it won't be for syncing faster. Rather it will generate the filters for every block it receives and store that. The filters allow rescans to happen much faster, regardless of whether the node is pruned or not. In both cases, the filter will indicate whether a block contains a transaction pertaining to the wallet so the block can be fetched (either from disk or over the network) and the necessary transactions pulled out. BIP 157 helps both pruned and archival nodes at no security risk. It only affects disk space and sync time (making filters takes time). – Ava Chow Jan 18 '19 at 04:56
  • @AndrewChow Why minimumchainwork is intended to defend against DoS attack? I see no block height data tied to it. – Chris Chen Jan 18 '19 at 06:32
  • I also found this: https://bitcoin.stackexchange.com/questions/76018/how-does-headers-first-prevent-fill-disk-attack?noredirect=1&lq=1#comment87815_76021 – Chris Chen Jan 18 '19 at 06:33
  • minimumchainwork does not need to have a block height tied to it. The chainwork is what determines what the best blockchain is. The blockchain that your node uses is the one with the most chainwork, not the one with the most blocks. Otherwise I could trivially make a blockchain that is millions of blocks long with all blocks being difficulty 1. minimumchainwork protects from those kinds of DoS attacks because it immediately ignores those low difficulty blocks since the blockchain they are part of does not have sufficient chainwork. – Ava Chow Jan 18 '19 at 16:32
  • @AndrewChow According to the link above, I see minimumchainwork doesn't prevent header-fill attacks, since it actually allows the attacker to produce trillions of garbage block headers, which would trick a node into meaningless header validation work. – Chris Chen Jan 18 '19 at 19:34
  • Checkpoints don't prevent that either. You can make trillions of garbage block headers now and trick a node into meaningless header validation work. Validating headers is not particularly expensive though. Checkpoints make it harder to create a large low work fork, but minimumchainwork prevents your node from being forced to validate all of those blocks (you would still validate the headers). – Ava Chow Jan 18 '19 at 22:36
  • Validating a header is almost free, fortunately, though connecting them and storing them is not. – Claris Jan 18 '19 at 22:37