5

How scalable is the BerkleyDB format (the technology used within a wallet), and what are its limitations?

What technology would replace or supplement BerkleyDB to address those scalability challenges?

Michael Folkson
  • 15,313
  • 3
  • 17
  • 53
makerofthings7
  • 12,726
  • 11
  • 62
  • 130

1 Answers1

7

In the next major release (probably 0.8), the reference client will most likely switch to LevelDB instead of BDB for blockchain data. For wallets, a different solution is needed, but we'll most likely move away from BDB for those too.

BDB has several problems - that's not to say it is bad software, but our use case simply doesn't match what it's designed for. BDB is intended to provide databases with extremely high reliability, in complex multi-proces environments, on high-end systems where database logs are backed up and where upgrades are performed manually by a database administrator. BDB doesn't even guarantee compatibility between its environment files between minor releases.

LevelDB - an open-source rewrite of Google's own database system - was designed for efficiency and consistency on commodity hardware, and outperforms BDB by an order of magnitude in some settings. Preliminary tests with LevelDB show very good results.

For wallets, LevelDB would be overkill and a burden. LevelDB uses an entire directory per database (instead of a single file), and wallets are effectively a simple key-value store that's loaded entirely in memory anyway. I assume wallet.dat will be replaced by some simple but solid hand-rolled format.

Pieter Wuille
  • 105,497
  • 9
  • 194
  • 308
  • As developer of BIP32, when would you like to see wallet.dat format upgraded? – Serith Sep 23 '12 at 21:46
  • I've worked on implementing BIP32 for the reference client (but not right now); it still only required the wallet to be a generic key-value store, and that is probably not going to change. So, it's quite independent of whether wallets are still BDB or something else at that point. To answer the other question: I'd like to move away from BDB as soon as possible, but someone has to implement an alternative. – Pieter Wuille Sep 24 '12 at 10:17