I’m trying to create a Bitcoin address database using btcrecover and Google BigQuery data. While the Ethereum database works fine, my Bitcoin database fails to return any addresses, even though test addresses and random addresses from the blockchain exist in the dataset.
Here’s what I tried:
1.Created the database using:
python create-address-db.py --inputlistfile C:\Users\test\Desktop\btc-addresses-db-20250816 --dbfilename btc-addresses-db-20250816.db --dblength 31
2.Checked addresses using:
python check-address-db.py --dbfilename btc-addresses-db-20250816.db --checkaddresslist ./addressdb-checklists/BTC.txt
python check-address-db.py --dbfilename btc-addresses-db-20250816.db --checkaddresses bc1qxy2kgdygjrsqtzq2n0yrf2493p83kkfjhx0wlh
Observations:
- Ethereum DB works with –dblength 29.
- When I create the Bitcoin DB with –dblength 31 using the full dataset (~2000 BigQuery files, 16 GB DB), it finds no addresses.
- A smaller subset (4 BigQuery files out of 2000) works correctly, with both –dblength 27 (1GB DB) and –dblength 31 (16GB DB).
- Splitting the dataset into halves or subsets leads to inconsistent results: some address ranges are found, others are not, even within the same database.
- Tried multiple Python versions (3.9–3.13) and btcrecover versions (1.6.0, 1.12.0, master) with the same results.
- 64-bit system, all dependencies installed, Rust installed, virtual environments used.
- Official Bitcoin database from the author (addresses-BTC-2011-to-2021-03-31.zip) works correctly.
Example of inconsistent results from subsets:
| File Range | Addresses Found? |
|---|---|
| 500–520 | Yes |
| 500–599 | No |
| 500–570 | No |
| 550–570 | No |
| 550–559 | Yes |
| 560–569 | Yes |
| 570–579 | Yes |
| 563–579 | Yes |
As you can see, some databases successfully returned addresses, while others did not. Initially, I thought the problem might be caused by file 570. However, in a subsequent test, the database that included file 570 worked correctly.
In another test, I created a database from files 550–570 and tested addresses from each file as samples for this database. The results were as follows:
- Sample address from file 570: found
- Sample address from file 553: not found
- Sample address from file 563: found
So even within the same database, some addresses were successfully retrieved while others were not. This pattern suggests that the issue is not simply with a single file, but might be related to how certain addresses are indexed or stored in the database.
I’ve tried this on both a Windows server and my own PC (with 32GB of
RAM). I even rebuilt the full database several times and verified the
hashes afterward to check for any storage or corruption issues, both
database files had identical hashes, so it doesn’t seem like a saving
or file corruption problem. It’s worth mentioning that I built the
Ethereum database using the same process, and that one works perfectly
fine, the issue only happens with the Bitcoin database.
Question:
Has anyone experienced similar issues creating large Bitcoin databases with btcrecover? Could this be a dataset issue, a bug in btcrecover, or a problem with how addresses are indexed when –dblength is large?