The Guru College

ZFS Dedup

It’s made the putback, and ZFS dedupe is on it’s way to us. The scheduled build for putback is b128 – which should hit our grubby little mitts in about a month’s time. Using zdb -S, I was able to pull a block signature list for my root pool (rpool) on the secondary fileserver, thor. There are currently 473,912 unique non-zero blocks in the pool, of which 376,823 are unique. This suggests I can dedup more than 97,000 blocks – or just over %20 of the blocks in the pool. I didn’t count block sizes directly, but the average blocksize on the rpool is just over 32K. That would be close to 3GB saved on a 15GB pool. This piqued my interest. I ran the same block signature list generator from the zpool I had been sending Time Machine backups to. Total: 3.43 million blocks. After dedup’ing out duplicate checksums – 2.39 million. That’s a little over %30 of the blocks that could be dedup’ed. Here’s the kicker – the average blocksize for the pool is 124K. That’s over 120 GB of duplicate blocks, using my fuzzy math.

Thanks to Jeff Bonwick and all the others at Sun who have made this happen. The only other distantly promised feature I wait for now is the ability to add and remove top-level vdevs from raidz/raidz2 pools.

Photo Management | Home | Schumpeter’s Gale