UPDATE: (06/16/2014)

This project is on hold until after I refresh my fileserver and move the environment to our new house. There are significant parts of the storage stack that aren’t cost effective enough at the moment to allow me to get 12TB of usable space online, have multi-node redundancy and bring up a multi-node virtualization environment at the moment. For now, I’m just going to execute on my FreeNAS upgrade and noodle around with a couple of low-power embedded systems. I will likely return to this project later this year or early next spring.

END UPDATE

As a preview of the tech projects I referenced in my last post, I’m setting up an oVirt cluster on my home network to help me better understand how oVirt works. The idea is that it will let me manage the various resources provided on the home network in a sensible fashion. This includes file services, backup services, and Plex when listing the visible stuff, but also includes DHCP, DNS, temperature sensors, external web services and ownCloud.

There is also the desire to really get down to the meat of this all – set DHCP options, have machines PXE boot to gPXE/iPXE and then come up off an iSCSI volume provided by my fileserver. And that’s just one part of it. I know I can do it – I’ve used these systems at work to death – but I’ve never owned the whole stack myself.

The primary enabler/driver of this is AMD’s new AM1 platform – they are cheap enough (and powerful enough) to be nodes in an oVirt cluster without breaking the bank. The motherboards are $30-$50 and the CPU’s are $30-$65. This means that with a cheap case, a power supply, CPU, RAM and motherboard you are looking at a total cost of $210 or so for a diskless node. With the APU’s TDP of something in the order of 25W, a silent 12V DC-DC ATX power supply starts to look incredibly tempting, especially if you aren’t spinning platters. If you forego the case and like to make modifications to IKEA furniture, you can do something like the Helmer cluster in a file cabinet – which is almost exactly what I want to do. Not getting a case or an AC-DC power supply drops the per node price to $195 or so, each. This is for 2Ghz x 4 core, 8GB node with a TDP of less than 35W at full draw.

The things that need to be figured out, before funding any of it, is what I’m going to need in terms of I/O requirements from the fileserver. The plan is to move to RAIDZ2 for the main file store, and set up the old 750GB drives in a triple parity mirror to maximize random read IOPS, which is the primary need for the VMs that will be consuming that pool. I’d like to get away without having to budget in SSDs for L2ARC, but that means buying pricier 8GB DIMMS.

That’s the direction I’m moving. As the project moves along, I’ll post. There are other projects as well, but this is the one that is the furthest along.