The Guru College

Dropbox Pro Quota Levels

I was just notified that Dropbox Pro is 10x’ing their paid storage plans – this is good news for my family and I. I had been thinking of moving off Dropbox and over to iCloud Drive this fall when Mac OS X Yosemite ships (assuming it works), but I’m less likely to move now.

Which is probably exactly the reason Dropbox is doing this.

Motorola Surfboard Metrics

Announcing another quick-and-dirty perl script today: surfboard-metrics

I have been having a lot of trouble with my ISP over the last few weeks – constant disconnects triggered by a modem reboot – which were taking us offline for 3-5 minutes at a time (or longer). This would be a mere annoyance, other than the fact that I work from home pretty frequently, and the reboot takes the connection out long enough to drop VPN and ssh sessions. The longest outage to date is 4 hours. I finally decided to start collecting data to see if I needed to add a powered amplifier to my cable system.

I’m using a Motorola Surfboard 6121, and while the spec sheet lists SNMP v2 and v3, they appear to only allow SNMP access over the coax interface. It’s for the ISP to use, not for the end user. This means screen-scraping the web interfaces, which by default live at There is also a log event page at but I’m not dealing with that yet.

The important thing to watch for are power levels and the signal-to-noise ratios for the upstream and downstream channels. If upstream goes above 55dBvM, or if downstream gets much above 40dB, performance will go to crap, and you will start seeing No Ranging Response received - T3 time-out or Unicast Ranging Received Abort Response - initializing MAC and the modem will eventually reboot itself.

The script I wrote screen scrapes the pages, and outputs to a graphite/carbon server all the various metrics. I’ve tried to make it reasonably flexible – if you have a higher speed connection and have multiple bonded upstream or downstream channels, the script should be able to create a metric for each channel – but as I only have my modem for reference, I can’t verify some of that. As it might be helpful, I’ve put it on github. If you have a Motorola Surfboard, give it a spin. I welcome pull reuquests that add support for ohter modems or other outputs to metric engines.

Here’s a screen capture of my local grafana instance, looking at recent data:

Screen Shot 2014-07-12 at 12.15.51 PM

You can see a reboot at 711, around 14:30, and again on 712 at 11:30. I think the next step is to correlate temperature and humidity readings, as well as internet traffic levels, and see if any patterns emerge.

Aperture is Dead

Apple has decided to kill Aperture. I left Aperture years ago due to performance issues when the library got large enough. Now, it will be time to get my wife off of it. I hope that “Photos for OS X”, coming in the spring, is good enough. I loathe the idea of giving Adobe any money for Creative Cloud, but I don’t think that DarkTable is friendly enough for my wife to use every day.

FreeNAS: First Impressions

I’ve been “production” on my new FreeNAS based file server for a few days now. I finished the data migration over the weekend. I’m still drinking in the differences, and trying to get a handle on all the ways FreeNAS (and recent FreeBSD builds) are different than an years-old install of OpenSolaris. I am mostly impressed. I am going to miss FMA on Solaris and a few of those sort of features that made the OS so incredibly good (cfgadm comes to mind), and I’m not sure how FreeNAS handles drive failures and removals – I’m sure I will find out over the next weeks, months and years.

The cautionary tale I have for today is using “AFP” shares in FreeNAS, which I assume is netatalk. My photo library lives on the NAS, and has for years. One of the things I was looking forward to moving off OpenSolaris for was a better stack for CIFS/AFP file services. This evening, I finally settled down to get some image editing done – and the system was so incredibly slow as to be unusable. Going into the Finder, pulling a directory listing was taking 20-30 seconds. I was starting to lose my mind, but I decided to fall back to NFS, which I had been using to move data around, and had seemed quick for bulk transfers. Everything was lightning fast.

I don’t have anything conclusive yet, but it seems that NFS is an order of magnitude faster than AFP, out of the box, on FreeNAS. I’ll look into this further, but I’m tired, and I don’t want to try to dig up numbers tonight.

Personal Projects: Moving to FreeNAS

Another personal technology project I have put into motion this summer is abandoning OpenSolaris once and for all and moving to FreeNAS. This involves a totally new machine, which has been ordered, and will move the data from 4 pairs of mirrored disks to an 8 disk RAIDZ2. This gives me much better data protection and a lot more usable space.

I’m buying NAS-rated drives for the RAIDZ2 pool, which is for general file and media storage. NAS-rated 2TB drives are now $99 on NewEgg. Using RAIDZ2 lets me lose any two drives simultaneously and not have a loss of data. I can currently lose 4 drives in my mirrored pair setup, but if any two drives are from the same pair, the whole pool is lost. Considering the age of the oldest drives, I’m not liking my odds any more. I was looking at 3TB and 4TB drives, but I’m not comfortable with the URE rates and rebuild times when larger drives are involved, so I’ve stuck with the smaller drives.

I have also moved to a Xeon processor, 16 GB of ECC RAM and a new PCIe 2.0 LSI HBA. This is going to give me significant throughput gains over my current setup, and the 16GB of RAM should help buffer when I move from mirrored pairs to RAID. The smallest files I do work with are Nikon NEF files, which are 12MB+ each, so I don’t have as much issue with small file I/O as some people do.

I toyed with the idea of getting a SSD for a slog or an L2ARC, but I decided against it for now. The ZIL/slog is most useful for doing random NFS writes, which I do very little of, and the L2ARC has a memory cost that is pretty steep – 5GB of L2ARC consumes 1GB of system RAM. A 60GB SSD would chew through 12GB of RAM, and I’m only buying 16GB to start with. There are two slots open on the new motherboard, so I can add more RAM if needed, and that is a much more performant place to add ARC.

The order is placed and the machine will be here over a couple of days next week. I’m looking forward to building a new system. I’ll take pictures as I build and post them here.

The New Apple

Last week, Apple held WWDC 2014, their annual developer’s conference. A number of tech analysts have been bemoaning the lack of new laptops, beating the “Apple is doomed” drum because Apple didn’t introduce a supersized iPhone, or losing their minds and credibility by saying that if Apple doesn’t release a watch/fitness device in the next 60 days they will become irrelevant.

Since Apple dropped attendance at MacWorld, there are now pretty much two Apple events every year that you can count on: the iPhone Event in September and WWDC in the summer. There are often other events sprinkled in, depending on product readiness and release cycle, but those are the two that you can take to the bank. The a lot of industry analysts have come to expect hardware miracles every time an iPhone is released, and now, every time WWDC happens. They are used to Apple holding it’s cards very close to it’s chest.

What they missed is that Apple has played it’s hand. In full. What they were announcing last week was, as best as I can tell, a New Apple. This is the point where people will look back and say “2014 was the year that Tim Cook’s leadership really started being evident outside of Apple”. Not in the supply chain side of things, but in terms of making Apple focus on tearing down silos and building a better sense of community and collaboration, both within Apple and with the larger developer community.

Apple has given developers a lot of the things they have been asking for for years. Opening up inter-app communication on the iPhone is the most obvious part of this, and everything it enables. App store metrics, beta testing services, new languages, closer integration between the desktop and mobile, access to the raw data in iCloud and so much more. Pretty much they only thing they didn’t do was replace HFS+ and provide an SDK to program for the Apple TV.

Apple is also taking a more realistic approach to it’s fabled secrecy. The NDA’s on the conference, for example, don’t prohibit attendees from talking about the iOS 8 or Yosemite betas. All the conference videos are posted – along with their slide decks – out in public. You don’t need an Apple Developer login to get to them. You don’t need to sign a NDA to agree not talk about the session content.

That’s not to say that Apple isn’t being secretive. I’m not sure how many companies can keep a whole new language a total secret for as long as Apple did with Swift. Nobody really knows what they are doing with sapphire crystal. The Apple TV and the Mac Mini are in desperate need of a refresh, but there’s nothing leaking about where those products are going.

Personally, I’m happy Apple focused on the Developer part of the World Wide Developers Conference. That’s where the focus should be. We’ll see where all of this goes, and if it’s part of a larger trend or just a one-time event. I’m betting that this is a New Apple, and that we are going to see a lot more WWDC’s in the future like this one.

Between iCloud and ownCloud

I don’t see much of a need for Dropbox anymore. It costs a lot more than the alternatives ($0.10/GB/month vs $0.02/GB/month), now that Apple has announced prices for iCloud Storage that are on par with the Google Drive storage prices. Further, the Dropbox iOS app doesn’t sync for photo bursts on iOS7 (we don’t know about iOS8 yet), and Dropbox seems dedicated to bringing out new services like Carousel and Mailbox – which are services I don’t want. Finally, if Apple actually fixes Photo Stream, I won’t be using any of the features Dropbox provides other than occasionally sharing folders with other Dropbox users. I’m pretty confident that my shared folder use will fit within my free Dropbox quota.

To further compound issues, if Photo Stream actually works in iOS8, I will probably move my file sharing over to a self-hosted copy of ownCloud. No storage limits, and the files are local to my fileserver. Which can easily be backed up to CrashPlan. There is also a chance that I will be able to use iCloud Family Sharing to share purchased quota, even further driving down my monthly costs.

Of course, all of these are forward looking statements based on a product announcement yesterday. No changes will be made until after iOS8 ships and I can see if it works in practice.

Newer Posts | Home | Older Posts