[volunteergrid2-l] Solving hardware problems with software

[Steve Dodson]

Just now catching up on some email; Thanks Shawn for posting this.  Had 
never heard of ATA over Ethernet.  Fascinating server setup; what do you 
use to monitor your drives / RAID?

On 02/09/2011 08:35 AM, Shawn Willden wrote:
> This really has very little to do with the grid, but it's something I
> thought was cool and that the folks on this list might appreciate.
> It's about using ATA over Ethernet to solve disk connection problems.
>
> My file server (which is the box that hosts my Tahoe nodes) has a
> bunch of disks in it, all broken into small pieces with partitions,
> when are then recombined in various ways in RAID arrays, which are
> then pooled in LVM volumes, which are then carved out into logical
> volumes for use.  All very complicated and very flexible.  I love it.
>
> However, I find myself running low on disk space (not on the volumes
> hosting Tahoe nodes, elsewhere), so I recently bought a 2 TB drive,
> with the intention of replacing one of the existing 500 GB drives.
> Doing the replacement involves temporarily adding the new drive in
> addition to the existing drives (I could just yank-and-replace, but
> that would result in my RAID arrays being degraded longer than
> necessary).
>
> So, to do that, I bought an inexpensive external drive adapter.  It
> connects via either USB or eSATA and has a slot where you can
> "hotplug" a SATA drive, either 3.5" or 2.5".  Very nifty, right?
> Well... it turns out that my server has only USB1.1.  So when I
> connected the drive via USB and started rebuilding an array with it,
> it was sloooowww.  So slow that it was going to take me two weeks just
> to get the new drive integrated, and during that entire time I'd have
> at least one array degraded.  Not such a big deal for the RAID6
> arrays, but degraded state leaves RAID5 very vulnerable.
>
> So, I ordered a PCI card with an eSATA port.  That should be fast,
> right?  Indeed it should.  Unfortunately, it didn't work at all.  The
> card works; it has some internal ports which I'm using, but connecting
> its external eSATA port to the adapter produces exactly nothing.  The
> machine doesn't even recognize the drive's presence.
>
> Finally, I remembered hearing something once about ATA over Ethernet.
>   So, I found a tutorial, and it looked simple enough.  I installed the
> 2 TB drive in my desktop machine, and ran:
>
> sudo aptitude install vblade
> sudo modprobe aoe
> sudo vblade 0 0 eth0 /dev/sdb
>
> Then I ssh'd into my file server and ran:
>
> sudo aptitude install aoe-tools
> sudo modprobe aoe
> sudo aoe_discover
> sudo aoe_stat
>
> ... and I found I had access to a new, 2TB, AOE device!  Looking in
> /dev/etherd I found that there were device nodes for each partition on
> the device.  I proceeded to add one into a RAID array and start the
> build process in motion.  Over Gig-E it's darned near as fast as if
> the drive were installed in the file server, and it works like a
> charm.
>
> This isn't a long-term solution.  As soon as the RAID arrays have
> integrated the new drive I'll shut the file server down, swap out the
> now-unused 500 GB drive with the now-integrated 2 TB drive, and it
> should start right up.
>
> Anyway, the point of this whole thing is... Linux is cool :-)
>

-- 

soli Deo gloria