[tahoe-dev] Tahoe on large filesystems

[Shawn Willden]

On Fri, Feb 4, 2011 at 3:51 AM, Jan-Benedict Glaw <jbglaw at lug-owl.de> wrote:

> Consider a 2TB
> filesystem on a 2TB disk. Sooner or later, you will face a read (or
> even write) errors, which will at least easily result in a r/o
> filesystem. For reading other shares, that's not much of a problem.
> But you're instantly also loosing a hugh *writeable* area.
>
> So with disks that large, do you use a small number of large
> partitions/filesystems (or even only one), or do you cut it down to,
> say, 10 filesystems of 200GB each, starting a separate tahoe node for
> each filesystem. Or do you link the individual filesystems into the
> storage directory?
>

I don't think using lots of partitions really helps.  I've long used many
partitions on my big disks (starting back when 10 GB was a "big disk") for
reasons of flexibility.  I have multiple large disks, each broken into many
partitions, then I create RAID arrays on the "parallel" partitions,
including one from each disk, then bind the RAID arrays together with LVM
and finally carve out logical volumes for actual use.  Without getting into
the advantages/disadvantages of that approach, the reason I mention it is
because what I've observed is that when a disk gets an I/O error on any one
of the partitions, the OS assumes that the whole disk is having trouble and
drops _all_ the partitions out of their RAID arrays.

I believe the same thing happens if you place file systems directly on the
partitions; an I/O error on one of them will cause them all to be put in
read-only mode.  Given that, unless you have other reasons to prefer many
partitions, I think a single big partition makes more sense.


> Running like 10 tahoe nodes on one physical HDD would create another
> problem: what if all (or most of all) shares get stored to that single
> HDD, with all being lost with a single drive crash?
>

Yeah, multiple nodes on one HDD hugely increases the impact of a common
failure mode.  I think it's just a bad idea.

-- 
Shawn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20110204/acef31b3/attachment-0001.html>