[tahoe-dev] Fwd: best practice for wanting to setup multiple tahoe instances on a single node

[Jimmy Tang]

doh, I forgot to hit reply all again, sorry.


---------- Forwarded message ----------
From: Jimmy Tang <jcftang at gmail.com>
Date: Wed, Jan 11, 2012 at 8:20 AM
Subject: Re: [tahoe-dev] best practice for wanting to setup multiple
tahoe instances on a single node
To: Greg Troxel <gdt at ir.bbn.com>


On Wed, Jan 11, 2012 at 12:30 AM, Greg Troxel <gdt at ir.bbn.com> wrote:

>
> I think the key point is about the redundancy that you have vs the
> redundancy that tahoe perceives - it seems dangerous to have 20 nodes
> that appear independent but are all actually on the same box.  If they
> are on 20 physical disks that are independent enough that if the box
> fails you can reconstitute all but of the nodes, it might be ok, but it
> seems subject to correlated failures.

I agree with the above comments, short of installing solaris and
running zfs I don't really have too much choice. I was hoping that I
could set a failure group for a bunch of tahoe nodes to get around the
problem (this is how we setup GPFS in work with really unreliable
nodes, we mark a bunch of independant disks in a node or a group of
nodes as failure groups to keep the storage balanced and replicated to
provide reliable storage)

>
> If there were a way to have the nodes express their correlation groups,
> and the share placement be aware of this, then I think it might be ok.
> But we don't have that yet, and thus it seems to me that if you want the
> survivability properties tahoe gives you, you should run only one node
> per computer.  And perhaps only one node per site, if you want that kind
> of redundancy.
>
> Have you measured performance?  I have the impression that if you don't
> value the decentralized distributed resilient nature of tahoe, it's not
> a good choice.
>

I haven't measured performace, but I do value the k-of-n
characteristics, so instead of doing full replication of my data and
losing 50% of my space I could start doing 3-of-4 or 3-of-5 and start
to reclaim some of the space, I think my nodes in my private setup are
reliable enough for me to do this and I happy enough to take this risk
with my data.

Jimmy.


--
http://www.sgenomics.org/~jtang/


-- 
http://www.sgenomics.org/~jtang/