[volunteergrid2-l] New grid? [ was: Membership]

[RCV's Tahoe Account]

Hi, Ed.

I, too joined this group about a month ago, hoping to learn what some of the issues were, so that
perhaps I could help start another group that would have fewer issues.  I offered disk space and
asked for the introducer furl.  Although I was admitted to this mailing list, I was not given
the introducer furl, so I have been unable to study (first hand, anyway) what the issues
might be.  I'll try to give my opinions (based on very limited knowledge) inline, below...

On Sun, Mar 17, 2013 at 11:24:43AM +0100, Ed Kapitein.org wrote:
> Hi,
> 
> i just found out that the volunteergrid2 will be terminated by the end 
> of the month.
> So joining seems to be useless ;-)
>
> I am thinking about starting a new volunteergrid, once v 1.10 is out.
> But i would like to learn as much as possible about the pitfalls 
> volunteergrid2 suffered from.
> 
> One of the things i would like to know the rationale behind some of the 
> rules on [1]
> Why just 20 members, why not 200? or 2000?
> Why at least 500G, but no more then 1 Tb?

I *think* they wanted 20 "committed" members, not 200 "half-hearted" members.  But,
I have no first-hand knowledge.

The current Tahoe-LAFS space assignment algorithm gives every active node an equal
probability of holding a share.  At least until a node fills up.  Nodes that offer
significantly less than the average would tend to fill up quickly, leaving fewer
nodes for those who add files later in the life of a grid.  I'm sure there's a
ticket that discusses this in more detail, but I don't know the ticket number.
[I *think* this is the reason for asking for a similar storage donation from each
storage server.]


> Are there more things that, in retrospect, should have been done 
> different in setting up a volunteergrid?
> How did you attract people to join? (IRC, tahoe-lafs wiki...)
> What did you do when the introducer furl was leaked?
> 
> How was the interaction between the developers and the volunteergrid, 
> once it became clear that the grid wasn't ready for "production use"?
> ( I can imagine, that some developers are actually members of the 
> volunteer grid )
> 
> Are there still members on this list that wouldn't mind contributing 
> diskspace, even when they are not going to use the v1.10 volunteergrid 
> for anything important?

I would be happy to participate in a new volunteer grid.  I am pretty sure
I would provide a custom storage server.  My storage server would refuse
to take more than a fraction (say 1/7) of the shares necessary to reconstruct
a file.  (If the parms were 7 of 10, I would take up to 1 share per file.
If the parms were 21 of 30, I would take up to 3 shares per file.  If
the parms were 2 of 3, I wouldn't take any shares.)  I also don't think
I'd accept shares if the parms didn't intend to place sufficient redundant
shares.  (If the parms were 21 of 21, I wouldn't want any shares.)  I
think the grid members should decide on a minimum redundancy, and a recommended
repair schedule, based on experience with the grid.  My rationalle
is two-fold.  First, and most importantly, I don't want to be excessively
responsible for another person's files.  While I would endeavor to maintain
high availability, many things are beyond my control.  A second component
of this reason is the repair facility is rather poor at handling a grid
that changes dynamically.  [I've suggested changes to Zooko and B.Warner to
improve handling of a dynamic grid.]  If there has been any churn to a grid
after a file was first placed, then after a repair, there will often
be more shares than desired (or desirable) on one storage server.  There
will also often be duplicate shares scattered around the grid.  By
refusing to accept excess shares, I think my storage server would tend
to keep the grid more healthy.  The second reason is less important, and is
one of liability.  If my storage server accepted a 1 of n share, it is
possible (even though encrypted) that I could be held liable for what somebody
else puts on my node.  By accepting a superminority of shares, I think
it is much less likely I could be held liable for another's actions.

(I am aware there may be some harmful consequences to this behavior.
On balance, I think it would result in a more robust grid.)


> I know this is a long list of questions, but it could save me a lot of 
> time, avoiding the problems you already have overcome.

I believe several attempts have been made for a public grid, and I think
at least two or three have failed.  I wish I understood better the common
elements that led to the failures.

Until or unless Tahoe supports multiple introducers, the introducer should be
on a well-connected node that doesn't require somebody to manually restart it
from time to time.  [I don't know about VG2, but I think this has been a thorn
in the side of some previous grids.]  Perhaps an introducer that runs on an
Amazon micro-instance (or equivalent), and which automatically restarts on
reboot or when kicked in its semi-secret restart interface by any of the grid
members would help.  More than one trusted member with access to the introducer
node also seems important.

I also think it would be helpful if grid members had some skin in the game.
Maybe users should (at least) pay their share of the introducer node's fees.
But I don't quite know how to do this, while maintaining the "volunteer"
nature of the grid.  Should those providing reliable storage be "paid" a
stipend?  Should those who use the storage expect to pay a fee to the
reliable storage nodes?  [I think some of the Tahoe-LAFS developers like
Bitcoin.  Is there a way (manually) to accept and pay real money as well
as Bitscript without incurring transaction fees that make this idea
utterly unworkable?]

I hope others with more real experience will weigh-in.

-- Rocke


> Kind regards,
> Ed
> 
> [1] https://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l
> On 03/14/13 14:53, Ed Kapitein wrote:
> >Hi all,
> >
> >I would like to use and contribute to the volunteergrid2.
> >I have 300G disk space i can bring to the party, and need to use about
> >50G myself.
> >I am running tahoe on a rpi, and that seems to work well in the pubgrid.
> >
> >Is there still room for one more member?
> >
> >Kind regards,
> >Ed
> >
> >
> >_______________________________________________
> >volunteergrid2-l mailing list
> >volunteergrid2-l at tahoe-lafs.org
> >https://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l
> >http://bigpig.org/twiki/bin/view/Main/WebHome
> 
> _______________________________________________
> volunteergrid2-l mailing list
> volunteergrid2-l at tahoe-lafs.org
> https://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l
> http://bigpig.org/twiki/bin/view/Main/WebHome