[volunteergrid2-l] Should we increase the node minimum capacity?

[Jody Harris]

Both of my nodes are currently just below the 500 GB limit. I honestly
thought it would never be a problem.

One node is a little net top box from System76 -- I can add a 2 GB external
drive to that one.

The other is my primary home desktop (always on), Tahoe is running on a LVM
comprised of a couple of retired 250 GB drives. I can just replace those
with a single drive and move on.

I am actually pleased to have this problem!

Jody

575-208-4567
Think carefully
On Nov 2, 2011 3:08 PM, "Shawn Willden" <shawn at willden.org> wrote:

> At present, we've specified that nodes should offer a minimum of 500 GB of
> storage and a maximum of 1TB.  The reason for having an upper limit is so
> we don't get into a situation where one user provides, say, 20 TB and then
> expects to be able to *use* 20 TB of storage.
>
> To see why that would be a problem, assume we had 14 nodes with 500 GB
> each, all empty, and one node with 20 TB.  This means the grid has a total
> of 27 TB of storage... but only 7.5 TB of that is usable. Why?  Because
> that 7.5 TB would get spread evenly across all 15 nodes, at which point 14
> of them would be full and one would still have 19.5 TB free -- but we don't
> want to put all of our data on that one node; that's the whole point of the
> grid!
>
> Given N servers, the grid begins to "degrade" (have servers that can't
> accept more data) as soon as N*s data has been stored in it, where s is the
> capacity of the smallest server.  With our specified minimum size of 0.5
> TB, and 14 nodes, that means our grid has a non-degraded capacity of 7 TB.
>  This means that one user can "swamp" the grid by uploading 7 TB, even if
> that user is being perfectly fair.
>
> To prevent this, we put a cap of 1 TB per node (technically, nodes are
> allowed to provide as much as they want, but users can't consume more than
> min(1TB, node_capacity).  At that rate, 7 users maxing their usage could
> swamp the grid, so it's still a potential problem, just much less likely,
> which is why I originally suggested that we stick with a 2:1 max/min ratio.
>  I still think that's a good idea, although our present usage is such that
> we're far, far from having any problem.
>
> However, I find that I may actually want to back up more than 1 TB (after
> FEC expansion) -- but that's only possible if everyone on the grid is okay
> with increasing the minimum node size.
>
> So, what do all of you think?  Is it feasible to raise the minimum node
> size to 750 GB?  Or 1 TB?
>
> I know we actually have one node in the grid with < 500 GB, so talking
> about going even higher is going to be a challenge, but I want to get a
> feel for what everyone thinks about this.  This doesn't have to be an
> immediate thing, either... especially given that HDD prices are spiking a
> bit right now.
>
> --
> Shawn
>
> _______________________________________________
> volunteergrid2-l mailing list
> volunteergrid2-l at tahoe-lafs.org
> http://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l
> http://bigpig.org/twiki/bin/view/Main/WebHome
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/cgi-bin/mailman/private/volunteergrid2-l/attachments/20111102/94d4d2f5/attachment.html>