[volunteergrid2-l] Thoughts on catastrophic events and garbage collection/expiration

[Billy Earney]

Why not go with a larger expiration period (make it quite high, so that a
node could be off the network for a while, without the shares being
deleted).. maybe a year (365 days)?  That seems to be a compromise between
the two extremes.. :)

On Wed, Feb 2, 2011 at 4:11 PM, Shawn Willden <shawn at willden.org> wrote:

> On Wed, Feb 2, 2011 at 2:55 PM, Jody Harris <jharris at harrisdev.com> wrote:
>
>> If this happens with 3 members/year, that's a 2TB drive a year buried
>> under dead weight.... and it's going to be cumulative. [...] 140GB/yr,
>> expanded into ~300GB/year of accumulated and dead shares.
>>
>> ..... which in the end may not be that big of a deal.
>>
>
> If storage capacities continue to follow the curve that they have -- and we
> know that data volumes tend to track pretty closely to available storage --
> those sources of dead shares may not matter at all.  One way to think of it
> is that storage capacity and demand remains constant, but the space consumed
> by old dead stuff decreases by 50% every 1.5 years.  This exponential
> falloff of wasted space may mean that the waste never becomes relevant.
>
> Or maybe usage will climb faster than storage, or storage capacity growth
> will tail off or other users will have much more dynamic data sets (and the
> bandwidth to push them into the grid)... who knows?
>
> But my thought is that until we actually see that storage is getting low,
> why not just let the cruft accumulate?  If we get to that point, we can just
> get confirmation from everyone on the list that they've recently renewed the
> leases all the shares they care about, and then turn on expiration.
>
>
>> I guess one think that would help would be a peer-reviewed drive upgrade
>> strategy. That way, when I replace my 500 LVM in (say) 6 months with a 2 TB
>> drive, if we have <20 nodes, I might move all of the shares to the new
>> drive, but if we had >20 nodes, we might decide to just let the shares die
>> with the old drive, which would give us a somewhat graceful way to retire
>> old, de-linked shares, and active capabilities would be repaired within a
>> month.
>>
>
> My immediate reaction to this idea is that I don't like it.  Deliberately
> allowing shares to be lost when they could be retained rubs me the wrong
> way... but it really shouldn't be a big deal.  I have to think about that.
>
> --
> Shawn
>
> _______________________________________________
> volunteergrid2-l mailing list
> volunteergrid2-l at tahoe-lafs.org
> http://tahoe-lafs.org/cgi-bin/mailman/listinfo/volunteergrid2-l
> http://bigpig.org/twiki/bin/view/Main/WebHome
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/cgi-bin/mailman/private/volunteergrid2-l/attachments/20110202/124406de/attachment.html>