[tahoe-dev] tahoe-lafs suitable for 500TB ~ 3PB cluster?

[Zooko O'Whielacronx]

Hi Avi! Your comments were all pretty accurate.


On Sun, Apr 21, 2013 at 3:53 PM, Avi Freedman <freedman at freedman.net> wrote:
>
> We're going to be taking 9 45-disk dual E5520 24gb RAM systems
> and testing Tahoe (one proc per physical disk).  All JBOD controllers
> (LSI 1068 or such) and Supermicro JBOD/servers, though in production
> you might want higher quality stuff as the LSI backplanes in the
> Supermicros introduce latency when disks start to fail, and that could
> increase the chance of any given fetch experiencing latency unless
> or even if Tahoe-LAFS has aggressive timeouts before it grabs extra
> chunks.
>
> And Dieter, you're welcome to join in when we do it and have a local
> machine in the local infrastructure if you want to do some testing.

Yeah, it's gonna be a Load Testing Party. ☺

> I can say that our planned deployment if we were to use it for Havenco
> and/or ServerCentral would be to use enough extra parity that the expected
> drive failure rate in a reasonably worst case would allow us to have at
> least 6-9 months to worry about doing a centralized distribtued automated
> rebuild system or incent others to with $.  There are also if I understand
> correctly some security issues with having a well-connected proxy do rebuild
> on behalf of the end user but for just-efficiency for local storage for an
> enterprise all that goes out the window.

Right. This is ticket #568. See also #543 and #483.

Regards,

Zooko

https://tahoe-lafs.org/trac/tahoe-lafs/ticket/568# make immutable
check/verify/repair and mutable check/verify work given only a verify
cap

https://tahoe-lafs.org/trac/tahoe-lafs/ticket/543# 'rebalancing manager'

https://tahoe-lafs.org/trac/tahoe-lafs/ticket/483# repairer service


>
> Another set of issues to look at is around expiring old content -
> not sure if that would be a concern for you.  I think (again I'm still
> conceptual and haven't fully digested all the docs) right now the
> clients are expected to refresh the TTL on the content.  In your
> scenario that may not be an issue, and there may also already be
> a module that can do that in bulk for *.  Or maybe one could set
> everything to not-expire than manually set leases to expire for
> specific chunks?  Again apologize for my fuzziness here but someone
> knowledge-able will probably chime in, I expect.
>
> There are a bunch of tickets around this that I looked but didn't digest.
>
>> Regards,
>>
>> Zooko
>
> Avi
>
>
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at tahoe-lafs.org
> https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev