[tahoe-dev] Tahoe use in a small cluster

[Paul Boulanger]

My apologies on my previous empty message.

I'd like a little feedback before I dive into a full buildout  as my intended usage for Tahoe-LAFS deviates from the norm.

My intended use is in a datacenter. I plan on building 5 storage nodes each with 2TB drives. These would be "all up, all the time" nodes with client app using them in 3 of 5 configuration.  The clients (and introducer) would be in a vmware stack (for fault tolerance) and accessed solely by an in-house program. Basically, I would generate reports and put them into Tahoe, which would then be a cheaper, more secure, slower (but faster than generating them on the fly) solution than overloading an expensive iSCSI SAN with what is essentially archive documents.

My math tells me that for files averaging 5MB, 3 of 5 would take 5MB*(1/3)*5 = 8.33MB of storage. So with these 5 nodes, I would get enough storage space for 10TB/8.33MB = 1,200,000 files.

I've also created Cacti (http://www.cacti.net/) host templates and scripts to pull activity data from tahoe storage nodes (>= 1.7.1) and tahoe client nodes (>=1.8 when JSON output was added) which I'd like to contribute back to this project but am unsure on what the best procedure to do this is or even if there is any demand for this.

Thoughts on my math and proposed usage would be appreciated. I'm basically saying that I expect nodes to be up, thus tossing 3 of 10 away. I'm still not particularly clear what the resolution will be on a node failure (few and far between, but do I need to dump and reload every file to "rebalance" ?)

--paul

Paul Boulanger
Sea Networks
paul at seanetworks.com


On Apr 5, 2011, at 3:27 PM, Paul Boulanger wrote:

> 
> Paul Boulanger
> Sea Networks
> paul at seanetworks.com
> 
> 
> On Apr 4, 2011, at 5:34 PM, Zooko O'Whielacronx wrote:
> 
>> Hi! I'm glad you're messing tahoe-w32-client. I don't know for sure
>> what the "itech" branch is for. Peter Secor would.
>> 
>>> c:\...>python extractver.py
>>> Traceback (most recent call last):
>>>  File "extractver.py", line 1, in (module)
>>>    from darcsver import darcsvermodule
>>> ImportError: No module named darcsver
>> 
>> You can install darcsver from here:
>> 
>> http://pypi.python.org/pypi/darcsver
>> 
>> Or by running "easy_install darcsver".
>> 
>> Let us know what happens next! :-)
>> 
>> Regards,
>> 
>> Zooko
>> _______________________________________________
>> tahoe-dev mailing list
>> tahoe-dev at tahoe-lafs.org
>> http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
> 
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at tahoe-lafs.org
> http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev