[tahoe-dev] Largest Scale of Tahoe grids

[Jimmy Tang]

Hi All,

On Wed, Nov 2, 2011 at 5:06 PM, Lars Liedtke, IAI, KIT Campus Nord <
lars.liedtke at kit.edu> wrote:

> **
>
> Hey,
>
> I wrote a mail to Zooko and he replied that i should post this here, so I
> just copied the mail to Zooko into this mail.
>
> I would be very happy, if someone could help me with this.
>
> Regards
>
> Lars
>
> >Hey,
>
>  >my name is Lars and I'm Student at the Coorperate State University of
>
> >Karlsruhe (http://www.dhbw-karlsruhe.de), Germany. During my practical
> phases
>
> > I work at the Karlsruhe Institute of Technology (http://www.kit.edu)
> and there
>
> > I have been given the Task of comparing distributed file systems.
>
> > I found out a lot about different ones and Tahoe might be the one which
> is most
>
> > suitable because of its security based concept. The Data that shall be
> saved
>
> > might be person-related and in Germany we have privacy-laws which are
>
> > protecting person-related data. Other DFS don't really care about privacy
>
> > themselves or leave it to the OS beyond.
>
> > I read your and Brian Warner's Paper about Tahoe from 2008 and there you
> say
>
> > you set up Tahoe at the allmydata.com environment. The Size of data is
> 9.5TB
>
> > there. This was good for 2008 but do you have any experience how much is
>
> > possible with Tahoe itself or how much you have running now at maximum?
> Or
>
> > where it becomes ineffective?
>
> > The Data that shall be saved in the project I work for can have several
> Peta-
>
> > Bytes or more, because they're scientific measuring data. Do you have
> some
>
> > statistics or something with which I can make a statement about "wil do"
> or
>
> > "will not do"?
>
> > The environment in which this will be set is a scentific grid located in
> one
>
> > building.
>
> > I would be very happy if you had any answers to my questions, or better a
>
> > paper about this.
>
> > Thank you in advance
>
> > Regards
>
> > Lars
>
> --
>
> Karlsruher Institut für Technologie (KIT)
>
> Institut für Angewandte Informatik
>
> Lars Liedtke
>
> Student der DHBW
>
> Hermann-von-Helmholtz-Platz 1
>
> Gebäude 445
>
> 76344 Eggenstein-Leopoldshafen
>
> Telefon: +49 7247 82-3804
>
> Fax: +49 7247 82-
>
> E-Mail: Lars.Liedtke at kit.edu
>
> Web: http://www.kit.edu/
>
> KIT – Universität des Landes Baden-Württemberg und
>
> nationales Forschungszentrum in der Helmholtz-Gemeinschaft
>
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at tahoe-lafs.org
> http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
>
>
I'm also interested in the maximum sizes of a tahoe-lafs system, given that
it is possible to build nodes with ~20tb of a filesystem with machines
bought from dell for about ~10k euro per node, you could easily have a
100tb system for less than 100k euro.

Also assuming that I do build a 100tb tahoe-lafs system across say 6
machines, what do you think the sensible maximum file sizes you would want
to use? I've been thinking about using tahoe-lafs as a storage backend for
a large scale preservation project where the average file size might be
~400mbytes up to a few gigabyte in size (i have no control over the file
sizes)

Thanks
Jimmy
-- 
http://www.sgenomics.org/~jtang/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20111104/effa35fb/attachment-0001.html>