[tahoe-dev] [tahoe-lafs] #947: Add file-with-metadata caps

[Justin Stottlemyer]

I'll start off with mentioning that I'm attempting to build an extremely low
cost (but reliable) storage platform for Shutterfly.com.  If I can drive
costs down I can enable the business to store more data more cheaply and
expand the business further.

I've been considering dozens of potential DFS candidates.  I'm open to
everything. Closed source commercial products to fully unsupported open
source. I'm ok buying support, or hiring engineers to hack on it for me.

I'm looking at very  inexpensive and dense systems, I've already had a few
of these built
http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/

 I'm hopefully not going to have to do my own hardware support, but it may
end up making sense. I'm still figuring that portion out and in negotiations
with vendors and manufacturers.

I've been having some discussions with drive vendors and at this point 4T
and 8T drives are right around the corner.  Anyone that has ever waited on a
2T drive rebuild knows that RAID is not going to be an option for large
drives moving forward.   Rebuild times will range from days to weeks. This
is an unacceptable amount time to have unprotected data.

The only options at scale at this point are RAIN style erasure coding and
object style stores that recover N Objects from Y Nodes.

I had looked at TahoeFS earlier and started to write it off as a darknet
WAREZ based DFS. Some of this was based on some blackhat style conversations
and also the documentation being very unclear on sourceforge and freshmeat
as to where the actual repository was located.

When I recently got the new time lines on drives I decided that I had to
commit to erasure coding sooner rather than later.

-- Justin






On Fri, Feb 12, 2010 at 4:19 PM, Zooko Wilcox-O'Hearn <zooko at zooko.com>wrote:

> On Friday, 2010-02-12, at 16:16 , Justin Stottlemyer wrote:
>
> > Hey Tahoe Devs,
> >                   I'm interested in a commercial installation. I'm
> > interested in talking with you guys offline if possible.  If not,
> > let me know how you would like to talk.
>
> Hi there, Justin Stottlemyer:
>
> Allmydata.com, CEO Peter Secor, is currently the most experienced
> company at commercial Tahoe-LAFS installations. Write to
> <partnerships at allmydata.com>. There may be other folks out there who
> have started doing it to, in which case I encourage them to speak up.
>
> Also, speaking not as a commercial installer but as an open source
> dev, I'm sure the open source project could benefit from hearing
> about your use case, so feel free to tell us what you are planning
> and what your requirements are. I've noticed that people who do that
> on public developer lists often receive free technical advice in
> return! :-)
>
> Regards,
>
> Zooko
> ---
> Your cloud storage provider does not need access to your data.
> Tahoe-LAFS -- http://allmydata.org
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at allmydata.org
> http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://allmydata.org/pipermail/tahoe-dev/attachments/20100212/c4ac36c9/attachment-0001.htm