[tahoe-dev] blocks instead of files?

[David-Sarah Hopwood]

Chris Palmer wrote:
> Jody Harris writes:
>> David-Sarah Hopwood wrote:
>>> The argument is straightforward. If you need k out of N shares to
>>> reconstruct a segment, but you need all segments in order to reconstruct
>>> a whole file, and the segment shares are stored on *different* subsets
>>> of k servers, then you will end up needing more than k servers to
>>> reconstruct the file.
>>
>> This argument seems to "cut both ways."

How so?

>> I tend to agree with Chris Palmer, but I'm not convinced that I'm right in
>> doing do so.
> 
> Well, it's apples and oranges. Octavia does not do erasure coding, it simply
> does dumb copying.

Replication is identical to erasure coding with k = 1, and the same argument
applies. Suppose that each segment is copied to an independent random choice
of m of the available servers. Then if all m of the servers for *any*
segment die, then part of the file will be lost. Losing part of a file
is essentially equivalent to losing all of it for most applications.

If all segments are copied to the same set of m servers, OTOH, then the file
is only lost if all m of that particular set die, which is less likely for
any given m.

> To the extent that its servers will let it (e.g. disk
> usage quotas, throughput quotas, et c. as determined by server policy), an
> Octavia client will copy the entirety of its files to as many servers as it
> can/wants to/its user configured it to. The result might be that an Octavia
> client gets all its file segments (I call them "blocks") from a single
> server until that server dies, and then the client moves on to the next
> server, and so on. Whatever works.

Tahoe can also, in principle, get the shares used to reconstruct different
segments from different sets of servers. This doesn't currently allow it to
tolerate servers dying during a download (ticket #287), but that's an
implementation rather than a protocol problem.

<http://allmydata.org/trac/tahoe-lafs/ticket/287>
'download: tolerate lost or missing servers'

<http://allmydata.org/trac/tahoe-lafs/ticket/873>
'upload: tolerate lost or missing servers'

-- 
David-Sarah Hopwood  ⚥  http://davidsarah.livejournal.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 292 bytes
Desc: OpenPGP digital signature
Url : http://allmydata.org/pipermail/tahoe-dev/attachments/20100311/c0180069/attachment.pgp