[tahoe-dev] zfec right for me?

[Zooko O'Whielacronx]

Reading back through the last year's mailing list archive in search of
Kyle Markley's benchmark reports and commentary thereon, I happened to
see this letter:

http://tahoe-lafs.org/pipermail/tahoe-dev/2010-July/004688.html

I'm not sure, but perhaps Jack is overlooking an under-documented
feature of zfec. Zfec does offer a header which contains K, N (which
in zfec code and docs is named "M"), how many bytes of 0-padding are
needed, and the number of the share:

http://tahoe-lafs.org/trac/zfec/browser/trunk/zfec/zfec/filefec.py?annotate=blame&rev=363#L35

Tahoe-LAFS does not use this feature since that information is already
present elsewhere in the Tahoe-LAFS data structures.

Zfec does not have an implementation of identifiable magic numbers nor
per-share checksums. That would be useful. Perhaps a good way to get
that would be to combine zfec with Ludovic Courtès's libchop [1],
which has checksums (including Merkle Trees) but doesn't have erasure
coding.

Jack: want to share that draft of the file format that you wrote up?

Regards,

Zooko

[1] http://www.nongnu.org/libchop