Opened at 2007-10-04T19:19:29Z
Closed at 2009-03-29T05:02:42Z
#167 closed enhancement (fixed)
try out Jerasure
| Reported by: | zooko | Owned by: | zooko |
|---|---|---|---|
| Priority: | minor | Milestone: | undecided |
| Component: | code-encoding | Version: | 0.6.0 |
| Keywords: | Cc: | ||
| Launchpad Bug: |
Description
Professor James Plank has published a new library that has implementations of various erasure coding algorithms, Jerasure:
http://www.cs.utk.edu/~plank/plank/papers/CS-07-603.html
It would be fun to benchmark these against the current zfec
http://pypi.python.org/pypi/zfec
and if some algorithm in there is faster, switch the internals of zfec to use it instead.
Change History (3)
comment:1 Changed at 2007-10-04T19:19:34Z by zooko
- Status changed from new to assigned
comment:2 Changed at 2008-06-01T20:58:19Z by warner
- Milestone changed from eventually to undecided
comment:3 Changed at 2009-03-29T05:02:42Z by zooko
- Resolution set to fixed
- Status changed from assigned to closed
Note: See
TracTickets for help on using
tickets.
Prof. James Plank and co-authors (including me) did some good investigation about comparative performance:
http://www.cs.utk.edu/~plank/plank/papers/FAST-2009.html
The upshot is that zfec is definitely fast enough, and is actually competitive with the others in the sense of being consistently fast on different machines and different K and M combinations. The paper doesn't show graphs of numbers of check blocks (a.k.a. secondary shares) larger than 4, but I think the alternatives get slower with larger numbers of secondary shares faster than zfec gets slower. Also note that Jack Lloyd has contributed some code for zfec to use SIMD when appropriate for even better performance. Closing this as "fixed".