[tahoe-lafs-trac-stream] [tahoe-lafs] #393: mutable: implement MDMF

[tahoe-lafs]

#393: mutable: implement MDMF
-------------------------+-------------------------------------------------
     Reporter:  warner   |      Owner:  zooko
         Type:           |     Status:  new
  enhancement            |  Milestone:  1.9.0
     Priority:  major    |    Version:  1.0.0
    Component:  code-    |   Keywords:  newcaps performance random-access
  mutable                |  privacy gsoc mdmf mutable backward-
   Resolution:           |  compatibility forward-compatibility review-
Launchpad Bug:           |  needed
-------------------------+-------------------------------------------------

Comment (by kevan):

 Replying to [comment:134 davidsarah]:
 > I'm reviewing [http://tahoe-lafs.org/trac/tahoe-
 lafs/changeset/5125/ticket393-MDMF-2?contextall=1 changeset 5125]. It's a
 very substantial rewrite of the original code, which feels risky -- can
 you explain why this much of a rewrite was needed to support multiple
 segments? The new code is used for both SDMF and MDMF, which means there
 is a significant risk of regressions in SDMF support.

 The short answer is that it probably isn't necessary to support multiple
 segments (and that I agree with you about the riskiness of a rewrite).

 When I wrote the publisher, part of the goal of MDMF was to publish MDMF
 files segment-by-segment (that is, to write the encrypted+encoded blocks
 to the grid as they are produced on the client), as immutable files are
 published. The patch attached to comment:39 contains a publisher that
 handles this sort of publishing, which is heavier on deferreds/callbacks
 and coordination logic than the current mutable publisher. I can't speak
 precisely to my reasoning at the time I wrote that; probably I felt that
 there wasn't a good/clean/maintainable way to add that sort of
 coordination logic to the current uploader without a substantial rewrite
 (maybe that was hasty; I recall exploring smaller, less invasive changes
 before the rewrite and not liking them for various reasons, but it's
 possible that I missed something simple). Anyway, comment:38 describes an
 issue with segment-by-segment publishing for mutable files, and is
 ultimately why we don't publish segment-by-segment now. I guess at that
 point I felt that it was easier to tweak the revised publisher to work
 with the relatively minor {{{layout.py}}} changes induced by comment:38
 than to go back to the drawing board and re-evaluate the existing publish
 code in the context of the modified publishing semantics (a significant
 oversight, in retrospect, since so many of the publisher changes were due
 to the segment-by-segment publishing semantics).

 I can take another look at the existing publisher to see if it can be made
 to handle MDMF with fewer/smaller changes. I can't say how long that would
 take (or whether I'd be done by the 09/01/2011 deadline for 1.9). It does
 seem like a good idea, though.

 > Also, it seems like there's a lot of near-duplicated code between the
 update and publish methods of the Publish class (update is new).

 Yes, that setup logic seems like a good target for refactoring.

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/393#comment:135>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage