[tahoe-lafs-trac-stream] [tahoe-lafs] #1496: make SFTP frontend handle updates to MDMFs without downloading and uploading the entire file

[tahoe-lafs]

#1496: make SFTP frontend handle updates to MDMFs without downloading and
uploading the entire file
------------------------------+-----------------------------------
     Reporter:  zooko         |      Owner:  davidsarah
         Type:  defect        |     Status:  assigned
     Priority:  major         |  Milestone:  1.10.0
    Component:  code-mutable  |    Version:  1.8.2
   Resolution:                |   Keywords:  sftp performance mdmf
Launchpad Bug:                |
------------------------------+-----------------------------------
Changes (by davidsarah):

 * owner:   => davidsarah
 * status:  new => assigned
 * milestone:  1.9.0 => 1.10.0


Comment:

 Replying to [comment:2 davidsarah]:
 > Replying to [ticket:1496 zooko]:
 > > It appears that the current version of the #393 branch, in the SFTPD
 frontend,
 [source:ticket393-MDMF-2/src/allmydata/frontends/sftpd.py?annotate=blame&rev=5151#L815
 downloads the entire MDMF file and then uploads the entire new version of
 it], even if the SFTP client has overwritten only a portion of it. This
 isn't a regression—Tahoe-LAFS v1.8 didn't have MDMF's at all, but did
 [source:trunk/src/allmydata/frontends/sftpd.py?annotate=blame&rev=5127#L828
 the same download-entire-file-and-upload-entire-new-version] in order to
 let an SFTP client appear to "overwrite" a portion of an immutable file.
 > >
 > > However, I think this should probably be considered a blocker for 1.9
 final.
 >
 > b) for MDMF files: when the SFTP file handle is closed, overwrite only
 segments that have changed.

 As explained in ticket:393#comment:38 and ticket:393#comment:135, the MDMF
 uploader always uploads whole shares, even if its client tells it the
 regions that have changed. So making the SFTP frontend tell it which
 regions have changed should not be a blocker for 1.9.

 (I'm not sure I agree with the reasoning in ticket:393#comment:38, but
 that's a separate issue.)

 > If we don't implement this optimization for 1.9, we would just need to
 add a note that the SFTP frontend does not have any MDMF-specific
 optimizations, so its performance for MDMF is the same as for SDMF.

 Actually, the memory usage for downloads should be better than for SDMF.

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1496#comment:3>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage