[tahoe-lafs-trac-stream] [tahoe-lafs] #1513: memory usage in MDMF publish

[tahoe-lafs]

#1513: memory usage in MDMF publish
--------------------------+-------------------------
 Reporter:  warner        |          Owner:
     Type:  defect        |         Status:  new
 Priority:  minor         |      Milestone:  1.9.0
Component:  code-mutable  |        Version:  1.9.0a1
 Keywords:  mutable mdmf  |  Launchpad Bug:
--------------------------+-------------------------
 I did a 'tahoe push --mdmf --mutable-type=mdmf foo' of a 210MB file. The
 client process swelled to 1.15GB RSS, making my entire system pretty
 unresponsive. The publish eventually succeeded, and the memory usage went
 back to normal.

 I'm guessing that either there's a design problem in which it's trying to
 upload all segments in parallel, or there's a failure in the Pipeline code
 such that it's holding all shares in memory at the same time.

 Since MDMF is supposed to make it possible to work with large files, I
 think the memory usage should be similar to CHK files: capped at a small
 constant times the segsize.

 It would be nice to fix this for 1.9, but since MDMF is still
 experimental, I'm willing to ship without it.

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1513>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage