[tahoe-lafs-trac-stream] [tahoe-lafs] #1523: MDMF upload via web-API uses much more memory in the gateway process than expected

[tahoe-lafs]

#1523: MDMF upload via web-API uses much more memory in the gateway process than
expected
-------------------------+-------------------------------------------------
     Reporter:           |      Owner:  davidsarah
  davidsarah             |     Status:  assigned
         Type:  defect   |  Milestone:  undecided
     Priority:  major    |    Version:  1.9.0a1
    Component:  code-    |   Keywords:  mdmf memory-leak tahoe-put
  frontend-web           |  performance
   Resolution:           |
Launchpad Bug:           |
-------------------------+-------------------------------------------------

Comment (by davidsarah):

 For an upload using {{{tahoe put --mutable --mutable-type=mdmf}}} of a
 100000000-byte file, the peak gateway memory usage is 713.8 MiB RSS, and
 the final usage is:
 {{{
   PID   RSS    VSZ S TTY          TIME COMMAND
 15837 40124 164148 S ?        00:00:00 [...]/tahoe restart
 ../grid/introducer
 15853 162204 245664 S ?       00:00:04 [...]/tahoe restart ../grid/server1
 15863 155156 238736 S ?       00:00:04 [...]/tahoe restart ../grid/server2
 15873 119860 203288 S ?       00:00:02 [...]/tahoe restart ../grid/server3
 15883 119756 203440 S ?       00:00:02 [...]/tahoe restart ../grid/server4
 15893 404656 532308 S ?       00:00:33 [...]/tahoe restart
 }}}
 (i.e. 395.2 MiB RSS for the gateway).

 So, the major part of the leak is, unsurprisingly, roughly proportional to
 the filesize.

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1523#comment:14>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage