Changes between Initial Version and Version 18 of Ticket #29


Ignore:
Timestamp:
2007-06-07T18:09:37Z (13 years ago)
Author:
warner
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Ticket #29

    • Property Status changed from new to assigned
    • Property Summary changed from uses up lots of RAM to web upload uses up lots of RAM
    • Property Priority changed from major to critical
    • Property Version changed from to 0.2.0
    • Property Owner changed from somebody to zooko
  • Ticket #29 – Description

    initial v18  
    1 It seems to use up RAM proportional to the size of the file being uploaded * 4.
     1Uploading files through the 'webish' frontend (with the upload form) results in a memory footprint of at least 2 * filesize. Downloading files might do the same.
     2
     3Zooko's first observations suggest this might be more like 4x.
     4
     5The main culprit seems to be the stdlib 'cgi' module, which twisted.web uses to parse the multipart-encoded upload form. The file to be uploaded appears as an input field in this form.
     6
     7A secondary thing to look at (if/when we fix the upload side) is to make the download side streaming (producer/consumer), to avoid buffering the whole file in the twisted Transport queue.