Opened at 2011-09-02T15:52:48Z
Last modified at 2011-09-03T15:59:38Z
#1523 closed defect
MDMF upload via web-API uses much more memory in the gateway process than updating the same file via SFTP — at Initial Version
| Reported by: | davidsarah | Owned by: | |
|---|---|---|---|
| Priority: | major | Milestone: | undecided |
| Component: | code-frontend-web | Version: | 1.9.0a1 |
| Keywords: | mdmf memory-leak tahoe-put performance | Cc: | |
| Launchpad Bug: |
Description ¶
Split from #113:
The web-API interface does not support streaming (#113, #320), so it is expected for the gateway to need to hold the whole file in memory in order to upload it. However, when using tahoe put to upload an MDMF file, the increase in memory usage of the gateway process seems to be more than double the file size. For example, when uploading a 191 MiB MDMF file in 1.9alpha using tahoe put --mutable --mutable-type=mdmf, the peak RSS of the gateway (which was also a storage server in this test) was about 510 MiB greater than when updating the same file using SFTP (which streams to, and then from, a temporary file).
Note: See
TracTickets for help on using
tickets.
