| 3 | | The web-API interface does not support streaming (#113, #320), so it is expected for the gateway to need to hold the whole file in memory in order to upload it. However, when using {{{tahoe put}}} to upload an MDMF file, the increase in memory usage of the gateway process seems to be more than double the file size. For example, when uploading a 191 MiB MDMF file in 1.9alpha using {{{tahoe put --mutable --mutable-type=mdmf}}}, the peak RSS of the gateway (which was also a storage server in this test) was about 510 MiB greater than when updating the same file using SFTP (which streams to, and then from, a temporary file). |
| | 3 | The web-API interface does not support streaming (#113, #320), so it is expected for the gateway to need to hold the whole file in memory in order to upload it. However, when using {{{tahoe put}}} to upload an MDMF file, the increase in memory usage of the gateway process seems to be more than double the file size. For example, when uploading a 191 MiB MDMF file in 1.9alpha using {{{tahoe put --mutable --mutable-type=mdmf}}}, the peak RSS of the gateway (which was also a storage server in this test) was over 1300 MiB. There is also a huge memory leak of more than 700 MiB after the upload has finished. |
| | 4 | |
| | 5 | I originally thought that the memory usage was larger when using the web-API than when updating the same file using SFTP, but apparently that was wrong (I may have been misled by at first doing the SFTP experiment without restarting the nodes). |
