[tahoe-lafs-trac-stream] [Tahoe-LAFS] #395: why are so many helper files being abandoned?

[Tahoe-LAFS]

#395: why are so many helper files being abandoned?
-----------------------------+--------------------------------------------
     Reporter:  warner       |      Owner:  somebody
         Type:  task         |     Status:  new
     Priority:  major        |  Milestone:  eventually
    Component:  operational  |    Version:  1.0.0
   Resolution:               |   Keywords:  upload-helper space-efficiency
Launchpad Bug:               |
-----------------------------+--------------------------------------------
Changes (by zooko):

 * keywords:  helper space-efficiency => upload-helper space-efficiency


Old description:

> Our prodnet munin graphs show that our helper is receiving ciphertext for
> a lot of files which are then abandoned, either part-way through the
> ciphertext fetch phase, or during the encode+push phase. The clients then
> never try again.
>
>  * add code (or admin tools) to delete old files from the helper's
>    directory after a while. We have something like 38GB of files in
>    there right now
>  * figure out where these are coming from. It must be the windows
>    FUSE plugin (that's the only deployment we have on prodnet that
>    uses the helper), but what sorts of files are so large and then
>    abandoned? It's possible that encode+push takes so long that
>    the application gives up and tries again (with a new and
>    different file).

New description:

 Our prodnet munin graphs show that our helper is receiving ciphertext for
 a lot of files which are then abandoned, either part-way through the
 ciphertext fetch phase, or during the encode+push phase. The clients then
 never try again.

  * add code (or admin tools) to delete old files from the helper's
    directory after a while. We have something like 38GB of files in
    there right now
  * figure out where these are coming from. It must be the windows
    FUSE plugin (that's the only deployment we have on prodnet that
    uses the helper), but what sorts of files are so large and then
    abandoned? It's possible that encode+push takes so long that
    the application gives up and tries again (with a new and
    different file).

--

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/395#comment:4>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage