[tahoe-lafs-trac-stream] [tahoe-lafs] #1456: High latency for 'tahoe get' if 'tahoe put' in parallel

[tahoe-lafs]

#1456: High latency for 'tahoe get' if 'tahoe put' in parallel
-------------------------+-------------------------------------------------
     Reporter:  T_X      |      Owner:  T_X
         Type:  defect   |     Status:  new
     Priority:           |  Milestone:  undecided
  critical               |    Version:  1.8.2
    Component:  code     |   Keywords:  download upload latency performance
   Resolution:           |  gateway vm kvm vpn trickle
Launchpad Bug:           |
-------------------------+-------------------------------------------------

Comment (by warner):

 Are the GET and the PUT referencing the same tahoe-side directory? I
 wonder if the basic locking we do around the directory's mutable-file
 objects (to prevent e.g. two simultaneous writes from the same gateway)
 is causing a PUT to block a GET.

 There are a couple of convenience methods on dirnodes: things like "add
 a new child", or "get a child by a specific name". The add-child method
 is really a wrapper around two operations: upload-file, then
 attach-child, and only the latter really needs the lock.

 My hypothesis is that PUT is using a convenience method which locks the
 dirnode during the upload-file phase (which takes a long time), and then
 the GET tries to do a list-children operation (to find out the immutable
 file that should be downloaded), which blocks until the add-child method
 has finished.

 If we observed the CHK download to not start until after the PUT's
 directory-modification completed (using the "recent uploads and
 downloads" page, or maybe {{{flogtool tail}}}), that'd be supporting
 evidence.

 Some code notes:

  * src/allmydata/web/filenode.py
    ({{{ReplaceMeMixin.replace_me_with_a_child}}}) calls
    {{{self.parentnode.add_file}}}
  * src/allmydata/dirnode.py ({{{DirectoryNode.add_file}}}) calls
    {{{upload()}}}, then {{{self._create_and_validate_node}}}, then
    {{{self.set_node}}}

 Hrm, maybe not: the locks are all in the mutable-file object, not the
 dirnode, so the lock shouldn't be claimed until after the upload is
 completely finished. Anyways, maybe there's some other bug in this
 direction.. worth considering.

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1456#comment:9>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage