[tahoe-dev] [tahoe-lafs] #1264: Performance regression for large values of K

[tahoe-lafs]

#1264: Performance regression for large values of K
------------------------------+---------------------------------------------
     Reporter:  francois      |       Owner:                                
         Type:  defect        |      Status:  new                           
     Priority:  major         |   Milestone:  soon                          
    Component:  code-network  |     Version:  1.8.0                         
   Resolution:                |    Keywords:  performane regression download
Launchpad Bug:                |  
------------------------------+---------------------------------------------

Comment (by francois):

 Replying to [comment:4 warner]:

 > Works for me. I ought to get some analysis time this upcoming (US
 holiday) weekend. My plan is to point the JS-based visualizer at it, and
 measure the time spent recomputing the desire/satisfaction bitmaps/Spans,
 since I've observed a surprising amount of time spent in that portion of
 the code (it's being called much too frequently, and I think it gets worse
 with large {{{k}}}).

 Great! I also hope to find some free time this weekend to work on that.

 An quick analysis of [attachment:"Tahoe-LAFS - File Download Status.html"]
 shows that some segments requests were delayed because a single share
 request was taking too long. The average time for a share request during
 this download was about 100 ms, but it raised to a bit more than one
 second in a few cases such as this one.

 {{{
 qnsglseg        21      [6590:+6554]    +0.896405s      +1.985190s
 6554    1.09s
 }}}

 I think that those share requests taking too long are probably due to the
 overall load on the grid and are probably difficult to avoid. However, it
 seems that each segment request is currently serialized (#1187). That
 means that a single slow share request will prevent the next segment
 download to start which will slowdown the overall file download. The
 bigger {{{k}}} is, the more shares the client will have to download and
 therefore the probability of slow share requests will increase.

 Now, I don't really understand if and why this issue was not present in
 v1.7.1 because it seems that segement pipelining wasn't already
 implemented in the previous downloader.

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1264#comment:5>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage