[tahoe-dev] [tahoe-lafs] #1264: Performance regression for large values of K

tahoe-lafs trac at tahoe-lafs.org
Tue Nov 23 22:25:47 UTC 2010


#1264: Performance regression for large values of K
------------------------------+---------------------------------------------
     Reporter:  francois      |       Owner:                                
         Type:  defect        |      Status:  new                           
     Priority:  major         |   Milestone:  soon                          
    Component:  code-network  |     Version:  1.8.0                         
   Resolution:                |    Keywords:  performane regression download
Launchpad Bug:                |  
------------------------------+---------------------------------------------

Comment (by warner):

 zooko said:

 > I don't understand why a few of the share requests would take ten times
 as
 > long as normal. Is the delay on the client, the server, or the network?
 > Brian hypothesized that it had something to do with how the spans data
 > structure gets used more when K is higher.

 Actually my hypothesis is that the time spent between receipt of the block
 request and the transmission of the next block request will be higher for
 larger values of k. This time would not be "charged" against the
 individual
 server: it occurs after the clock has been stopped.

 [http://tahoe-lafs.org/trac/tahoe-lafs/attachment/ticket/1170/180c2-viz-
 delays.png This picture]
 (and the other pictures on #1170) show part of this: it's the delay
 between
 the receipt of the last block() request (e.g. at 2.60s) and the
 transmission
 of the next request (at around 2.67s).

 [http://tahoe-lafs.org/trac/tahoe-lafs/attachment/ticket/1170/viz-3.png
 Zooming in]
 shows what's happening in that gap. If you could read the "desire" words
 on
 the small boxes (which of course is easier when you can interactively zoom
 the chart, rather than looking at a screenshot), you could see that
 recomputing the desire/satified bitmaps is being done three times, but the
 second and third passes are redundant (no new responses have arrived, so
 the
 computed bitmap is exactly the same, so that's just wasted time).

 If I remember right, the downloader was doing one redundant pass for each
 hash received, and we fetch more hashes for some segments than others
 (peaking for NUMSEGS/2), resulting in an uneven distribution of extra
 delays.
 My comments in http://tahoe-lafs.org/trac/tahoe-
 lafs/ticket/1170#comment:92
 suggest that it's not enough to explain the delays seen here (8% on a 12MB
 file on a one-CPU testnet), but it *would* be worse with more segments.

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1264#comment:8>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage


More information about the tahoe-dev mailing list