[tahoe-lafs-trac-stream] [tahoe-lafs] #1264: Performance regression for large values of K

[tahoe-lafs]

#1264: Performance regression for large values of K
-----------------------------+---------------------------------------------
     Reporter:  francois     |      Owner:  francois
         Type:  defect       |     Status:  new
     Priority:  major        |  Milestone:  soon
    Component:  code-        |    Version:  1.8.0
  network                    |   Keywords:  performance regression download
   Resolution:               |
Launchpad Bug:               |
-----------------------------+---------------------------------------------

Comment (by warner):

 That last image shows a comparison of LAN download times for a 100MB file
 at various 'k' values (the graphs for 10MB and 1MB files have the same
 shape, but more variance). The faster blue samples are for 1.7.1, while
 the slower red samples are current trunk (probably around [5277]). Both
 graphs are quite linear, but trunk (with the new downloader) is
 significantly slower than the old downloader in 1.7.1 across the board,
 probably about 3x.

 Higher 'k' slows down the process too, more severely for the new
 downloader. For 1.7.1, k=60 is about 3x slower than k=15, and for trunk,
 the difference is about 3.8x .

 I'm still suspecting the foolscap/message-handling overhead. I'd also like
 to find a way to get a speed-limited WAN-like pipe (simulated, for
 consistency across runs) and compute similar graphs in an environment
 where the network is the bottleneck. I believe the new-downloader is
 faster than the old code in the WAN (that's why I wrote it in the first
 place), but it looks like something about the complexity or small-reads
 makes the LAN case slower.

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1264#comment:21>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage