[tahoe-dev] Benchmarks - 1.7.1 vs. 1.8.0c3

[Brian Warner]

On 9/7/10 6:08 AM, Kyle Markley wrote:
> Zooko,
>  
>> Oh say it ain't so! Upgrading to 1.8.0c3 from 1.7.1 will make your
>> large file downloads go about half as fast as they used to?
> 
> Don't despair -- my result might be true only for low-latency grids!  We
> already have lots of data showing that 1.8.0c3 is an improvement for large
> files on a real-world grid.

Please eyeball a couple of the download-status pages and see if you can
learn anything about the number of servers being selected. I imagine
that on your local grid, all the servers are identical, but since the
performance variations we've seen on the testgrid/v-grid have been
closely correlated to server-selection, it'd be good to find a way to
rule that out here.

And yeah, for low-latency (or even single-CPU all-local) grids, there
may be more CPU overhead to the new downloader, causing a performance
reduction when the network effectively has infinite bandwidth and is
able to keep up.

What I'd love to have is a test framework in which we could simulate
multiple servers (of varying CPU speeds) connected by virtual pipes with
controllable latency and bandwidth limits. Then we could run an
experiment like measuring download speed as the bandwidth limits are
raised, where I'd expect to see it taper off as we approach the
CPU-speed limit. (I remember writing this desire up on a whiteboard at
allmydata about four years ago and labelling it "and also, I want a
pony"). A lot of the performance measurements we've been making are at
unlabled points along some axes of CPU-speed, network-speed, and share
distribution.. being able to label those points and compare apples to
apples would probably help us understand what's going on a lot better.

cheers,
 -Brian