[tahoe-dev] Benchmarks - 1.7.1 vs. 1.8.0c3

[Sam Mason]

On Tue, Sep 07, 2010 at 10:14:42AM -0400, Kyle Markley wrote:
> The variance is large but the general conclusion
> from earlier appears correct -- on my low-latency benchmarking grid, 1.7.1
> is better for downloading large files and 1.8.0c3 is better for downloading
> small files.  This result seems fairly consistent, but it's also not large
> relative to the run-to-run variance.

Doing a t-test on the numbers agrees with you; there's no statistically
significant difference between the upload speeds, while within the
downloads, the large download is significant yet the difference in small
download speeds isn't quite statistically significant.  After removing
the outliers, both download speed differences become significant, while
the upload speeds remain insignificant.

Speeds are basically 8% slower and 12% faster, in the large and small
downloads respectively, when doing from 1.7.1 to 1.8.0c3.

-- 
  Sam  http://samason.me.uk/