[tahoe-dev] try out the new faster download code!

Fri Jan 29 23:45:52 PST 2010

Hey, that really worked!

I went to http://allmydata.org/trac/tahoe/wiki/TestGrid which has a
matrix of live web gateways running different versions of Tahoe-LAFS.
The most recent version is 1.5.0-r4207. I viewed the publicly
writeable test directory with 1.5.0-r4207 and with the v1.5.0 release,
and selected a 4.5 MB image file with each one:

v1.5.0-r4207:
http://testgrid.allmydata.org:4207/file/URI%3ACHK%3A3twjjckj4kbcmgjnnqeriecgie%3Apssjtbkutggxj62ezsygnfafj7neawthzpfg53ivwxlhadjlfogq%3A3%3A10%3A4509606/@@named=/494.jpg

v1.5.0:
http://testgrid.allmydata.org:3150/file/URI%3ACHK%3A3twjjckj4kbcmgjnnqeriecgie%3Apssjtbkutggxj62ezsygnfafj7neawthzpfg53ivwxlhadjlfogq%3A3%3A10%3A4509606/@@named=/494.jpg

The web gateway running -r4207 served the image noticeably faster.
Then I went to the Welcome Page, clicked on "Recent Uploads and
Downloads", and clicked on the most recently download from each
gateway. The results are shown below. The new version took 9.69s to
server the file, the old version took 27s. You can see that the old
version used a server which had 520 ms response times and the new
version used only servers that had < 20 ms response times.

I don't know why that "Paused by client" field is so large -- more
than 7s. The client I used was Safari on Mac OS X.  Oh yeah if I rerun
the test with wget instead of Mac OS X then the "Paused by client"
field is 73 ms and the test results are correspondingly more dramatic:
it took 2.2s with the new version (wget reporting a throughput of 2.05
MB/s) and 20.9s with the old version (wget reporting a throughput of
232 KB/s).

I wonder why the download was "Paused by client" so much when using Safari...

Of course, maybe the new version got lucky this time around. We need
more tests! Let me know what you find. :-)

With v1.5.0-r4207 and Safari:

Timings:
File Size: 4509606 bytes
Total: 9.69s (465.5kBps)
Peer Selection: 34ms
UEB Fetch: 22ms
Hashtree Fetch: 19ms
Segment Fetch: 9.61s (469.3kBps)
Cumulative Fetching: 1.81s (2.48MBps)
Cumulative Decoding: 10ms (446.90MBps)
Cumulative Decrypting: 248ms (18.17MBps)
Paused by client: 7.55s
Per-Server Segment Fetch Response Times:
[fcmlx6em]: 27ms, 21ms, 17ms, 16ms, 18ms, 16ms, 994ms, 22ms, 20ms,
17ms, 22ms, 19ms, 19ms, 19ms, 20ms, 19ms, 21ms, 18ms, 17ms, 20ms,
16ms, 18ms, 21ms, 19ms, 19ms, 21ms, 17ms, 19ms, 22ms, 18ms, 18ms,
22ms, 18ms, 19ms, 17ms
[ivjakubr]: 38ms, 16ms, 15ms, 14ms, 15ms, 13ms, 15ms, 20ms, 14ms,
14ms, 14ms, 14ms, 14ms, 14ms, 14ms, 14ms, 16ms, 13ms, 14ms, 15ms,
13ms, 16ms, 16ms, 14ms, 15ms, 20ms, 15ms, 16ms, 16ms, 15ms, 16ms,
16ms, 15ms, 16ms, 16ms
[xiktf6ok]: 52ms, 17ms, 18ms, 17ms, 19ms, 18ms, 10ms, 23ms, 17ms,
18ms, 16ms, 16ms, 16ms, 16ms, 17ms, 16ms, 17ms, 19ms, 17ms, 17ms,
35ms, 19ms, 18ms, 21ms, 19ms, 16ms, 18ms, 19ms, 18ms, 19ms, 19ms,
22ms, 20ms, 21ms, 16ms

With v1.5.0 and Safari:

Timings:
File Size: 4509606 bytes
Total: 27 seconds (164.5kBps)
Peer Selection: 1.12s
UEB Fetch: 29ms
Hashtree Fetch: 16ms
Segment Fetch: 26 seconds (171.8kBps)
Cumulative Fetching: 18 seconds (242.5kBps)
Cumulative Decoding: 2.7ms (1652.66MBps)
Cumulative Decrypting: 226ms (19.87MBps)
Paused by client: 7.43s
Per-Server Segment Fetch Response Times:
[fcmlx6em]: 26ms, 17ms, 12ms, 12ms, 12ms, 12ms, 12ms, 12ms, 583ms,
12ms, 25ms, 12ms, 14ms, 12ms, 12ms, 12ms, 12ms, 12ms, 12ms, 12ms,
12ms, 12ms, 597ms, 12ms, 12ms, 12ms, 12ms, 15ms, 12ms, 12ms, 12ms,
12ms, 12ms, 12ms, 10ms
[ivjakubr]: 39ms, 22ms, 15ms, 16ms, 15ms, 19ms, 16ms, 15ms, 16ms,
15ms, 14ms, 16ms, 18ms, 16ms, 16ms, 15ms, 16ms, 15ms, 15ms, 15ms,
15ms, 16ms, 14ms, 15ms, 16ms, 15ms, 15ms, 19ms, 16ms, 15ms, 16ms,
15ms, 15ms, 15ms, 10ms
[2gn6njsm]: 753ms, 512ms, 518ms, 524ms, 512ms, 512ms, 546ms, 537ms,
548ms, 525ms, 527ms, 520ms, 528ms, 523ms, 521ms, 532ms, 517ms, 524ms,
511ms, 519ms, 522ms, 524ms, 529ms, 512ms, 513ms, 521ms, 521ms, 520ms,
531ms, 523ms, 518ms, 522ms, 523ms, 521ms, 365ms

With v1.5.0-r4207 and wget:

Timings:
File Size: 4509606 bytes
Total: 2.20s (2.05MBps)
Peer Selection: 29ms
UEB Fetch: 17ms
Hashtree Fetch: 15ms
Segment Fetch: 2.14s (2.11MBps)
Cumulative Fetching: 1.84s (2.45MBps)
Cumulative Decoding: 10ms (446.15MBps)
Cumulative Decrypting: 220ms (20.42MBps)
Paused by client: 74ms
Per-Server Segment Fetch Response Times:
[fcmlx6em]: 24ms, 12ms, 11ms, 11ms, 12ms, 11ms, 12ms, 12ms, 11ms,
12ms, 12ms, 11ms, 12ms, 12ms, 12ms, 12ms, 12ms, 12ms, 11ms, 13ms,
13ms, 16ms, 12ms, 12ms, 11ms, 13ms, 12ms, 12ms, 13ms, 1.03s, 12ms,
12ms, 11ms, 12ms, 8.4ms
[ivjakubr]: 35ms, 20ms, 23ms, 21ms, 19ms, 23ms, 21ms, 20ms, 23ms,
21ms, 20ms, 21ms, 22ms, 20ms, 21ms, 20ms, 20ms, 23ms, 22ms, 20ms,
23ms, 25ms, 21ms, 21ms, 21ms, 22ms, 21ms, 24ms, 21ms, 20ms, 23ms,
23ms, 21ms, 21ms, 14ms
[7tlov7eg]: 39ms, 19ms, 18ms, 20ms, 18ms, 18ms, 21ms, 19ms, 18ms,
21ms, 19ms, 21ms, 21ms, 19ms, 16ms, 20ms, 20ms, 18ms, 21ms, 20ms,
23ms, 24ms, 20ms, 17ms, 20ms, 17ms, 17ms, 18ms, 20ms, 16ms, 19ms,
19ms, 16ms, 17ms, 11ms

With v1.5.0 and wget:

Timings:
File Size: 4509606 bytes
Total: 20 seconds (215.7kBps)
Peer Selection: 1.11s
UEB Fetch: 12ms
Hashtree Fetch: 12ms
Segment Fetch: 19 seconds (228.1kBps)
Cumulative Fetching: 19 seconds (231.6kBps)
Cumulative Decoding: 2.8ms (1596.71MBps)
Cumulative Decrypting: 237ms (19.02MBps)
Paused by client: 72ms
Per-Server Segment Fetch Response Times:
[fcmlx6em]: 723ms, 15ms, 12ms, 14ms, 12ms, 12ms, 12ms, 12ms, 12ms,
15ms, 12ms, 12ms, 12ms, 13ms, 11ms, 12ms, 14ms, 16ms, 871ms, 14ms,
12ms, 12ms, 12ms, 12ms, 12ms, 12ms, 12ms, 12ms, 12ms, 12ms, 12ms,
15ms, 12ms, 12ms, 57ms
[ivjakubr]: 28ms, 14ms, 15ms, 13ms, 15ms, 15ms, 15ms, 15ms, 16ms,
14ms, 15ms, 15ms, 16ms, 16ms, 15ms, 15ms, 13ms, 20ms, 13ms, 20ms,
15ms, 15ms, 15ms, 15ms, 15ms, 16ms, 15ms, 16ms, 15ms, 15ms, 16ms,
27ms, 15ms, 15ms, 13ms
[2gn6njsm]: 827ms, 518ms, 520ms, 517ms, 520ms, 522ms, 534ms, 517ms,
525ms, 516ms, 552ms, 549ms, 525ms, 522ms, 520ms, 516ms, 520ms, 576ms,
536ms, 693ms, 524ms, 526ms, 529ms, 533ms, 535ms, 524ms, 530ms, 536ms,
523ms, 522ms, 519ms, 680ms, 616ms, 557ms, 328ms