[tahoe-dev] [tahoe-lafs] #1154: mplayer triggers two bugs in Tahoe's new downloader

[Zooko O'Whielacronx]

On Thu, Aug 5, 2010 at 12:15 PM, Brian wrote (on a trac ticket):
>
>  Oh, I should mention what I'm guessing mplayer was doing. I think it
>  issued two simultaneous parallel webapi GET requests: one for the index
>  near the end of the file, and another for the first block of the file.
>  They both would have been delivered to the same {{{FileNode}}} instance,
>  creating two {{{Segmentation}}} requests (one for each {{{read()}}} call),
>  creating two calls to {{{DownloadNode.get_segment()}}}. The second one
>  would wait for the first one to finish, since to keep the memory footprint
>  low, {{{DownloadNode}}} only works on one segment at a time.

This is a good example of how pipelining download of blocks (#1110)
could help. Previously I thought of it as a performance improvement
when downloading successive blocks of the same file. Therefore I
figured that if you were doing streaming processing of the file, such
as if it was a movie and you were playing it out at normal speed, then
a sufficiently large segment size would make the download faster than
your playout speed, so pipelining would not matter for that. But this
example shows that for some cases the segment size is irrelevant—in
this case (if Brian's guess is correct) a read-block-pipeline depth of
>= 2 would take one round-trip off of the startup time.

Regards,

Zooko

http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1110# pipeline download
blocks for better performance