Benchmarks for basic immutable Tahoe operations (upload and download)

[itamarst]

Motivation:

1. We don't want HTTP to be slower than Foolscap.
2. It's not clear how fast these operations are in general.

Thus: there should be a way to measure how long these operations take.

First pass will run over loopback, but latency can be added-on on Linux (e.g. ​https://bencane.com/2012/07/16/tc-adding-simulated-network-latency-to-your-linux-server/).

[itamarst]

For a small 2000 byte file with a single share uploaded to a single node, an upload is 3 sequential HTTP requests, and a download is 4 sequential HTTP requests. So with high latency this gets quite slow.

Next, I tested over loopback, but with RTT latency of 60ms (30ms one way) artificially added. Bandwidth is quite high, so not likely to be a bottleneck.

• When uploading the 2000 byte file, upload time was 300ms, and download was 300ms.
• When uploading a 2MB file, upload time was 300ms, and download time was 3200ms.

Thus:

• Immutable uploads use parallelism, and main bottleneck is latency, which only impacts small files. However, sequential uploads will have minimal time no matter what the bandwidth.
• Immutable downloads have a latency hit, but even worse are not utilizing parallelism well, so large files suffer from latency hit even when bandwidth is not a bottleneck.

[itamarst]

Changing default segment size to be bigger helps a lot (mostly a workaround).

[itamarst]

The problems with increasing default segment size:

• Some code actually relies on it (e.g. ​https://github.com/tahoe-lafs/tahoe-lafs/blob/master/src/allmydata/immutable/downloader/node.py#L74) though _hopefully_ that's just an optimization. Would need testing of all relevant places that rely on it, at least.
• It doesn't help with existing downloads.