#3952 closed enhancement (fixed)

Benchmarks for basic immutable Tahoe operations (upload and download)

Reported by: itamarst Owned by:
Priority: normal Milestone: HTTP Storage Protocol
Component: unknown Version: n/a
Keywords: Cc:
Launchpad Bug:



  1. We don't want HTTP to be slower than Foolscap.
  2. It's not clear how fast these operations are in general.

Thus: there should be a way to measure how long these operations take.

First pass will run over loopback, but latency can be added-on on Linux (e.g. https://bencane.com/2012/07/16/tc-adding-simulated-network-latency-to-your-linux-server/).

Change History (6)

comment:1 Changed at 2022-12-12T16:26:00Z by itamarst

  • Summary changed from Benchmarks for basic immutable Tahoe operations to Benchmarks for basic immutable Tahoe operations (upload and download)

comment:2 Changed at 2022-12-12T20:01:04Z by itamarst

For a small 2000 byte file with a single share uploaded to a single node, an upload is 3 sequential HTTP requests, and a download is 4 sequential HTTP requests. So with high latency this gets quite slow.

Next, I tested HTTP over loopback, but with RTT latency of 60ms (30ms one way) artificially added. Bandwidth is quite high, so not likely to be a bottleneck.

File size Upload clock time Upload CPU time Download time Download CPU time
2KB 300ms 30ms 300ms 65ms
2MB 700ms 70ms 3,200ms 680ms
20MB 3,500ms 600ms 31,000ms 6,200ms

This is all sharing a single CPU core for both client and server, it's one big process.

So at first glance:

  • Immutable uploads use parallelism, and main bottleneck is latency. Upload time is not linear with size, . However, sequential uploads will have minimal time no matter what the bandwidth.
  • Immutable downloads have a massive latency hit, probably due to not utilizing parallelism well, so large files suffer from latency hit even when bandwidth is not a bottleneck. Also using a lot of CPU.

I then tested Foolscap with 20MB file. Upload 3,200ms clock time, 500ms CPU time, so HTTP upload is worse but not too bad in comparison. Foolscap download was 20,800ms clock time and 2,100 CPU time. So HTTP download is much worse than Foolscap, but Foolscap downloads are still quite bad.

Last edited at 2022-12-13T16:36:34Z by itamarst (previous) (diff)

comment:3 Changed at 2022-12-13T18:19:57Z by itamarst

Last edited at 2022-12-14T15:48:51Z by itamarst (previous) (diff)

comment:4 Changed at 2022-12-13T18:42:49Z by itamarst

Changing default segment size to be bigger helps a lot (mostly a workaround).

comment:5 Changed at 2023-01-23T16:51:42Z by itamarst

The problems with increasing default segment size:

comment:6 Changed at 2023-02-23T15:30:54Z by itamarst

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.