#3957 new enhancement

Make HTTP no slower than Foolscap for mutable uploads and downloads

Reported by: itamarst Owned by:
Priority: normal Milestone: HTTP Storage Protocol v2
Component: unknown Version: n/a
Keywords: Cc:
Launchpad Bug:

Description

It seems like the fix for #3954 mostly helps with this, but I am having hard time testing larger files due to #3956, so need to wait until that's fixed to do more testing.

Change History (4)

comment:1 Changed at 2023-01-06T16:18:43Z by itamarst

For a tens-of-MB mutable SDMF upload, with no network latency, HTTP upload is about twice as slow as Foolscap. Downloads are slower, but not as much. The issue seems to mostly be CPU.

With network latency of 60ms, the difference between HTTP and Foolscap becomes less marked. The latency penalty is not terrible.

Conclusion: As a first pass, for SDMF figure out why HTTP is using more CPU.

comment:2 Changed at 2023-01-20T16:44:16Z by itamarst

Looking at profiling results. 10-15% of CPU time is at least CDDL validation and CBOR decoding. And there's some HTTP parsing. So that's probably a big part of it.

comment:3 Changed at 2023-01-23T13:37:59Z by itamarst

My first attempt to speed up CDDL validation was a failure. As a short-term solution (and probably a good idea anyway) I'm going to move CDDL validation and CBOR parsing into a thread, so at least it doesn't block the event loop (or block it as much). CDDL validation is the bulk of the time at the moment (which is embarassing, but OK) and it will release the GIL. This won't improve latency, but it will help server parallelism: #3968

Last edited at 2023-01-23T16:38:36Z by itamarst (previous) (diff)

comment:4 Changed at 2023-01-23T16:44:54Z by itamarst

  • Milestone changed from HTTP Storage Protocol to HTTP Storage Protocol v2
Note: See TracTickets for help on using tickets.