[tahoe-dev] Perf-related architecture question

[Kyle Markley]

Zooko,

>> I am running a helper,
> 
> Why?

My grid has 1 desktop and 3 laptops.  The helper is the desktop so that if
any of the laptops happen to be plugged into the network -- and a
disconnected laptop is the uploader -- the storage nodes will receive
shares over the wire instead of over the air.  This conserves the
relatively scarcer wireless bandwidth, because the uploader only needs to
send the ciphertext to the helper wirelessly, versus all the shares. 
(Other benefits: offloading a little work to the desktop marginally
improves battery life on the uploading laptop, the desktop is faster, and
is underutilized anyway.)

All that said, I do appreciate your reasons for not liking the helper.  :)

> Aha! The uploader (whether used by the storage client in your client
> or the storage client in the erasure-coding helper) is uploading to
> each of the storage servers in parallel. Its rate will therefore be
> the rate of the slowest storage servers times K.
> 
> To let the fastest storage servers run ahead and finish uploading
> would either require storing all of the file data in RAM or revisiting
> the file (as stored on disk) more than once.

Understood.  I was able to see this happening and wanted to understand it,
but there's little practical reason to dig more deeply into this.  The
first-order issue for me is that neither my CPUs nor my network are seeing
high utilization.  But it's still good to understand what's going on under
the hood.

-- 
Kyle Markley