[tahoe-lafs-trac-stream] [tahoe-lafs] #398: allow users to disable use of helper: direct uploads might be faster

[tahoe-lafs]

#398: allow users to disable use of helper: direct uploads might be faster
-------------------------+-------------------------------------------------
     Reporter:  warner   |      Owner:
         Type:  task     |     Status:  closed
     Priority:  major    |  Milestone:  undecided
    Component:  code-    |    Version:  1.0.0
  encoding               |   Keywords:  helper performance upload bandwidth
   Resolution:  invalid  |
Launchpad Bug:           |
-------------------------+-------------------------------------------------
Changes (by daira):

 * status:  new => closed
 * resolution:   => invalid


Old description:

> It occurred to me and Zandr that clients with fast uplinks may achieve
> faster
> throughput by not using the helper at all. The protocol improvements in
> #397
> are nice and all, but really it's the helper that's getting overloaded.
> The
> purpose of the helper was to allow folks with slow uplinks to avoid
> paying
> the 3x expansion penalty on the wrong end of the wire, but folks with
> multi-megabit uplinks don't really need this help. At those speeds,
> sending
> 3x shares directly to the storage servers will get faster uploads than
> sending 1x ciphertext to an overloaded helper.
>
> We might manage to farm out the helpers enough to improve performance to
> make
> this not true, but fundamentally the goal of a distributed system is to
> reduce centralization and push the work out to the edge. The helper is a
> concession to improving user experience with slow uplinks, but on fast
> uplinks it should probably not be used.
>
> So the task is to give advanced end users of the commercial service a way
> to
> not use the helper. Simply deleting or renaming BASEDIR/helper.furl and
> restarting the node would suffice, but it might be nice to have a better
> UI
> for this. The goal is to let users with fast uplinks try it both ways and
> see
> which gives them better service.
>
> Eventually, we might consider having the node automatically decide
> whether to
> use the helper or not, based upon the upload speeds measured by the first
> few
> direct uploads.

New description:

 It occurred to me and Zandr that clients with fast uplinks may achieve
 faster
 throughput by not using the helper at all. The protocol improvements in
 #397
 are nice and all, but really it's the helper that's getting overloaded.
 The
 purpose of the helper was to allow folks with slow uplinks to avoid paying
 the 3x expansion penalty on the wrong end of the wire, but folks with
 multi-megabit uplinks don't really need this help. At those speeds,
 sending
 3x shares directly to the storage servers will get faster uploads than
 sending 1x ciphertext to an overloaded helper.

 We might manage to farm out the helpers enough to improve performance to
 make
 this not true, but fundamentally the goal of a distributed system is to
 reduce centralization and push the work out to the edge. The helper is a
 concession to improving user experience with slow uplinks, but on fast
 uplinks it should probably not be used.

 So the task is to give advanced end users of the commercial service a way
 to
 not use the helper. Simply deleting or renaming BASEDIR/helper.furl and
 restarting the node would suffice, but it might be nice to have a better
 UI
 for this. The goal is to let users with fast uplinks try it both ways and
 see
 which gives them better service.

 Eventually, we might consider having the node automatically decide whether
 to
 use the helper or not, based upon the upload speeds measured by the first
 few
 direct uploads.

--

Comment:

 This related to Allmydata's service, which is no longer relevant. If
 anyone thinks it is really important to dynamically choose whether to use
 a helper (I'm skeptical), then please file another ticket.

-- 
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/398#comment:8>
tahoe-lafs <https://tahoe-lafs.org>
secure decentralized storage