devchat notes 20-March-2018

[Jean-Paul Calderone]

Tahoe-LAFS devchat 20-March-2018

A few topics arose but I focused my note-taking on the topic of porting
Tahoe-LAFS to Python 3.  The focus of the discussion was how to deal with
the Foolscap dependency.  One option is to port Foolscap to Python 3.
However, another option is to port Tahoe-LAFS to a non-Foolscap protocol
which already has a Python 3 implementation.  This is the option we
discussed.

The concerns dictating a replacement for Foolscap are:

   - a Python 3 implementation
   - *other* language implementations to encourage more alternate-language
   implementations
   - Marketing considerations ("look we use this industry standard
   protocol")
   - Firewall penetration capabilities and other network-related concerns
   - Bidirectionality (though Tahoe-LAFS doesn't really use this now it
   would be beneficial for a number of future enhancements such as improved
   magic-folder change notification)

HTTP is an obvious choice based on these concerns.  Combined with
WebSockets it should satisfy every bullet point.  Brian has a branch where
he started some HTTP work.

What would an HTTP protocol look like?  Should it be ReSTful?  Should the
HTTP be purely a transport for some more structured data represented with a
protocol like BSON, etc?  How should authentication and integrity be
handled - should requests be signed?  should requests be accepted if they
result in valid share state (the write-enabler makes this trickier for
mutable files - should the write enabler be removed?  or is there a way to
preserve it with a new protocol?  it's derived from the storage server's
TLS certificate - can the new HTTP(S) protocol re-use this certificate?)

What is the immutable file protocol?

   - Read (offset, length)
   - Advise corrupt share
   - Allocate Buckets (storage-index, secrets, share-numbers, share-size)
   returning set[existing-share-indices], dict[bucket-writers]
   - BucketWriter - write (offset, data), close, abort


How does that map to HTTP?

Accounting rears its head (or maybe it's scope creep wearing an accounting
mask).  How can clients be limited in their storage usage?  Uploads could
be signed by a key authorizing some consumption (and therefore creating
accountability for storage usage for the key-holder).  Variations:

   - Allocate Buckets could be signed
   - Close could be signed

Different approaches have different consequences for upload complexity (eg
in terms of number of passes over the data).

And download authorization?  Present system allows anyone with access
unlimited download bandwidth.  Accounting should deal with this as well.

What about the other places Tahoe-LAFS uses Foolscap?  There's
client<->introducer, client<->storageserver, and
introducer<->storageserver.  We don't have to port all of these at once.

Action items:

   - Complete HTTP-based spec for the immutable file interface.  Evaluate
   for feasibility, feature-equivalence to current API (including security,
   performance, extensibility).
   - Repeat for mutable files, pubsub (considering potential future magic
   folder features).
   - Brian will provide a reference to the already-started HTTP efforts.


Also some stuff about setuptools wrt some old, hopefully obsolete cruft in
Tahoe-LAFS setup.py et al and specifically relating to some problems on CI
with very old setuptools.  Details too complicated to accurately record
during the conversation.  Brian went ahead and did Some Stuff[tm] to try to
address this shortly after the devchat.

Jean-Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20180326/d1c52792/attachment.html>