[tahoe-lafs-trac-stream] [tahoe-lafs] #869: Allow Tahoe filesystem to be run over a different key-value-store / DHT implementation

[tahoe-lafs]

#869: Allow Tahoe filesystem to be run over a different key-value-store / DHT
implementation
-------------------------+-------------------------------------------------
     Reporter:           |      Owner:  nobody
  davidsarah             |     Status:  new
         Type:           |  Milestone:  undecided
  enhancement            |    Version:  1.5.0
     Priority:  major    |   Keywords:  scalability performance  forward-
    Component:  unknown  |  compatibility backward-compatibility
   Resolution:           |  availability newcaps docs anti-censorship
Launchpad Bug:           |
-------------------------+-------------------------------------------------
Description changed by daira:

Old description:

> source:docs/architecture.txt describes Tahoe as comprising three layers:
> '''key-value store''', '''filesystem''', and '''application'''.
>
> Most of what makes Tahoe different from other systems is in the
> filesystem layer -- the layer that implements a cryptographic capability
> filesystem. The key-value store layer implements (a little bit more than)
> a Distributed Hash Table, which is a fairly well-understood primitive
> with many implementations. The Tahoe filesystem and applications could in
> principle run on a different DHT, and it would still behave like Tahoe --
> with different (perhaps better, depending on the DHT) scalability,
> performance, and availability properties, but with confidentiality and
> integrity ensured by Tahoe without relying on the DHT severs.
>
> However, there are some obstacles to running the Tahoe filesystem layer
> on another DHT:
>  * the code isn't strictly factored into layers (even though most code
> files belong mainly to one layer), so there isn't a narrow API between
> the key-value store and filesystem-related abstractions.
>  * the communication with servers currently needs to be encrypted
> (independently of the share encryption), and other DHTs probably wouldn't
> support that.
>  * because the filesystem has only been used with one key-value store
> layer up to now, it may make assumptions about that layer that haven't
> been clearly documented.
>
> Note that even if the Tahoe code was strictly layered, we should still
> expect there to be some significant effort to port Tahoe to a particular
> DHT. The DHT servers would probably have to run some Tahoe code in order
> to verify shares, for example.

New description:

 source:docs/architecture.txt describes Tahoe as comprising three layers:
 '''key-value store''', '''filesystem''', and '''application'''.

 Most of what makes Tahoe different from other systems is in the filesystem
 layer -- the layer that implements a cryptographic capability filesystem.
 The key-value store layer implements (a little bit more than) a
 Distributed Hash Table, which is a fairly well-understood primitive with
 many implementations. The Tahoe filesystem and applications could in
 principle run on a different DHT, and it would still behave like Tahoe --
 with different (perhaps better, depending on the DHT) scalability,
 performance, and availability properties, but with confidentiality and
 integrity ensured by Tahoe without relying on the DHT servers.

 However, there are some obstacles to running the Tahoe filesystem layer on
 another DHT:
  * The code isn't strictly factored into layers (even though most code
 files belong mainly to one layer), so there isn't a narrow API between the
 key-value store and filesystem-related abstractions.
  * The communication with servers currently needs to be encrypted
 (independently of the share encryption), and other DHTs probably wouldn't
 support that.
  * Because the filesystem has only been used with one key-value store
 layer up to now, it may make assumptions about that layer that haven't
 been clearly documented.

 Note that even if the Tahoe code was strictly layered, we should still
 expect there to be some significant effort to port Tahoe to a particular
 DHT. The DHT servers would probably have to run some Tahoe code in order
 to verify shares, for example.

--

-- 
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/869#comment:9>
tahoe-lafs <https://tahoe-lafs.org>
secure decentralized storage