Changes between Version 5 and Version 152 of Ticket #999


Ignore:
Timestamp:
2019-09-08T22:55:09Z (5 years ago)
Author:
amontero
Comment:

Legend:

Unmodified
Added
Removed
Modified

  • Ticket #999

    • Property Status changed from new to closed
    • Property Cc wilcoxjg@… mk.fraggod@… amontero@… added
    • Property Owner set to davidsarah
    • Property Version changed from 1.6.0 to n/a
    • Property Milestone changed from undecided to eventually
    • Property Keywords s3-backend storage added; gsoc backend s3 removed
    • Property Resolution changed from to fixed

  • Ticket #999 – Description

    v5 v152  
    11The focus of this ticket is (now) adapting the existing codebase to use multiple backends, rather than supporting any particular backend.
    2 
    32We already have one backend -- the filesystem backend -- which I think should be a plugin in the same sense that the others will be plugins (i.e.: other code in tahoe-lafs can interact with a filesystem plugin without caring very much about how or where it is storing its files -- otherwise it doesn't seem very extensible). If you accept this, then we'd need to figure out what a backend plugin should look like.
    4 
    53There is backend-independent logic in the current server implementation that we wouldn't want to duplicate in every other backend implementation. To address this, we could start by refactoring the existing code that reads or writes shares on disk, to use a local backend implementation supporting an IStorageProvider interface (probably a fairly simplistic filesystem-ish API).
    6 
    74(This involves changing the code in [source:src/allmydata/storage/server.py] that reads from local disk in its [source:src/allmydata/storage/server.py@4164#L359 _iter_share_files()] method, and also changing [source:src/allmydata/storage/shares.py@3762 storage/shares.py], [source:src/allmydata/storage/immutable.py@3871#L39 storage/immutable.py], and [source:src/allmydata/storage/mutable.py@3815#L34 storage/mutable.py] that write shares to local disk.)
    8 
    95At this point all the existing tests should still pass, since we haven't actually changed the behaviour.
    10 
    116Then we have to add the ability to configure new storage providers. This involves figuring out how to map user configuration choices to what actually happens when a node is started, and how the credentials needed to log into a particular storage backend should be specified. The skeletal RIStorageServer would instantiate its IStorageProvider based on what the user configured, and use it to write/read data, get statistics, and so on.
    12 
    137Naturally, all of this would require a decent amount of documentation and testing, too.
    14 
    158Once we have all of this worked out, the rest of this project (probably to be handled in other tickets) would be identifying what other backends we'd want in tahoe-lafs, then documenting, implementing, and testing them. We already have Amazon S3 and Rackspace as targets -- users of tahoe-lafs will probably have their own suggestions, and more backends will come up with more research.