[tahoe-dev] Getting my root writecap for the production grid

[Ian Levesque]

There's no "important" data just yet, but I will be keeping a close  
eye on Tahoe development.  This client is eventually destined for "the  
masses" so being a good AllMyData citizen is important.  I would  
definitely like to participate with a private key and play nice with  
the garbage collection.  Here are two points that would be important  
for my client going forward:

1. Hopefully the client can mark all "important" files simply by  
traversing down from the root URI.  I don't really want to maintain a  
DB of all published files simply to keep them from being removed!  
Especially since at least one device my client presently runs on has  
no local persistent storage! Better yet, why not garbage collect any  
files that aren't linked to a name inside any directory linked from  
the root (perhaps through more directories)?

2. Let me also put in a vote for keeping at least some sort of  
affordable "unlimited" account.  It's one thing that makes AllMyData  
attractive vs. Amazon S3.  Perhaps you should keep an unlimited option  
with some lower service guarantee (perhaps less redundancy or a slower  
transfer rate)? Unlimited = good for large, relatively unchanging  
backups.

Exciting stuff, all of it.

3. Also, if anyone is interested, I would be willing to open source  
(at least) the lowest level Tahoe client portion of my project.  That  
portion is a very simple Ruby library for performing basic Tahoe  
operations (put, get, rename, delete, mkdir, list, attributes).  
Similar to the AWS::S3 (amazon.rubyforge.org) library.

Thanks,
-Ian

On Jul 30, 2008, at 6:58 PM, Brian Warner wrote:

>> Brian will probably comment on some of the issues associated with  
>> this and
>> what will change going forward when we implement accounting.
>
> If you're building a custom client to work against the allmydata grid,
> (specifically if you're uploading important data to the allmydata grid
> through non-allmydata clients), keep a careful eye on new Tahoe  
> developments.
> In particular:
>
> * when we implement and turn on Accounting, all clients will need a
>   per-account private key to connect to the grid, and your client  
> will need
>   to do the same
> * when we implement garbage collection, clients will need to mark  
> their
>   files as being not-garbage (possibly using a timer-based expiration
>   mechanism, but we're still deciding on the details). Your client  
> will need
>   to participate in this protocol to prevent your files from being
>   identified as deleted/expired garbage and removed.
>
> When we turn on these features on the production grid, we'll make an
> announcement.. if that announcement doesn't give you enough  
> information, feel
> free to write back with questions.
>
> Basically, allmydata customers using allmydata clients will get  
> taken care of
> by our usual upgrade and/or automatic maintenance code. Files added  
> outside
> this domain just need some extra attention.
>
> Incidentally, I'm really pleased that you're working on a custom  
> client..
> we're trying to make the allmydata service into an actual flexible  
> storage
> service (you pay money for space on the production grid), as opposed  
> to
> merely a one-client backup product (you pay money for the ability to  
> run a
> windows-only program that somehow does some kind of storage thing).  
> Clear
> protocol definitions, a security model that does not depend upon  
> keeping the
> implementation secret, and a clear separation of the Tahoe system  
> from the
> AllMyData business model are all parts of this effort.
>
> cheers,
> -Brian
>
>