append-only files

[warner]

It's a long-term goal, but it would be nice to have append-only mutable files in Tahoe. Specifically, that means some sort of mutable slot that has the following caps:

• writecap (permits arbitrary modification)
• appendcap (permits appending new data, not removing existing data)
• readcap (permits reading arbitrary data)
• verifycap

Note that the appendcap and readcap are peers: neither can be derived from the other.

       writecap
       |      |
       v      v
 appendcap   readcap
       |      |
       v      v
       verifycap

This requires some tricky encryption, and will rely upon the servers to enforce certain types of message updates. (this means that a sufficiently-large set of colluding servers will be able to roll back appended messages until someone with the writecap comes along and flattens them all into the base file). I've got some notes on this scheme somewhere, which I'll eventually transcribe into this ticket. The basic approach is that the appendcap gets you a signing key, and the server will accept signed append messages, and put them next to all of the other messages. The writecap gets you a different signing key, with which the servers will accept arbitrary rewrite operations. I also have notes on revocation schemes (where you can create multiple "subwritecaps" for an object, but revoke them later), which involves even more sets of signing keys.

This could be integrated with LDMF mutable files, in which the servers hold a series of append-only revision nodes, forming a version-history graph (which I still think should be based on Mercurial's "revlog" format). In this case, writecap holders get to add revision nodes, but not remove older ones.