[tahoe-dev] safety and Tahoe Lock Files

[zooko]

On Mar 3, 2008, at 8:43 PM, Brian Warner wrote:
>
> Interesting! I'll think more deeply about this later, for now I  
> have just a
> couple of smaller practical issues.
...
> Since we don't know whether we're going to eventually share a  
> mutable file
> write-cap or not, really this means we need to double the length of  
> the
> write-caps, so that they always contain both M1 and L1.
...
> This will slow down the common case, since we always have to write  
> to the
> lock file, even if contention is unlikely. I'm not sure how much of a
> slowdown it represents, but my hunch is between 3x and 4x.

These are good practice considerations.  We can minimize these costs  
by using Tahoe Lock Files at the application layer instead of the  
Decentralized Filesystem Layer.

Consider the use case of "The Allmydata.com Windows Product".  This  
product (which is fully open sourced, for those who are interested,  
although I think you have to ask Peter to send you the source code  
because it isn't all properly hosted yet) has the Windows Vista built- 
in backup service, an SMB<->Tahoe layer due to Mike Booker, and Tahoe  
client.  It also has a web user interface which people can use to  
browse and change their stored files from any web browser, served up  
by https://beta.allmydata.com .

Uncoordinated writes can happen in this setup, either because two  
different Windows machines have the backup product installed and they  
are backing up to a shared directory, or else because a Windows  
machine is changing the state of the user's stored files at the same  
time as the user changes their files through the web site.

If we started using Tahoe Lock Files, then the minimum number of lock  
files that we'd have to create are one per user, rather than one per  
mutable file or directory.  Likewise, the minimum times that you had  
to grab the lock is once per user action that writes to a shared  
directory, rather than once per write to a mutable file or directory.

This would require e.g. Mike Booker to implement the creation and  
usage of lock files himself in the Windows SMB layer, because that's  
where the knowledge is present of what file or directories could  
possibly be written to by the other computers, and whether a user  
gesture (such as the drag and drop of a large number of files) has  
happened.

On the other hand, if the use of the lock files were implemented in  
the Decentralized Filesystem Layer and used automatically for every  
mutable file and directory and for every write, then this would be  
simpler from the point of view of Mike's code, but less efficient in  
a few ways as you pointed out above.


> I'll think more about this tonight. The part that's bothering me  
> right now is
> that mutable files don't themselves provide atomic updates: that's  
> part of
> the joy of a distributed system :). So I'm not certain that seeing  
> your own
> lock value in L1 is a particularly reliable way to know that nobody  
> else
> grabbed the lock. After all, what if you just happened to contact a  
> different
> set of servers than the contender did?

This is a good question.  I imagine Tahoe Lock Files as an immediate  
fix to an immediate problem, which is simple and modular enough that  
we can deploy it and be sure that we're not risking data loss.

But thinking about Tahoe Lock Files causes me to start thinking about  
how the same safety could in the future be implemented at the  
Decentralized Filesystem Layer, gaining efficiency and making it  
easier for application writers to use it.  The first thought is that  
you could have a "lock object" which has the same storage index as  
the mutable file that it protects, which means that (usually) you  
would use the same or nearly the same set of servers for locking as  
for writing the data.  The end of that train of thought has a two- 
phase commit protocol between client and storage servers, which  
minimizes public key operations and round trips and bandwidth... ;-)

Regards,

Zooko