[tahoe-dev] [tahoe-lafs] #935: zandr's FUSE/NAS idea

[tahoe-lafs]

#935: zandr's FUSE/NAS idea
-----------------------------------------------------+----------------------
 Reporter:  warner                                   |           Owner:            
     Type:  enhancement                              |          Status:  new       
 Priority:  major                                    |       Milestone:  eventually
Component:  code-frontend                            |         Version:  1.5.0     
 Keywords:  fuse smb sftp webdav cache preservation  |   Launchpad_bug:            
-----------------------------------------------------+----------------------
Changes (by davidsarah):

  * keywords:  fuse smb sftp webdav cache => fuse smb sftp webdav cache
               preservation
  * milestone:  undecided => eventually


Comment:

 This is not really a different frontend, it's just adding a cache to
 client storage nodes. The cache would be shared between frontend protocols
 (HTTP(S)/WebDAV, SFTP, and FTP), which is an advantage over putting a
 forward proxy between the node and the frontend client. Being able to
 cache only ciphertext is also an advantage.

 The ticket description essentially describes write-behind caching. For
 this kind of system, there are two significant 'commit' events for any
 write:
  * the time at which the write is "locally durable" -- subsequent reads
 via ''this'' gateway will see the write, and if the gateway machine or
 process crashes and then restarts, it will still try to complete the
 upload to the grid.
  * the time at which the write is "globally durable" -- subsequent reads
 via ''any'' gateway will see the write, and the servers-of-happiness
 criterion is met, so that the file has Tahoe's normal availability and
 preservation properties (see #946).

 Unfortunately existing apps and filesystem protocols don't distinguish
 these events. Ideally an operation such as an HTTP PUT or a file close()
 would notify its client twice, once for the local commit, and once for the
 global commit (this abstraction would be a good fit for distributed
 storage systems in general, not just Tahoe). But since they don't, we have
 to choose when to notify the client that the operation has succeeded. The
 client doesn't have any way to tell us what kind of notification it wants
 (there are no relevant flags defined either by POSIX or by HTTP or SFTP),
 so we have a Hobson's choice between performance or durability.

 I don't know how to resolve this, but I do think that caching on the
 gateway is going to be essential for good WebDAV or SFTP performance.
 sshfs does maintain a local cache, but when that is switched on it
 essentially assumes that only this instance of sshfs is accessing the
 filesystem, so unless we switch it off (see
 http://code.google.com/p/macfuse/wiki/FAQ#sshfs), sshfs users would not
 see updates to the Tahoe filesystem by other users. dav2fs would
 presumably have similar issues.

-- 
Ticket URL: <http://allmydata.org/trac/tahoe/ticket/935#comment:3>
tahoe-lafs <http://allmydata.org>
secure decentralized file storage grid