wiki:SftpFrontend

Version 8 (modified by davidsarah, at 2010-05-29T03:25:38Z) (diff)

discuss concurrent writes and snapshotting on read

The SFTP frontend is a server that optionally runs as part of a gateway node, and provides read/write access to the Tahoe grid via the SSH File Transfer Protocol.

See source:docs/frontends/FTP-and-SFTP.txt for how to enable and set up the SFTP frontend on a gateway. This page is for compatibility issues with particular SFTP clients, and assumes that you are using Tahoe-LAFS v1.7.0beta or later. Please add any more issues that you discover.

General compatibility issues

Before uploading a file to a Tahoe filesystem, the whole file has to be available. This means that the upload can only start when the file has been closed in the SFTP session. Particularly when writing large files, the client may time out between sending the close request and receiving the response (ticket #1041). This is known to be a problem for at least the WinSCP client, which has a close timeout of 15 seconds.

sshfs

sshfs is an SFTP client that allows filesystem access via FUSE (a user-space filesystem layer). It works on Linux and other Unix systems that provide FUSE. For Mac OS X, a patched version of sshfs is included as part of MacFUSE.

Tahoe's SFTP frontend includes several workarounds and extensions to make it function correctly with sshfs.

When writing a file to the Tahoe filesystem, sshfs does not wait for the 'close' request to complete before reporting to the application that the file has been successfully closed (#1059). Therefore, you should not shut down your gateway node immediately after writing files via sshfs, otherwise those files may be lost. It is possible that an upload could fail (due to a network error, lack of storage space, etc.); such failures will not be reported to applications using sshfs. (This patch makes sshfs wait for close requests to complete, but may cause its own compatibility problems; the patch is provided only for testing purposes.)

Some applications may make assumptions that are incompatible with Tahoe. For example, 'flushing' a file does not guarantee that written data is reflected in the Tahoe filesystem, so opening the same file via another handle and attempting to read that data before the original handle is closed will not work.

If a file is written via two handles concurrently, the eventual contents will be the data written via one handle or the other, not an interleaving as would be the case for a POSIX filesystem. Also, the contents read via any handle will be a snapshot at about the time of the open. These differences from the POSIX semantics are arguably improvements, but in principle they could confuse some applications.

The MacFUSE version of sshfs stores "extended attributes" in files with names starting with "._". For example the attributes for "foo.txt" would be stored in a file called "._foo.txt". Since some Mac OS X applications may depend on these attributes (especially for their own file formats), if you need to copy or move the original file then you should copy or move the attribute file along with it. The OS X cp and mv commands will do this by default; operations using the Tahoe WUI or CLI will not. Note that filenames beginning with "." are not listed by default by ls.

Gnome virtual filesystem (gvfs)

gvfs is a set of filesystem adapters provided with the Gnome window system. It can be used in two ways: either via the GIO API, or via a FUSE layer called gvfs-FUSE (not to be confused with sshfs).

Apps that use the GIO API, such as the Nautilus file browser, seem to work correctly with Tahoe.

gvfs-FUSE, on the other hand, is not recommended for use with Tahoe. This is because it has to map POSIX filesystem requests onto GIO requests, and this mapping loses information -- some combinations of 'open' flags cannot be expressed in the GIO API, for example. Therefore it is impossible for gvfs-FUSE to provide a fully correct FUSE filesystem.

It may not be entirely clear to users whether a particular Gnome app is using GIO or gvfs-FUSE.

Attachments (1)

Download all attachments as: .zip