[tahoe-dev] Tahoe-LAFS + sshfs latency issues

[Linus Lüssing]

Hi everyone,

I've recently started looking at Tahoe-LAFS and I'm currently
trying to get a feeling of how it works. The use-case I'm
currently aiming at is to have an ftpd and httpd (proftpd +
lighttpd) running on a distributed file system, where anyone can
just add and contribute to the storage capacity. Furthermore I'm
going to use it within an open, public, decentral wireless+vpn
mesh network (e.g. using the mesh routing protocol
B.A.T.M.A.N.-Advanced and the VPN software tinc).

To have it running transparently on the gateway node, I was
experimenting with tahoe-lafs and sshfs, I'll further just call
this gateway node "Node A". Then there is also a second storage
node, node B. Node A is configured to simply store everything in
the local tahoe-lafs storage, but also mirror it to node B
(shares.needed = 1, shares.happy = 2). sshfs is mounted on node A
via:

sshfs -p 8022 -o uid=107,gid=33,nonempty,cache_timeout=180,allow_other root at 127.0.0.1:/ /home/ftp

Now I'm uploading a large file with 'tahoe put' on node A. I'm
wating until the CPU usage is very low again and until node A is
starting to transfer the data to node B with about 1mbit/s.

The first thing I noticed was that lighttpd and proftpd, which are
using /home/ftp/ on node A, become quite unresponsive at this time.
To narrow things down and to benchmark things better I tried both
a normal cp and an scp from tahoe-lafs of a quite small file
(~4kB) with the following results:

time scp -P 8022 root at 127.0.0.1:/music/8bitpeoples/8BP102.gif /tmp/:
0m1.050s, 0m0.838s, 0m0.893s

time cp /home/ftp/music/8bitpeoples/8BP102.gif /tmp/:
0m25.539s, 0m29.389s, 0m27.414s


Without the 'tahoe put' running everything works as expected:

time scp -P 8022 root at 127.0.0.1:/music/8bitpeoples/8BP102.gif /tmp/:
0m0.937s, 0m1.001s, 0m0.962s

time cp /home/ftp/music/8bitpeoples/8BP102.gif /tmp/
0m0.651s, 0m0.587s, 0m0.624s


Anyone experiencing similar issues? Does anyone have an idea
what might be causing this high delay for the sshfs mount?

Cheers, Linus