[tahoe-dev] Tahoe performance

[Luke Scharf]

Shawn Willden wrote:
> On Wednesday 11 February 2009 12:18:07 am Lazy Stream wrote:
>   
>> Why is anything necessary other than "mount Tahoe as a filesystem"
>> followed by "rsync -av /home/you/ /mnt/tahoe" or "rsync.exe -av
>> \users\you z:\" ? Restartable, orthogonal, smurfilicious.
>>     
>
> Can't work.
>
> Rsync can only work its magic if the remote machine can read the files (and 
> can run a copy of the rsync program).  The remote rsync instance needs to be 
> able to generate rolling checksums of the content and send them to the other 
> end which uses them to figure out what's changed, but one of the fundamental 
> design requirements for Tahoe was privacy -- the server should NOT be able to 
> read the files.  Thus rsync cannot work.
>   

He's running both sides of rsync locally.  You can use rsync to sync 
local filesystems, which is what he's suggesting.  All you need in order 
for that to work is for the filesystem to be mountable and behave in a 
minimally POSIX-ish way.

It's just like the following bash tricks for copying subtrees around the 
box:
    # (cd /home ; tar -cf - .) | (cd /tahoe ; tar -xvf - )
or
    # cp -rpv /home/* /tahoe/home/
Except that rsync will optimize the process by only overwriting the 
files that appear to have changed.

I used the local rsync trick earlier today to shove 16GB worth of VMWare 
images out to a mounted OpenAFS filesystem today, just because I like 
the "rsync -avP" progress display.

-Luke