[tahoe-dev] Keeping local file system and Tahoe store in sync

[Brian Warner]

On Fri, 6 Feb 2009 13:52:56 -0700
Shawn Willden <shawn-tahoe at willden.org> wrote:

> [1] I've also been playing with using the FSEvents log on Mac OS X to avoid
> having to scan the file system, and I've read that there's a way to get
> access to the NTFS journal so a system service can record file system
> changes that way. It should be possible on those platforms to get away from
> having to actually scan the file system. Linux doesn't seem to have
> anything equivalent, unfortunately (inotify is fine if you only care about
> a small portion of the FS).

For linux, look at 'dnotify', I think it's closer to what you want.

If I remember right, for OS-X, fsevents should give you reliable "this
directory might have been modified" information (i.e. it won't give you a
false negative, but it might give you a false positive). You must scan every
directory it tells you about, but sometimes nothing will have changed.

> [2] The files uploaded may be full revisions or diffs. In either case, the
> encryption key used will be the full revision hash.

Be careful to not use the same key for multiple purposes. Tahoe always uses a
tagged hash when deriving a key in this fashion:

 key = SHA256d(netstring("file contents to encryption key v1") + contents)

Also be careful to not re-use the key in subsequent backup runs, if it's
going to be used on different data.

(I'll try to review and respond to the rest of your note later)

cheers,
 -Brian