Magic Folder on Windows: make subdirectories work

[daira]

It appears that the uploader does get notified about the subdirectory creation, but then hits an assertion when trying to upload it:

01:02:59.710 L20 []#366054 Magic Folder 'alice' uploader: inotify event None, FilePath(u'\\\\?\\D:\\local\\alice\\test'), 'modify, attrib'

01:02:59.710 L20 []#366055 Magic Folder 'alice' uploader: _get_relpath(FilePath(u'\\\\?\\D:\\local\\alice\\test'))
01:02:59.711 L20 []#366056 Magic Folder 'alice' uploader: segments = [u'test']
01:02:59.713 L20 []#366057 Magic Folder 'alice' uploader: _append_to_deque(u'test')
01:02:59.713 L20 []#366058 Magic Folder 'alice' uploader: objects_queued += 1
01:02:59.730 L20 []#366059 Magic Folder 'alice' uploader: _turn_deque
01:02:59.733 L20 []#366060 Magic Folder 'alice' uploader: popped u'test'
01:02:59.733 L20 []#366061 Magic Folder 'alice' uploader: objects_queued += -1
01:02:59.733 L20 []#366062 Magic Folder 'alice' uploader: _process(u'test')
01:02:59.733 L20 []#366063 Magic Folder 'alice' uploader: pending = set([u'test']), about to remove u'test'
01:02:59.734 L20 []#366064 Magic Folder 'alice' uploader: objects_failed += 1
01:02:59.734 L20 []#366065 Magic Folder 'alice' uploader: <twisted.python.failure.Failure <type 'exceptions.AssertionError'>> while processing u'test'
01:02:59.782 L20 []#366066 Magic Folder 'alice' uploader: _turn_deque
01:02:59.782 L20 []#366067 Magic Folder 'alice' uploader: deque is now empty

[daira]

The next step is to improve error reporting of assertions in the uploader.

[daira]

Aha!

01:39:03.246 L20 []#759 Magic Folder 'alice' uploader:
[Failure instance: Traceback: <type 'exceptions.AssertionError'>:
precondition: 'watch() can only be called before startReading()'
<type 'str'>, state: 'STARTED' type 'str'>
d:\tahoe\git\tahoe-magic\src\allmydata\frontends\magic_folder.py:169:<lambda>
d:\tahoe\git\tahoe-magic\src\allmydata\frontends\magic_folder.py:402:_process
C:\Python27\lib\site-packages\twisted\internet\defer.py:317:addCallback
C:\Python27\lib\site-packages\twisted\internet\defer.py:306:addCallbacks
--- <exception caught here> ---
C:\Python27\lib\site-packages\twisted\internet\defer.py:588:_runCallbacks
d:\tahoe\git\tahoe-magic\src\allmydata\frontends\magic_folder.py:354:_maybe_upload
d:\tahoe\git\tahoe-magic\src\allmydata\windows\inotify.py:222:watch
d:\tahoe\git\tahoe-magic\src\allmydata\util\assertutil.py:39:precondition
] while processing u'test5'

[daira]

This happens because the Uploader thinks that it must add watches for new subdirectories, which is true on Linux but not on Windows.

[daira]

This seems to be fixed by ​https://github.com/tahoe-lafs/tahoe-lafs/commit/14097eec7eb94a6418be94c4306c5469c8fa9644, although there's still some odd behaviour in the Downloader (it seems to be trying to repeatedly create directories even when it should know from its magic folder db that they already exist).

[daira]

Currently subdirectory watches are added with recursive=True. I'm not sure whether this will result in duplicate events being delivered if a watch for a directory is added twice.

[daira]

Replying to daira:

Currently subdirectory watches are added with recursive=True. I'm not sure whether this will result in duplicate events being delivered if a watch for a directory is added twice.

Actually that's not a problem because ​Twisted ignores duplicate watch requests. It's also necessary for the case where an existing directory is moved into a monitored one, so that there would otherwise be no notifications for its subdirectories.

[Brian Warner <warner@…>]

In ea14b1b/trunk:

Don't add subdirectory watches if the platform's notifier doesn't require them. refs #2559

Signed-off-by: Daira Hopwood <daira@…>

[daira]

This works and should have been closed.