Magic Folder on Mac OS X and other BSDs

[davidsarah]

The Magic Folder feature currently only works on Linux (where it uses inotify to detect filesystem changes), and Windows (where it uses ReadDirectoryChangesW). This ticket is about supporting the same feature on Mac OS X and possibly other BSD-based operating systems.

Mac OS X has the ​fsevents API, and all BSDs including Mac OS X have kqueue/kevent. If I understand correctly, neither fsevents nor kqueue/kevent give notifications of which files have changed, so the implementation will have to scan the directory and look at last-modified times to determine that. (This is occasionally also necessary for correctness on Linux and Windows; see #1430.)

[davidsarah]

Python's ​select module supports kqueue, which is a lot easier and less error-prone than accessing it via ctypes or an extension module. Note that a ​critical bug in that support was fixed in Python 2.6.5. (2.7 was released after 2.6.5 and has this fix.)

[malaparte]

I thought I'd just save this thought here, because I don't know if I have time to look at it closely. Have you considered 'watchdog' ​http://packages.python.org/watchdog/ to watch for changes on a folder across platforms?

[davidsarah]

watchdog uses a C ​extension module to access the fsevents API on OS X. I would strongly prefer to use ctypes for this and not add any more native code dependencies.

[davidsarah]

Also,

davidsarah@shinier:~/tahoe/git$ cloc --no3 --quiet watchdog

http://cloc.sourceforge.net v 1.09  T=4.0 s (15.0 files/s, 2563.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          36           1286           2238           4010
C                                3            168            473            738
Bourne Shell                    13             71            188            273
make                             3             46              5            193
DOS Batch                        2             27              1            172
XML                              1              0              0            155
C/C++ Header                     1             28             68             93
YAML                             1              0              0             21
-------------------------------------------------------------------------------
SUM:                            60           1626           2973           5655
-------------------------------------------------------------------------------

plus watchdog's ​PyYaml, argh and pathtools dependencies

compared to:

davidsarah@shinier:~/tahoe/git$ cloc --no3 --quiet trunk/support/lib/python2.6/site-packages/Twisted-12.0.0-py2.6-linux-x86_64.egg/twisted/internet/inotify.py

http://cloc.sourceforge.net v 1.09  T=0.5 s (2.0 files/s, 810.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                           1             84            140            181
-------------------------------------------------------------------------------
davidsarah@shinier:~/tahoe/git$ cloc --no3 --quiet trunk/src/allmydata/frontends/drop_upload.py

http://cloc.sourceforge.net v 1.09  T=0.5 s (2.0 files/s, 246.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                           1             25             10             88
-------------------------------------------------------------------------------

plus how ever much code it takes to support Windows (about ​200-300 code lines), and BSD/OS X.

[davidsarah]

I should mention that I don't care about lines of code per se; I'm just using it as a rough proxy measure for complexity and opportunity-for-bugs.

[daira]

Add magic-folder keyword to all drop-upload tickets.

[daira]

SpiderOak has an fsevents-based OS X filesystem watcher which is open-source (GPL): ​https://github.com/SpiderOak/spideroak_osx_fsevents

[Edit: I did not mean that we should necessarily use this as-is.]

[dawuud]

step number one seems to be: find an OSX VPS provider!

[zooko]

I think we should start with ​https://github.com/SpiderOak/spideroak_osx_fsevents and then consider changing it or switching to other techniques or whatever after we have it running.

[zooko]

Okay I looked at ​https://github.com/SpiderOak/spideroak_osx_fsevents/blob/master/main.c and other code in watchdog and so on and I changed my mind, because ​https://github.com/SpiderOak/spideroak_osx_fsevents doesn't have a Python API.

We already have a ctypes-based API to Linux and Windows, so I guess our code would be more consistent, and easy for us to write, if we also wrote a ctypes-based API to OSX?

[dawuud]

Of the several steps needed to setup the Apple async filesystem events API we need to firstly call FSEventStreamCreate.

There's a snippet of code showing the FSEventStreamCreate being used:

​https://developer.apple.com/library/mac/documentation/Darwin/Conceptual/FSEvents_ProgGuide/UsingtheFSEventsFramework/UsingtheFSEventsFramework.html

It doesn't look like ctypes allows the creation of c structs or objective c objects unless you redefine them and create them the ctypes way. So initially I was thinking that maybe recreating all the necessary structs/objective-c-objects in python ctypes manually is kinda tricky... and requires us to have the Apple source code or at least the .h header files with the struct/object definitions.

Perhaps the pyobjc library can help us to compose these objective-c structures so that we can use ctypes to pass them into API functions. Here need a python equivalent for composing the CFArray of CFString which holds the filesystems paths to watch:

    CFStringRef mypath = CFSTR("/path/to/scan");

    CFArrayRef pathsToWatch = CFArrayCreate(NULL, (const void **)&mypath, 1, NULL);

    void *callbackInfo = NULL; // could put stream-specific data here.

    FSEventStreamRef stream;

    CFAbsoluteTime latency = 3.0; /* Latency in seconds */

The pyobjc module requies xcode and when I pip installed it after properly installed xcode it downloaded many more modules; I wonder if this thing is too heavy weight. We probably want to avoid heavyweight dependencies?

I also discovered that google wrote a ctypes wrapper for Apple objective-c structures in this project:

​https://github.com/google/grr ​https://github.com/google/grr/blob/master/grr/client/osx/objc.py

oh and i also found this other ctypes wrapper API here: ​https://github.com/wbond/oscrypto/blob/master/oscrypto/_osx/_core_foundation_ctypes.py

[dawuud]

Currently I have a failed attempt at creating this CFArray of CFString`s; running it results in a segmentation fault... however the VM i'm using won't write a core file. What to do?

(virtenv-objc) xcloud114:~ Xcloud$ python test.py
Segmentation fault: 11

#!/usr/bin/env python                                                                                                             

import ctypes
from ctypes import POINTER, byref, create_string_buffer, addressof
from ctypes import cdll, c_void_p
from ctypes.util import find_library

# kCFStringEncodingUTF8                                                                                                           
UTF8 = 134217984
# kCFAllocatorDefault                                                                                                             
CF_DEFAULT_ALLOCATOR = None

core_services = cdll.LoadLibrary(find_library('CoreServices'))
core_foundation = cdll.LoadLibrary(find_library('CoreFoundation'))

def PyStringToCFString(pystring):
    return core_foundation.CFStringCreateWithCString(CF_DEFAULT_ALLOCATOR,
                                  pystring.encode('utf8'),
                                  UTF8)

my_cfarray = core_foundation.CFArrayCreate(None, None, 0, ctypes.c_int.in_dll(core_services, "kCFTypeArrayCallBacks"))
my_cfstr = PyStringToCFString("/Users/Xcloud/magic-folder")

core_foundation.CFArrayAppendValue(my_cfarray, my_cfstr)

[dawuud]

today i wrote a partial working prototype for Mac OS X using the python watchdog library, here in my dev branch:

​https://github.com/david415/tahoe-lafs/tree/1432.osx-watchdog.0

many of the magic-folder tests pass. however it seems that test_magic_folder and test_move_tree fail due to watchdog not recursively watching directories or not adding a watch to newly added directories or some such thing.

anyway we should be able to replace this usage of the watchdog api with a ctypes equivalent for OSX if we want to do that.

[zooko]

Whoo-hoo! Way to go!

[dawuud]

this actually is a BSD compatibility change since getting magic-folder to work with this watchdog based inotify API actually makes it work on all the BSD operating systems using kqueue and it should also work on Mac OS X using the native apple filesystem events api.

we could of course replace it with two ctypes based inotify APIs. one for kqueue and one for Mac OS X fs events api...

currently there are only two or three unit tests that fail and these failures are not so terrible... it fails because it's emitting more events than the unit tests expect.

[dawuud]

the latest watchdog inotify magic-folder branch is here: ​https://github.com/tahoe-lafs/tahoe-lafs/tree/1432.osx-watchdog.0

you may note that the travis ci tests fail but this is due to changes in the dot travis yaml file as we attempt to get it working in a linux container and an osx container. this difficulty with the travis configuration was due to the removal of the config options specifying python and instead telling travis to run in either a linux or osx container. then we are required to manually install many package dependencies so that pip and tahoe-lafs function properly and so on. i have previously gotten the travis configuration file to work properly for linux and osx... but this indeed involved changing other configuration options such as enabling sudo.

what's the next step to getting this merged into tahoe master or magic-folder stable?

[daira]

dawuud: please paste the current test failures.

[dawuud]

[FAIL]
Traceback (most recent call last):
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/test_magic_folder.py", line 1104, in <lambda>
    d.addCallback(lambda ign: self.failUnlessReallyEqual(self._get_count('uploader.objects_succeeded'), 4))
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/common_util.py", line 35, in failUnlessReallyEqual
    self.failUnlessEqual(a, b, msg=msg)
  File "/Users/Xcloud/virtenv-tahoe/lib/python2.7/site-packages/twisted/trial/_synctest.py", line 437, in assertEqual
    super(_Assertions, self).assertEqual(first, second, msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 513, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
    raise self.failureException(msg)
twisted.trial.unittest.FailTest: 6 != 4

allmydata.test.test_magic_folder.RealTest.test_move_tree
===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/test_magic_folder.py", line 687, in <lambda>
    d.addCallback(lambda ign: self._check_uploader_count('objects_not_uploaded', 1, magic=self.bob_magicfolder))
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/test_magic_folder.py", line 211, in _check_uploader_count
    expected)
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/common_util.py", line 35, in failUnlessReallyEqual
    self.failUnlessEqual(a, b, msg=msg)
  File "/Users/Xcloud/virtenv-tahoe/lib/python2.7/site-packages/twisted/trial/_synctest.py", line 437, in assertEqual
    super(_Assertions, self).assertEqual(first, second, msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 513, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
    raise self.failureException(msg)
twisted.trial.unittest.FailTest: 2 != 1

allmydata.test.test_magic_folder.RealTestAliceBob.test_alice_bob
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/Users/Xcloud/virtenv-tahoe/lib/python2.7/site-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/test_magic_folder.py", line 169, in _check_file
    os.unlink(path_u)
exceptions.OSError: [Errno 2] No such file or directory: '/Users/Xcloud/tahoe-lafs/_trial_temp/allmydata.test.test_magic_folder/MockTest/test_magic_fo
lder/gIRrj8/temp/local_dir/tempfile'

allmydata.test.test_magic_folder.MockTest.test_magic_folder
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/Users/Xcloud/virtenv-tahoe/lib/python2.7/site-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/test_magic_folder.py", line 169, in _check_file
    os.unlink(path_u)
exceptions.OSError: [Errno 2] No such file or directory: '/Users/Xcloud/tahoe-lafs/_trial_temp/allmydata.test.test_magic_folder/RealTest/test_magic_fo
lder/ff93bG/temp/local_dir/tempfile'

allmydata.test.test_magic_folder.RealTest.test_magic_folder
-------------------------------------------------------------------------------
Ran 35 tests in 171.881s

FAILED (failures=2, errors=2, successes=31)

[dawuud]

the above test results were actually from an earlier commit; this commit id 8939fbf16baa2cbd9ede7b23037b61dfbe323cdd

seems to have fewer test errors and failures than the HEAD of that branch. This is of course because the code review "corrections" caused more tests to fail... I'll have to track down which changes cause the problems. I suspect they are unicode related changes.

Furthermore our inotify/watchdog API must implement two more features:

1. allow the user to set an events mask to select which events to get notifications for

2. event attributes. the event messages may contain information about the file object which caused the events such as: file type (e.g. file or directory).

HOWEVER, given that the watchdog API passes nearly all the magic-folder unit tests, this means either the watchdog api is good enough or our unit test coverage sucks.

[dawuud]

i made a new dev branch with the watchdogi inotify stable changes on top of the latest stable magic-folder branch (stable 13): ​https://github.com/david415/tahoe-lafs/tree/1432.osx-watchdog-stable.1

pull request here: ​https://github.com/tahoe-lafs/tahoe-lafs/pull/287

[dawuud]

new dev branch based off of meejah's magic-folder stable 18 branch here: ​https://github.com/david415/tahoe-lafs/tree/1432.osx-watchdog-stable.3

no errors, two failures:

===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/Users/Xcloud/virtenv-tahoe/lib/python2.7/site-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/test_magic_folder.py", line 183, in _check_file
    previously_disappeared + 1)
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/common_util.py", line 35, in failUnlessReallyEqual
    self.failUnlessEqual(a, b, msg=msg)
  File "/Users/Xcloud/virtenv-tahoe/lib/python2.7/site-packages/twisted/trial/_synctest.py", line 437, in assertEqual
    super(_Assertions, self).assertEqual(first, second, msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 513, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
    raise self.failureException(msg)
twisted.trial.unittest.FailTest: 0 != 1

allmydata.test.test_magic_folder.RealTest.test_magic_folder
===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/test_magic_folder.py", line 1131, in <lambda>
    d.addCallback(lambda ign: self.failUnlessReallyEqual(self._get_count('uploader.objects_succeeded'), 4))
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/common_util.py", line 35, in failUnlessReallyEqual
    self.failUnlessEqual(a, b, msg=msg)
  File "/Users/Xcloud/virtenv-tahoe/lib/python2.7/site-packages/twisted/trial/_synctest.py", line 437, in assertEqual
    super(_Assertions, self).assertEqual(first, second, msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 513, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
    raise self.failureException(msg)
twisted.trial.unittest.FailTest: 6 != 4

allmydata.test.test_magic_folder.RealTest.test_move_tree
-------------------------------------------------------------------------------
Ran 35 tests in 182.773s

FAILED (failures=2, successes=33)

[dawuud]

however a simple change, ignore events outside the watched directory shows the score a bit closer to the target:

===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/Users/Xcloud/virtenv-tahoe/lib/python2.7/site-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/test_magic_folder.py", line 183, in _check_file
    previously_disappeared + 1)
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/common_util.py", line 35, in failUnlessReallyEqual
    self.failUnlessEqual(a, b, msg=msg)
  File "/Users/Xcloud/virtenv-tahoe/lib/python2.7/site-packages/twisted/trial/_synctest.py", line 437, in assertEqual
    super(_Assertions, self).assertEqual(first, second, msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 513, in 
assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 506, in 
_baseAssertEqual
    raise self.failureException(msg)
twisted.trial.unittest.FailTest: 0 != 1

allmydata.test.test_magic_folder.RealTest.test_magic_folder
===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/test_magic_folder.py", line 1131, in <lambda>
    d.addCallback(lambda ign: self.failUnlessReallyEqual(self._get_count('uploader.objects_succeeded'), 4))
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/common_util.py", line 35, in failUnlessReallyEqual
    self.failUnlessEqual(a, b, msg=msg)
  File "/Users/Xcloud/virtenv-tahoe/lib/python2.7/site-packages/twisted/trial/_synctest.py", line 437, in assertEqual
    super(_Assertions, self).assertEqual(first, second, msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 513, in 
assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 506, in 
_baseAssertEqual
    raise self.failureException(msg)
twisted.trial.unittest.FailTest: 3 != 4

allmydata.test.test_magic_folder.RealTest.test_move_tree
-------------------------------------------------------------------------------
Ran 35 tests in 188.915s

FAILED (failures=2, successes=33)

[dawuud]

new error output from tests show all tests pass except one:

    test_magic_folder ... set pending delay                                                                                                   [2/1850]
init INotifyEventHandler
START READING BEGIN
START READING END
PROCESS EVENT <FileCreatedEvent: src_path='/Users/Xcloud/tahoe-lafs/_trial_temp/allmydata.test.test_magic_folder/RealTest/test_magic_folder/Q5TLUN/te$p/local_dir/short'>
PROCESS EVENT <DirModifiedEvent: src_path='/Users/Xcloud/tahoe-lafs/_trial_temp/allmydata.test.test_magic_folder/RealTest/test_magic_folder/Q5TLUN/te$
p/local_dir'>
DO CALLBACKS
                                               [FAIL]
stopReading begin
stopReading end
wait until stopped

===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/Users/Xcloud/virtenv-tahoe/lib/python2.7/site-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/test_magic_folder.py", line 188, in _check_file
    self.failUnlessReallyEqual(actual_data, data)
  File "/Users/Xcloud/tahoe-lafs/src/allmydata/test/common_util.py", line 35, in failUnlessReallyEqual
    self.failUnlessEqual(a, b, msg=msg)
  File "/Users/Xcloud/virtenv-tahoe/lib/python2.7/site-packages/twisted/trial/_synctest.py", line 437, in assertEqual
    super(_Assertions, self).assertEqual(first, second, msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 513, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
    raise self.failureException(msg)
twisted.trial.unittest.FailTest: 'test' != 'different'

allmydata.test.test_magic_folder.RealTest.test_magic_folder
-------------------------------------------------------------------------------
Ran 2 tests in 5.686s

FAILED (failures=1, successes=1)

[dawuud]

in another dev branch i saved a commit with added debug print statements so i could track down the root cause of the above unit test failure: ​https://github.com/david415/tahoe-lafs/tree/1432.osx-watchdog-stable.3.wip.1

what is happening is that this test case fails: ​https://github.com/david415/tahoe-lafs/blob/e451e2eaa410d84aa3ae0b62a5aaf5a4d4aa786b/src/allmydata/test/test_magic_folder.py#L1240

_check_file is failing on the temporary file case; specifically here's the comparison that fails:

​https://github.com/david415/tahoe-lafs/blob/e451e2eaa410d84aa3ae0b62a5aaf5a4d4aa786b/src/allmydata/test/test_magic_folder.py#L182

yield self.failUnlessReallyEqual(self._get_count('uploader.objects_disappeared'),
                                             previously_disappeared + 1)

uploader.objects_disappeared 0
previously_disappeared + 1 == 1

OK so it uploads the temporary file when it shouldn't. But is that really so bad? Seems better than not uploading enough.

[dawuud]

I have had two conversations about magic-folder unit test race conditions; one with Daira and one with Meejah. Currently the plan is for Meejah to fix/refactor all the uploader unit tests after figuring out a better API for uploader testing similar in spirit to how we refactored the downloader tests. Here's the ticket for that:

​https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2790

[dawuud]

here i've put the watchdog inotify changes on top of meejah's stable 18 branch: ​https://github.com/david415/tahoe-lafs/tree/1432.osx-watchdog-stable.4

[dawuud]

oh dear the linux build passes but the osx build fails with the current travis yaml configuration::

RuntimeError?: You are linking against OpenSSL 0.9.8, which is no longer support by the OpenSSL project. You need to upgrade to a newer version of OpenSSL.

there's a ticket regarding automated testing on OSX here: ​https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2789

[warner]

BTW, that openssl thing was fixed. The travis OS-X builder was using the wrong version of python, and failing to pick up the newer openssl library. If you update the PR, travis should run with the proper version.

[dawuud]

the current dev branch is: ​https://github.com/david415/tahoe-lafs/tree/1432.osx-watchdog-stable.5

and current pull request is: ​https://github.com/tahoe-lafs/tahoe-lafs/pull/308

[dawuud]

currently on the darwin platform fails at least this test: allmydata.test.test_magic_folder.RealTest?.test_delete

I don't understand why this fails. I added a print statement to display the platform filesystem encoding which is UTF-8

` test_delete ... set pending delay

init INotifyEventHandler

START READING BEGIN

START READING END

PROCESS EVENT <FileCreatedEvent: src_path='/Users/travis/build/tahoe-lafs/tahoe-lafs/_trial_temp/allmydata.test.test_magic_folder/RealTest/test_delete/nvdp3V/temp/local_dir/foo'>

FILESYSTEM ENCODING: utf-8

Exception in thread Thread-8:

Traceback (most recent call last):

File "/System/Library/Frameworks/Python?.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in bootstrap_inner

self.run()

File "/Users/travis/build/tahoe-lafs/tahoe-lafs/.tox/py27/lib/python2.7/site-packages/watchdog/observers/api.py", line 199, in run

self.dispatch_events(self.event_queue, self.timeout)

File "/Users/travis/build/tahoe-lafs/tahoe-lafs/.tox/py27/lib/python2.7/site-packages/watchdog/observers/api.py", line 368, in dispatch_events

handler.dispatch(event)

File "/Users/travis/build/tahoe-lafs/tahoe-lafs/.tox/py27/lib/python2.7/site-packages/watchdog/events.py", line 322, in dispatch

self.on_any_event(event)

File "/Users/travis/build/tahoe-lafs/tahoe-lafs/src/allmydata/watchdog/inotify.py", line 73, in on_any_event

self.process(event)

File "/Users/travis/build/tahoe-lafs/tahoe-lafs/src/allmydata/watchdog/inotify.py", line 47, in process

event_filepath_u = abspath_expanduser_unicode(event_filepath_u, base=self._path)

File "/Users/travis/build/tahoe-lafs/tahoe-lafs/src/allmydata/util/fileutil.py", line 310, in abspath_expanduser_unicode

precondition_abspath(base)

File "/Users/travis/build/tahoe-lafs/tahoe-lafs/src/allmydata/util/fileutil.py", line 276, in precondition_abspath

raise AssertionError?("an abspath must be a Unicode string")

AssertionError?: an abspath must be a Unicode string `

[exarkun]

Some changes built on david's last branch - ​https://github.com/exarkun/tahoe-lafs/tree/1432.osx-watchdog-stable.9

In particular, this adds a direct inotify-level test suite (copied from Twisted!) and makes some of the tests pass - including the IN_CLOSE_WRITE notification.

[dawuud]

more progress here in this dev branch. i've changed the inotify test suite to skip many of the tests for inotify features that we do not use or need. I also modified some of the tests so that they now pass when using either the watchdog inotify and the linux native inotify. not yet tested with our windows inotify implementation.

​https://github.com/david415/tahoe-lafs/tree/1432.osx-watchdog-stable.9

[dawuud]

now that we have a inotify test suite perhaps we can remove the "real" inotify magic folder tests?

in the above mentioned dev branch i've removed the real tests in this commit here: ​https://github.com/david415/tahoe-lafs/commit/451dcf093b26b475378ef588e2cf68b40697cc8c

[meejah]

Is the reactor.callFromThread() because the "watchdog" library is running the on_any_event from its own thread(s)?

Otherwise looks plausible. And yes I'd like to get rid of some of the "long" magic-folder tests especially. If our test show that all 3 underlying inotify-like implementations are doing the right thing, then yes it would be great to get rid of the "Real" vs "Mock" dual tests in magic-folder as well! (i.e. just have the Mock ones).

[dawuud]

this latest dev branch is currently in a pull-request ​https://github.com/tahoe-lafs/tahoe-lafs/pull/381

all tests pass except for the appveyor testing on windows. currently i can see that the inotify tests are failing for windows; the test_attrib fails and test_closeWrite timesout.

meejah and i plan to look at this soon.

[dawuud]

i'm trying to make this branch pass all the tests. in my latest commit i'm trying to make the test_inotify tests be skipped on windows... we can make the tests pass on windows later. for now i'd like to get the bsd changes merged if possible.

[dawuud]

my latest dev branch: ​https://github.com/david415/tahoe-lafs/tree/1432.osx-watchdog-stable.10

all tests pass. the magic-folder "real inotify" tests have been disabled. the watch inotify implementation has it's own test suite that was borrowed and inspired from twisted's inotify test suite.

[dawuud]

attempting to skip the inotify tests on windows: ​https://github.com/tahoe-lafs/tahoe-lafs/pull/401

the tests seem to be skipped but prints dirty reactor errors like this:

test_closeWrite ... [SKIPPED] Traceback (most recent call last): Failure: twisted.trial.util.DirtyReactorAggregateError?: Reactor was unclean. DelayedCalls?: (set twisted.internet.base.DelayedCall?.debug = True to debug) <DelayedCall? 0x4af3eb8 [0.00999999046326s] called=0 cancelled=0 LoopingCall?<0.01>(INotify._poll, *(<function <lambda> at 0x069DE770>, 1487535067.863), {})()> [ERROR]

why!? how to fix?

[dawuud]

fixed it

[cypher]

Freshly resurrected in ​PR 516.

[exarkun]

Bad news. Watchdog upstream have decided to stop supporting Python 2.

​https://github.com/gorakhargosh/watchdog/issues/516

[exarkun]

They changed their mind. There are some other issues, though, like events being missed with the fsevents backend on macOS.

[GitHub <noreply@…>]

In c1e6f08/trunk:

Merge pull request #558 from tahoe-lafs/1432.watchdog-magic-folder-with-eliot

Add support for macOS to Magic-Folders.

Fixes: ticket:1432

[exarkun]

Note to all, the above merge really did only add macOS support - not BSD. Sorry. Tickets should be about one thing, though.