Twisted endpoints introduce a dependency on pywin32

[daira]

In #1274 we managed to eliminate Tahoe's problematic dependency on pywin32, which significantly complicated building on Windows. However this dependency seems to have crept in again via Twisted's endpoints code:

On 10/07/13 18:44, Mike Nation wrote:
>      File "C:\tahoe\support\Lib\site-packages\twisted-12.3.0-py2.7-win-amd64.egg\twisted\internet\endpoints.py", line 25, in <module>
>     from twisted.internet import stdio
>      File "C:\tahoe\support\Lib\site-packages\twisted-12.3.0-py2.7-win-amd64.egg\twisted\internet\stdio.py", line 28, in <module>
>     from twisted.internet import _win32stdio
>      File "C:\tahoe\support\Lib\site-packages\twisted-12.3.0-py2.7-win-amd64.egg\twisted\internet\_win32stdio.py", line 7, in <module>
>     import win32api
>
> exceptions.ImportError: No module named win32api
>
> Failed to load application: No module named win32api

Note that we probably didn't detect this earlier because of #1334.

[daira]

This would be fixed by ​http://twistedmatrix.com/trac/ticket/6032.

[daira]

Workaround: manually install the pywin32 package from ​http://sourceforge.net/projects/pywin32/files/. Make sure to get the correct file for the version of Python you are using -- e.g. ending in "py2.7.exe" for Python v2.7. If using 64-bit Windows, the file should have "win-amd64" in its name.

[daira]

Options:

1. Update docs/quickstart.rst to point to downloads of pywin32 (and describe how to select the right one). We used to have this information; adding it back is a usability regression.
2. Build eggs of pywin32 for 32-bit and 64-bit Python 2.7, and make sure they are used by a quickstart build. This may be hampered by #142.
3. Submit a patch to Twisted to remove the hard dependency on pywin32. Update src/allmydata/_auto_deps.py to allow only versions of Twisted that don't have the hard dependency (either before endpoints, or after the fix), when running on Windows.
4. Update src/allmydata/_auto_deps.py to allow only versions of Twisted that are before endpoints, when running on Windows.

[zooko]

Let's do option 4 now, and then work on option 3.

[daira]

Please review ​https://github.com/tahoe-lafs/tahoe-lafs/commit/228b5ef7dbcd56ff57c9a614c75df74a1c6ca0e1.

[daira]

Correction: ​https://github.com/tahoe-lafs/tahoe-lafs/commit/0c81c6e22bdcb641ab9eb6c9cef0d78ad5e2ec00.

[daira]

Correction: ​https://github.com/tahoe-lafs/tahoe-lafs/commit/d888b28b715020e879a52f5159b6e14c7dcab6b8.

[zooko]

I think the dependency on pywin32 is part of, or even maybe most of, #1093 (win32 build hell).

[zooko]

This interacts with ​https://github.com/twisted/nevow/issues/43 (why does Nevow depend on Twisted >= 13.0?).

[daira]

The combination of this problem, #2193, and #2249, are making it almost impossible to install or build on Windows :-(

[Daira Hopwood <daira@…>]

In b0b76a7c5b89c3fed5a65ef6732dc45e578f12f4/trunk:

Improve comments in _auto_deps.py. refs #2249, #2028, #2193, #2005, #1258

Signed-off-by: Daira Hopwood <daira@…>

[daira]

Fixed by [ec188d254cbb77204d1898d6a0dd8727e0f9bd6d/trunk], kind-of, but only by excluding Twisted >= 12.2.0 on Windows. We should revisit this if the upstream ​http://twistedmatrix.com/trac/ticket/6032 is fixed.

[daira]

Magic Folder requires Twisted >= 15.2.0, so we can no longer use the above workaround once that is landed.

​http://twistedmatrix.com/trac/ticket/6032 shows no sign of progress. Note that the pypiwin32 package mentioned there is a wheel distribution, therefore not compatible with zetuptoolz.

[daira]

See also #2044 and #2077.

[warner]

We are now ok with the dependency on pypiwin32, because we've switched to modern pip/setuptools/virtualenv, so the wheels work correctly.