Ticket #585: bb-freeze.darcs.3.patch

File bb-freeze.darcs.3.patch, 40.4 KB (added by davidsarah, at 2011-01-04T12:05:58Z)

Make bbfreeze work. This also refactors allmydata/init.py and _auto_deps.py and implements more robust checking of dependency requirements. It includes a change to the bundled darcsvar, to write the _version.py file with LF line endings.

Line 
16 patches for repository http://tahoe-lafs.org/source/tahoe-lafs/trunk:
2
3Sat Jan  1 11:01:41 GMT Standard Time 2011  david-sarah@jacaranda.org
4  * allmydata/__init__.py: don't use pkg_resources.require to get versions and paths. Also change pyOpenSSL to pyopenssl since that is the package name. fixes #1287
5
6Sat Jan  1 20:05:02 GMT Standard Time 2011  david-sarah@jacaranda.org
7  * allmydata/_auto_deps.py: add a comment reminding to change __init__.py when dependencies are added or removed. Clarify a comment in __init__.py. refs #1287
8
9Tue Jan  4 05:01:51 GMT Standard Time 2011  david-sarah@jacaranda.org
10  * bundled darcsver: write the _version.py file with LF line endings. This works around a problem with bb-freeze (refs #585). Don't apply this to trunk; instead, make a new darcsver release.
11
12Tue Jan  4 11:18:25 GMT Standard Time 2011  david-sarah@jacaranda.org
13  * Make bbfreeze work. This also refactors allmydata/__init__.py and _auto_deps.py and implements more robust checking of dependency requirements, as discussed in ticket:585#comment:10 . However, the executable is not yet testable. refs #585
14
15Tue Jan  4 11:23:00 GMT Standard Time 2011  david-sarah@jacaranda.org
16  * Delete obsolete code due to refactoring of allmydata/__init__.py and _auto_deps.py (done in a separate patch to avoid a confusing diff). refs #585
17
18Tue Jan  4 11:24:41 GMT Standard Time 2011  david-sarah@jacaranda.org
19  * Add static/tahoe.py, to support bbfreeze, py2exe etc. refs #585
20
21New patches:
22
23[allmydata/__init__.py: don't use pkg_resources.require to get versions and paths. Also change pyOpenSSL to pyopenssl since that is the package name. fixes #1287
24david-sarah@jacaranda.org**20110101110141
25 Ignore-this: 48593ba449d7453005c9d2683fed6a69
26] {
27hunk ./src/allmydata/__init__.py 178
28     else:
29         return platform.platform()
30 
31-def get_package_versions_from_setuptools():
32-    import pkg_resources
33-    return dict([(p.project_name, (p.version, p.location)) for p in pkg_resources.require(__appname__)])
34-
35 def package_dir(srcfile):
36     return os.path.dirname(os.path.dirname(os.path.normcase(os.path.realpath(srcfile))))
37 
38hunk ./src/allmydata/__init__.py 182
39 def get_package_versions_and_locations():
40-    # because there are a few dependencies that are outside setuptools's ken
41-    # (Python and platform, and sqlite3 if you are on Python >= 2.5), and
42-    # because setuptools might fail to find something even though import
43-    # finds it:
44-    import OpenSSL, allmydata, foolscap.api, nevow, platform, pycryptopp, setuptools, simplejson, twisted, zfec, zope.interface
45+    # pkg_resources.require returns the distribution that pkg_resources attempted to put
46+    # on sys.path, which can differ from the one that we actually import due to #1258,
47+    # or any other bug that causes sys.path to be set up incorrectly. Therefore we
48+    # must import the packages in order to check their versions and paths. Note that
49+    # this needs to be updated if dependencies are added.
50+
51+    import OpenSSL, allmydata, foolscap.api, nevow, platform, pycryptopp, setuptools, simplejson, twisted, zfec, zope.interface, Crypto
52     pysqlitever = None
53     pysqlitefile = None
54     sqlitever = None
55hunk ./src/allmydata/__init__.py 208
56         pysqlitefile = package_dir(sqlite3.__file__)
57         sqlitever = sqlite3.sqlite_version
58 
59-    d1 = {
60-        'pyOpenSSL': (OpenSSL.__version__, package_dir(OpenSSL.__file__)),
61+    return {
62+        'pyopenssl': (OpenSSL.__version__, package_dir(OpenSSL.__file__)),
63         __appname__: (allmydata.__version__, package_dir(allmydata.__file__)),
64         'foolscap': (foolscap.api.__version__, package_dir(foolscap.__file__)),
65         'Nevow': (nevow.__version__, package_dir(nevow.__file__)),
66hunk ./src/allmydata/__init__.py 221
67         'zope.interface': ('unknown', package_dir(zope.interface.__file__)),
68         'Twisted': (twisted.__version__, package_dir(twisted.__file__)),
69         'zfec': (zfec.__version__, package_dir(zfec.__file__)),
70+        'pycrypto': (Crypto.__version__, package_dir(Crypto.__file__)),
71         'python': (platform.python_version(), sys.executable),
72         'platform': (get_platform(), None),
73         }
74hunk ./src/allmydata/__init__.py 226
75 
76-    # But we prefer to get all the dependencies as known by setuptools:
77-    import pkg_resources
78-    try:
79-        d2 = get_package_versions_from_setuptools()
80-    except pkg_resources.DistributionNotFound:
81-        # See docstring in _auto_deps.require_auto_deps() to explain why it makes sense to ignore this exception.
82-        pass
83-    else:
84-        d1.update(d2)
85-
86-    return d1
87-
88 def get_package_versions():
89     return dict([(k, v) for k, (v, l) in get_package_versions_and_locations().iteritems()])
90 
91}
92[allmydata/_auto_deps.py: add a comment reminding to change __init__.py when dependencies are added or removed. Clarify a comment in __init__.py. refs #1287
93david-sarah@jacaranda.org**20110101200502
94 Ignore-this: edb7059211da0c2ce511680d76731da5
95] {
96hunk ./src/allmydata/__init__.py 186
97     # on sys.path, which can differ from the one that we actually import due to #1258,
98     # or any other bug that causes sys.path to be set up incorrectly. Therefore we
99     # must import the packages in order to check their versions and paths. Note that
100-    # this needs to be updated if dependencies are added.
101+    # this needs to be updated if dependencies are added or removed.
102 
103     import OpenSSL, allmydata, foolscap.api, nevow, platform, pycryptopp, setuptools, simplejson, twisted, zfec, zope.interface, Crypto
104     pysqlitever = None
105hunk ./src/allmydata/_auto_deps.py 8
106 # zetuptoolz, distribute, and perhaps in the future distutils2, for
107 # the require_auto_deps() function.)
108 
109+# If you add or remove dependencies, please remember to update
110+# get_package_versions_and_locations() in src/allmydata/__init__.py.
111+
112 install_requires=[
113                   # we require newer versions of setuptools (actually
114                   # zetuptoolz) to build, but can handle older versions to run
115}
116[bundled darcsver: write the _version.py file with LF line endings. This works around a problem with bb-freeze (refs #585). Don't apply this to trunk; instead, make a new darcsver release.
117david-sarah@jacaranda.org**20110104050151
118 Ignore-this: 3cb0bcf2af0dfa1d2cdc957a5aa42ab8
119] hunk ./darcsver-1.7.1.egg/darcsver/darcsvermodule.py 50
120 '''
121 
122 def write_version_py(verstr, outfname, EXE_NAME, version_body, pkgname):
123-    f = open(outfname, "wt+")
124+    f = open(outfname, "wb+")
125     f.write(version_body % {
126             'versiontool': EXE_NAME,
127             'pkgversion': verstr,
128[Make bbfreeze work. This also refactors allmydata/__init__.py and _auto_deps.py and implements more robust checking of dependency requirements, as discussed in ticket:585#comment:10 . However, the executable is not yet testable. refs #585
129david-sarah@jacaranda.org**20110104111825
130 Ignore-this: 60bfc2a64acf642db4f50c9d8c204b6a
131] {
132hunk ./src/allmydata/__init__.py 7
133 community web site: U{http://tahoe-lafs.org/}
134 """
135 
136-# We want to call require_auto_deps() before other imports, because the setuptools
137-# docs claim that if a distribution is installed with --multi-version, it might not
138-# be importable until after pkg_resources.require() has been called for it. We don't
139-# have an example of this happening at this time. It is possible that require() isn't
140-# actually needed because we set __requires__ in the generated startup script, but
141-# that would be an undocumented property of the setuptools implementation.
142-
143-from allmydata import _auto_deps
144-_auto_deps.require_auto_deps()
145-
146-# This is just to suppress DeprecationWarnings from nevow and twisted.
147-# See http://allmydata.org/trac/tahoe/ticket/859 and
148-# http://divmod.org/trac/ticket/2994 .
149-import warnings
150-warnings.filterwarnings("ignore", category=DeprecationWarning,
151-    message="the sha module is deprecated; use the hashlib module instead",
152-    append=True)
153-warnings.filterwarnings("ignore", category=DeprecationWarning,
154-    message="object.__new__\(\) takes no parameters",
155-    append=True)
156-warnings.filterwarnings("ignore", category=DeprecationWarning,
157-    message="The popen2 module is deprecated.  Use the subprocess module.",
158-    append=True)
159-warnings.filterwarnings("ignore", category=DeprecationWarning,
160-    message="the md5 module is deprecated; use hashlib instead",
161-    append=True)
162-warnings.filterwarnings("ignore", category=DeprecationWarning,
163-    message="twisted.web.error.NoResource is deprecated since Twisted 9.0.  See twisted.web.resource.NoResource.",
164-    append=True)
165-try:
166-    import nevow
167-    from twisted.persisted import sob
168-    from twisted.python import filepath
169-    hush_pyflakes = (nevow, sob, filepath)
170-    del hush_pyflakes
171-finally:
172-    warnings.filters.pop()
173-    warnings.filters.pop()
174-    warnings.filters.pop()
175-    warnings.filters.pop()
176-    # Don't pop the filter for the sha module warning because it is also generated
177-    # by pycrypto (which we don't want to import unless needed).
178-    # warnings.filters.pop()
179-
180-# This warning is generated by twisted, PyRex, and possibly other packages,
181-# but can happen at any time, not only when they are imported. See
182-# http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1129 .
183-warnings.filterwarnings("ignore", category=DeprecationWarning,
184-    message="BaseException.message has been deprecated as of Python 2.6",
185-    append=True)
186-
187 __version__ = "unknown"
188 try:
189     from allmydata._version import __version__
190hunk ./src/allmydata/__init__.py 127
191     else:
192         return platform.platform()
193 
194+
195+from allmydata._auto_deps import get_package_versions_and_locations
196+
197+_vers_and_locs = get_package_versions_and_locations()
198+_vers_and_locs['platform'] = (get_platform(), None)
199+
200+
201+def check_all_requirements():
202+    import platform
203+    from distutils.version import LooseVersion
204+    from allmydata._auto_deps import install_requires
205+
206+    # we require 2.4.4 on non-UCS-2, non-Redhat builds to avoid <http://www.python.org/news/security/PSF-2006-001/>
207+    # we require 2.4.3 on non-UCS-2 Redhat, because 2.4.3 is common on Redhat-based distros and will have patched the above bug
208+    # we require at least 2.4.2 in any case to avoid a bug in the base64 module: <http://bugs.python.org/issue1171487>
209+    if sys.maxunicode == 65535:
210+        if sys.version_info < (2, 4, 2) or sys.version_info[0] > 2:
211+            raise NotImplementedError("Tahoe-LAFS current requires Python v2.4.2 or greater "
212+                                      "for a UCS-2 build (but less than v3), not %r" %
213+                                      (sys.version_info,))
214+    elif platform.platform().lower().find('redhat') >= 0:
215+        if sys.version_info < (2, 4, 3) or sys.version_info[0] > 2:
216+            raise NotImplementedError("Tahoe-LAFS current requires Python v2.4.3 or greater "
217+                                      "on Redhat-based distributions (but less than v3), not %r" %
218+                                      (sys.version_info,))
219+    else:
220+        if sys.version_info < (2, 4, 4) or sys.version_info[0] > 2:
221+            raise NotImplementedError("Tahoe-LAFS current requires Python v2.4.4 or greater "
222+                                      "for a non-UCS-2 build (but less than v3), not %r" %
223+                                      (sys.version_info,))
224+
225+    def check_requirement(req):
226+        # We only support a single >= or ==.
227+        def _check(s, ok, relation):
228+            name = s[0].strip(' ').partition('[')[0]
229+            required = s[1].strip(' ')
230+            if name not in _vers_and_locs:
231+                raise AssertionError("no version info for %s" % (name,))
232+            (actual, location) = _vers_and_locs[name]
233+            actual = str(actual)
234+            if actual != 'unknown' and not ok(LooseVersion(actual), LooseVersion(required)):
235+                msg = ("We require %s version %s of %s, but could only find version %s.\n"
236+                      % (relation, required, name, actual))
237+                if location and location != 'unknown':
238+                    msg += "The version we found is from %r.\n" % (location,)
239+                msg += ("To resolve this problem, uninstall that version, either using your\n"
240+                        "operating system's package manager or by moving aside the directory.")
241+                raise AssertionError(msg)
242+
243+        s = req.split('>=')
244+        if len(s) == 2:
245+            _check(s, lambda x, y: x >= y, "at least")
246+            return
247+        s = req.split('==')
248+        if len(s) == 2:
249+            _check(s, lambda x, y: x == y, "exactly")
250+            return
251+        if req not in _vers_and_locs:
252+            raise AssertionError("no version info or could not understand requirement for %r" % (req))
253+
254+    for requirement in install_requires:
255+        check_requirement(requirement)
256+
257+check_all_requirements()
258+
259+
260 def package_dir(srcfile):
261     return os.path.dirname(os.path.dirname(os.path.normcase(os.path.realpath(srcfile))))
262 
263hunk ./src/allmydata/__init__.py 196
264-def get_package_versions_and_locations():
265+def old_get_package_versions_and_locations():
266     # pkg_resources.require returns the distribution that pkg_resources attempted to put
267     # on sys.path, which can differ from the one that we actually import due to #1258,
268     # or any other bug that causes sys.path to be set up incorrectly. Therefore we
269hunk ./src/allmydata/__init__.py 242
270         }
271 
272 def get_package_versions():
273-    return dict([(k, v) for k, (v, l) in get_package_versions_and_locations().iteritems()])
274+    return dict([(k, v) for k, (v, l) in _vers_and_locs.iteritems()])
275 
276 def get_package_locations():
277hunk ./src/allmydata/__init__.py 245
278-    return dict([(k, l) for k, (v, l) in get_package_versions_and_locations().iteritems()])
279+    return dict([(k, l) for k, (v, l) in _vers_and_locs.iteritems()])
280 
281 def get_package_versions_string(show_paths=False):
282hunk ./src/allmydata/__init__.py 248
283-    vers_and_locs = get_package_versions_and_locations()
284     res = []
285hunk ./src/allmydata/__init__.py 249
286-    for p in [__appname__, "foolscap", "pycryptopp", "zfec", "Twisted", "Nevow", "zope.interface", "python", "platform"]:
287-        (ver, loc) = vers_and_locs.get(p, ('UNKNOWN', 'UNKNOWN'))
288+    first = [__appname__, "foolscap", "pycryptopp", "zfec", "Twisted", "Nevow", "zope.interface", "python", "platform"]
289+    for p in first:
290+        (ver, loc) = _vers_and_locs.get(p, ('UNKNOWN', 'UNKNOWN'))
291         info = str(p) + ": " + str(ver)
292         if show_paths:
293             info = info + " (%s)" % str(loc)
294hunk ./src/allmydata/__init__.py 256
295         res.append(info)
296-        if vers_and_locs.has_key(p):
297-            del vers_and_locs[p]
298 
299hunk ./src/allmydata/__init__.py 257
300-    for p, (v, loc) in vers_and_locs.iteritems():
301-        info = str(p) + ": " + str(v)
302-        if show_paths:
303-            info = info + " (%s)" % str(loc)
304-        res.append(info)
305+    for p, (v, loc) in _vers_and_locs.iteritems():
306+        if p not in first:
307+            info = str(p) + ": " + str(v)
308+            if show_paths:
309+                info = info + " (%s)" % str(loc)
310+            res.append(info)
311+
312     return ', '.join(res)
313hunk ./src/allmydata/_auto_deps.py 3
314 # Note: do not import any module from Tahoe-LAFS itself in this
315 # file. Also please avoid importing modules from other packages than
316-# the Python Standard Library if at all possible (exception: we rely
317-# on importing pkg_resources, which is provided by setuptools,
318-# zetuptoolz, distribute, and perhaps in the future distutils2, for
319-# the require_auto_deps() function.)
320+# the Python Standard Library if at all possible. That includes
321+# setuptools and pkg_resources.
322 
323 # If you add or remove dependencies, please remember to update
324hunk ./src/allmydata/_auto_deps.py 7
325-# get_package_versions_and_locations() in src/allmydata/__init__.py.
326+# get_package_versions_and_locations() below.
327 
328 install_requires=[
329                   # we require newer versions of setuptools (actually
330hunk ./src/allmydata/_auto_deps.py 31
331 
332                   # Needed for SFTP. pyasn1 is needed by twisted.conch in Twisted >= 9.0.
333                   # pycrypto 2.2 doesn't work due to https://bugs.launchpad.net/pycrypto/+bug/620253
334-                  "pycrypto == 2.0.1, == 2.1, >= 2.3",
335+                  "pycrypto >= 2.3",
336                   "pyasn1 >= 0.0.8a",
337 
338                   # Will be needed to test web apps, but not yet. See #1001.
339hunk ./src/allmydata/_auto_deps.py 78
340     install_requires=[]
341 del sys # clean up namespace
342 
343-def require_python_version():
344+def old_require_python_version():
345     import sys, platform
346 
347     # we require 2.4.4 on non-UCS-2, non-Redhat builds to avoid <http://www.python.org/news/security/PSF-2006-001/>
348hunk ./src/allmydata/_auto_deps.py 100
349                                       "for a non-UCS-2 build (but less than v3), not %r" %
350                                       (sys.version_info,))
351 
352-def require_auto_deps():
353+def old_require_auto_deps():
354     """
355     The purpose of this function is to raise a pkg_resources exception if any of the
356     requirements can't be imported.  This is just to give earlier and more explicit error
357hunk ./src/allmydata/_auto_deps.py 124
358             # but it shows a too-old version, then we'll get a
359             # VersionConflict error instead of DistributionNotFound.
360             pass
361+
362+
363+def get_package_versions_and_locations():
364+    import warnings, os, sys
365+    def package_dir(srcfile):
366+        return os.path.dirname(os.path.dirname(os.path.normcase(os.path.realpath(srcfile))))
367+
368+    # pkg_resources.require returns the distribution that pkg_resources attempted to put
369+    # on sys.path, which can differ from the one that we actually import due to #1258,
370+    # or any other bug that causes sys.path to be set up incorrectly. Therefore we
371+    # must import the packages in order to check their versions and paths. Note that
372+    # this needs to be updated if dependencies are added or removed.
373+
374+    # This warning is generated by twisted, PyRex, and possibly other packages,
375+    # but can happen at any time, not only when they are imported. See
376+    # http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1129 .
377+    warnings.filterwarnings("ignore", category=DeprecationWarning,
378+        message="BaseException.message has been deprecated as of Python 2.6",
379+        append=True)
380+
381+    # This is to suppress DeprecationWarnings from nevow, twisted, and pycrypto.
382+    # See http://allmydata.org/trac/tahoe/ticket/859 and
383+    # http://divmod.org/trac/ticket/2994 .
384+    warnings.filterwarnings("ignore", category=DeprecationWarning,
385+        message="the sha module is deprecated; use the hashlib module instead",
386+        append=True)
387+    warnings.filterwarnings("ignore", category=DeprecationWarning,
388+        message="object.__new__\(\) takes no parameters",
389+        append=True)
390+    warnings.filterwarnings("ignore", category=DeprecationWarning,
391+        message="The popen2 module is deprecated.  Use the subprocess module.",
392+        append=True)
393+    warnings.filterwarnings("ignore", category=DeprecationWarning,
394+        message="the md5 module is deprecated; use hashlib instead",
395+        append=True)
396+    warnings.filterwarnings("ignore", category=DeprecationWarning,
397+        message="twisted.web.error.NoResource is deprecated since Twisted 9.0.  See twisted.web.resource.NoResource.",
398+        append=True)
399+    try:
400+        import nevow
401+        from twisted.persisted import sob
402+        from twisted.python import filepath
403+        import Crypto
404+        [sob, filepath]  # hush pyflakes
405+    finally:
406+        warnings.filters.pop()
407+        warnings.filters.pop()
408+        warnings.filters.pop()
409+        warnings.filters.pop()
410+        warnings.filters.pop()
411+
412+    import OpenSSL, allmydata, foolscap.api, platform, pycryptopp, simplejson, twisted, zfec, zope.interface, pyasn1
413+    pysqlitever = None
414+    pysqlitefile = None
415+    sqlitever = None
416+    try:
417+        import sqlite3
418+    except ImportError:
419+        try:
420+            from pysqlite2 import dbapi2
421+        except ImportError:
422+            pass
423+        else:
424+            pysqlitever = dbapi2.version
425+            pysqlitefile = package_dir(dbapi2.__file__)
426+            sqlitever = dbapi2.sqlite_version
427+    else:
428+        pysqlitever = sqlite3.version
429+        pysqlitefile = package_dir(sqlite3.__file__)
430+        sqlitever = sqlite3.sqlite_version
431+
432+    packages = {
433+        'pyopenssl': (OpenSSL.__version__, package_dir(OpenSSL.__file__)),
434+        allmydata.__appname__: (allmydata.__version__, package_dir(allmydata.__file__)),
435+        'foolscap': (foolscap.api.__version__, package_dir(foolscap.__file__)),
436+        'Nevow': (nevow.__version__, package_dir(nevow.__file__)),
437+        'pycryptopp': (pycryptopp.__version__, package_dir(pycryptopp.__file__)),
438+        'simplejson': (simplejson.__version__, package_dir(simplejson.__file__)),
439+        'pysqlite': (pysqlitever, pysqlitefile),
440+        'sqlite': (sqlitever, 'unknown'),
441+        'zope.interface': ('unknown', package_dir(zope.interface.__file__)),
442+        'Twisted': (twisted.__version__, package_dir(twisted.__file__)),
443+        'zfec': (zfec.__version__, package_dir(zfec.__file__)),
444+        'pycrypto': (Crypto.__version__, package_dir(Crypto.__file__)),
445+        'pyasn1': ('unknown', package_dir(pyasn1.__file__)),
446+        'python': (platform.python_version(), sys.executable),
447+        }
448+
449+    if sys.platform == 'win32':
450+        try:
451+            import win32api
452+            packages['pywin32'] = ('unknown', package_dir(win32api.__file__))
453+        except ImportError:
454+            pass
455+
456+    if not hasattr(sys, 'frozen'):
457+        # Don't try to get the setuptools version if using bbfreeze, py2exe etc. See #585.
458+        import setuptools
459+        packages['setuptools'] = (setuptools.__version__, package_dir(setuptools.__file__))
460+
461+    return packages
462hunk ./src/allmydata/windows/fixups.py 172
463     argc = c_int(0)
464     argv_unicode = CommandLineToArgvW(GetCommandLineW(), byref(argc))
465 
466-    # Because of <http://bugs.python.org/issue8775> (and similar limitations in
467-    # twisted), the 'bin/tahoe' script cannot invoke us with the actual Unicode arguments.
468-    # Instead it "mangles" or escapes them using \x7F as an escape character, which we
469-    # unescape here.
470-    def unmangle(s):
471-        return re.sub(ur'\x7F[0-9a-fA-F]*\;', lambda m: unichr(int(m.group(0)[1:-1], 16)), s)
472+    if hasattr(sys, 'frozen'):
473+        # If this is an executable produced by py2exe or bbfreeze, then it will have been
474+        # invoked directly, so the arguments just need to be encoded as UTF-8.
475 
476hunk ./src/allmydata/windows/fixups.py 176
477-    try:
478-        argv = [unmangle(argv_unicode[i]).encode('utf-8') for i in xrange(1, argc.value)]
479-    except Exception, e:
480-        _complain("%s:  could not unmangle Unicode arguments.\n%r"
481-                  % (sys.argv[0], [argv_unicode[i] for i in xrange(1, argc.value)]))
482-        raise
483+        argv = [argv_unicode[i].encode('utf-8') for i in xrange(0, argc.value)]
484+    else:
485+        # Because of <http://bugs.python.org/issue8775> (and similar limitations in
486+        # twisted), the 'bin/tahoe' script cannot invoke us with the actual Unicode arguments.
487+        # Instead it "mangles" or escapes them using \x7F as an escape character, which we
488+        # unescape here.
489+        def unmangle(s):
490+            return re.sub(ur'\x7F[0-9a-fA-F]*\;', lambda m: unichr(int(m.group(0)[1:-1], 16)), s)
491 
492hunk ./src/allmydata/windows/fixups.py 185
493-    # Skip option arguments to the Python interpreter.
494-    while len(argv) > 0:
495-        arg = argv[0]
496-        if not arg.startswith(u"-") or arg == u"-":
497-            if arg.endswith('.pyscript'):
498-                argv[0] = arg[:-9]
499-            break
500-        argv = argv[1:]
501-        if arg == u'-m':
502-            # sys.argv[0] should really be the absolute path of the module source, but never mind
503-            break
504-        if arg == u'-c':
505-            argv[0] = u'-c'
506-            break
507+        # unicode_argv[0] is the Python interpreter, so skip that, and unmangle subsequent
508+        # arguments.
509+        try:
510+            argv = [unmangle(argv_unicode[i]).encode('utf-8') for i in xrange(1, argc.value)]
511+        except Exception, e:
512+            _complain("%s:  could not unmangle Unicode arguments.\n%r"
513+                      % (sys.argv[0], [argv_unicode[i] for i in xrange(1, argc.value)]))
514+            raise
515+
516+        # Also skip option arguments to the Python interpreter.
517+        while len(argv) > 0:
518+            arg = argv[0]
519+            if not arg.startswith(u"-") or arg == u"-":
520+                if arg.endswith('.pyscript'):
521+                    argv[0] = arg[:-9]
522+                break
523+            argv = argv[1:]
524+            if arg == u'-m':
525+                # sys.argv[0] should really be the absolute path of the module source, but never mind
526+                break
527+            if arg == u'-c':
528+                argv[0] = u'-c'
529+                break
530 
531     sys.argv = argv
532}
533[Delete obsolete code due to refactoring of allmydata/__init__.py and _auto_deps.py (done in a separate patch to avoid a confusing diff). refs #585
534david-sarah@jacaranda.org**20110104112300
535 Ignore-this: 2f607f6e53ee0ba278f43a7ab3396edd
536] {
537hunk ./src/allmydata/__init__.py 193
538 check_all_requirements()
539 
540 
541-def package_dir(srcfile):
542-    return os.path.dirname(os.path.dirname(os.path.normcase(os.path.realpath(srcfile))))
543-
544-def old_get_package_versions_and_locations():
545-    # pkg_resources.require returns the distribution that pkg_resources attempted to put
546-    # on sys.path, which can differ from the one that we actually import due to #1258,
547-    # or any other bug that causes sys.path to be set up incorrectly. Therefore we
548-    # must import the packages in order to check their versions and paths. Note that
549-    # this needs to be updated if dependencies are added or removed.
550-
551-    import OpenSSL, allmydata, foolscap.api, nevow, platform, pycryptopp, setuptools, simplejson, twisted, zfec, zope.interface, Crypto
552-    pysqlitever = None
553-    pysqlitefile = None
554-    sqlitever = None
555-    try:
556-        import sqlite3
557-    except ImportError:
558-        try:
559-            from pysqlite2 import dbapi2
560-        except ImportError:
561-            pass
562-        else:
563-            pysqlitever = dbapi2.version
564-            pysqlitefile = package_dir(dbapi2.__file__)
565-            sqlitever = dbapi2.sqlite_version
566-    else:
567-        pysqlitever = sqlite3.version
568-        pysqlitefile = package_dir(sqlite3.__file__)
569-        sqlitever = sqlite3.sqlite_version
570-
571-    return {
572-        'pyopenssl': (OpenSSL.__version__, package_dir(OpenSSL.__file__)),
573-        __appname__: (allmydata.__version__, package_dir(allmydata.__file__)),
574-        'foolscap': (foolscap.api.__version__, package_dir(foolscap.__file__)),
575-        'Nevow': (nevow.__version__, package_dir(nevow.__file__)),
576-        'pycryptopp': (pycryptopp.__version__, package_dir(pycryptopp.__file__)),
577-        'setuptools': (setuptools.__version__, package_dir(setuptools.__file__)),
578-        'simplejson': (simplejson.__version__, package_dir(simplejson.__file__)),
579-        'pysqlite': (pysqlitever, pysqlitefile),
580-        'sqlite': (sqlitever, 'unknown'),
581-        'zope.interface': ('unknown', package_dir(zope.interface.__file__)),
582-        'Twisted': (twisted.__version__, package_dir(twisted.__file__)),
583-        'zfec': (zfec.__version__, package_dir(zfec.__file__)),
584-        'pycrypto': (Crypto.__version__, package_dir(Crypto.__file__)),
585-        'python': (platform.python_version(), sys.executable),
586-        'platform': (get_platform(), None),
587-        }
588-
589 def get_package_versions():
590     return dict([(k, v) for k, (v, l) in _vers_and_locs.iteritems()])
591 
592hunk ./src/allmydata/_auto_deps.py 78
593     install_requires=[]
594 del sys # clean up namespace
595 
596-def old_require_python_version():
597-    import sys, platform
598-
599-    # we require 2.4.4 on non-UCS-2, non-Redhat builds to avoid <http://www.python.org/news/security/PSF-2006-001/>
600-    # we require 2.4.3 on non-UCS-2 Redhat, because 2.4.3 is common on Redhat-based distros and will have patched the above bug
601-    # we require at least 2.4.2 in any case to avoid a bug in the base64 module: <http://bugs.python.org/issue1171487>
602-    if sys.maxunicode == 65535:
603-        if sys.version_info < (2, 4, 2) or sys.version_info[0] > 2:
604-            raise NotImplementedError("Tahoe-LAFS current requires Python v2.4.2 or greater "
605-                                      "for a UCS-2 build (but less than v3), not %r" %
606-                                      (sys.version_info,))
607-    elif platform.platform().lower().find('redhat') >= 0:
608-        if sys.version_info < (2, 4, 3) or sys.version_info[0] > 2:
609-            raise NotImplementedError("Tahoe-LAFS current requires Python v2.4.3 or greater "
610-                                      "on Redhat-based distributions (but less than v3), not %r" %
611-                                      (sys.version_info,))
612-    else:
613-        if sys.version_info < (2, 4, 4) or sys.version_info[0] > 2:
614-            raise NotImplementedError("Tahoe-LAFS current requires Python v2.4.4 or greater "
615-                                      "for a non-UCS-2 build (but less than v3), not %r" %
616-                                      (sys.version_info,))
617-
618-def old_require_auto_deps():
619-    """
620-    The purpose of this function is to raise a pkg_resources exception if any of the
621-    requirements can't be imported.  This is just to give earlier and more explicit error
622-    messages, as opposed to waiting until the source code tries to import some module from one
623-    of these packages and gets an ImportError.  This function gets called from
624-    src/allmydata/__init__.py .
625-    """
626-    require_python_version()
627-
628-    import pkg_resources
629-    for requirement in install_requires:
630-        try:
631-            pkg_resources.require(requirement)
632-        except pkg_resources.DistributionNotFound:
633-            # there is no .egg-info present for this requirement, which
634-            # either means that it isn't installed, or it is installed in a
635-            # way that pkg_resources can't find it (but regular python
636-            # might).  There are several older Linux distributions which
637-            # provide our dependencies just fine, but they don't ship
638-            # .egg-info files. Note that if there *is* an .egg-info file,
639-            # but it shows a too-old version, then we'll get a
640-            # VersionConflict error instead of DistributionNotFound.
641-            pass
642-
643 
644 def get_package_versions_and_locations():
645     import warnings, os, sys
646}
647[Add static/tahoe.py, to support bbfreeze, py2exe etc. refs #585
648david-sarah@jacaranda.org**20110104112441
649 Ignore-this: cc749f837d24c23979b740fd8a27a8f0
650] {
651adddir ./static
652addfile ./static/tahoe.py
653hunk ./static/tahoe.py 2
654 
655+# This checks that we can import the right versions of all dependencies.
656+# Import this first to suppress deprecation warnings.
657+import allmydata
658+
659+# nevow requires all these for its voodoo module import time adaptor registrations
660+from nevow import accessors, appserver, static, rend, url, util, query, i18n, flat
661+from nevow import guard, stan, testutil, context
662+from nevow.flat import flatmdom, flatstan, twist
663+from formless import webform, processors, annotate, iformless
664+from decimal import Decimal
665+
666+import allmydata.web
667+
668+# junk to appease pyflakes's outrage
669+[
670+    accessors, appserver, static, rend, url, util, query, i18n, flat, guard, stan, testutil,
671+    context, flatmdom, flatstan, twist, webform, processors, annotate, iformless, Decimal,
672+    allmydata,
673+]
674+
675+from allmydata.scripts import runner
676+
677+runner.run()
678}
679
680Context:
681
682[bin/tahoe-script.template: On non-Windows, invoke support/bin/tahoe directly as a script (rather than via python), so that 'top' for example will show it as 'tahoe'. On Windows, simplify some code that set argv[0], which is never used. fixes #174
683david-sarah@jacaranda.org**20101127232650
684 Ignore-this: 42a86f3eecfdc1ea7b76a7cc68626898
685]
686[test_runner: avoid unnecessary use of non-ASCII.
687david-sarah@jacaranda.org**20110101100101
688 Ignore-this: e2ff40dce6bb3b021306f2913d4e75df
689]
690[docs/quickstart.html: fix redundant, badly nested tag. refs #1284
691david-sarah@jacaranda.org**20110102175159
692 Ignore-this: 2ae9cc0b47d2e87b9eb64a0f517c4eef
693]
694[docs/quickstart.html: information about 'troublesome dependencies' and 'verified systems' de-emphasized by smaller italic font. Re-wrap so that the HTML source is readable (just about) as text. Minor wording tweaks. Improve organization by adding 'Windows Caveats' subsection. fixes #1284
695david-sarah@jacaranda.org**20110102174212
696 Ignore-this: e9dc57983974478200856651c5318fee
697]
698[NEWS: update entry for removal of Mac and Windows apps. refs #1282
699david-sarah@jacaranda.org**20101226042245
700 Ignore-this: c8099bc6e8235718d042c9a13c1e2425
701]
702[Move dependency imports from windows/depends.py (which has gone away) into src/allmydata/windows/tahoesvc.py. Also fix a pyflakes warning, and change the service display name from 'Allmydata Tahoe Node' to 'Tahoe-LAFS node'. refs #1282
703david-sarah@jacaranda.org**20101226042100
704 Ignore-this: ee45f324934e1251380206dbee6346d0
705]
706[Remove unmaintained Windows GUI app, except for windows/tahoesvc.py which is moved to src/allmydata/windows. refs #1282
707david-sarah@jacaranda.org**20101226040237
708 Ignore-this: cae37b6622a7dd5940acc7d3e6a98b90
709]
710[Remove the Makefile targets relating to the Mac GUI app. refs #1282
711david-sarah@jacaranda.org**20101226025859
712 Ignore-this: 75303be783974b41138744ec62b07965
713]
714[NEWS: remove unmaintained Mac GUI app. refs #1282
715david-sarah@jacaranda.org**20101226020858
716 Ignore-this: 40474a07f4a550b48563d35350be7ab5
717]
718[Remove unmaintained Mac GUI app. fixes #1282
719david-sarah@jacaranda.org**20101226020508
720 Ignore-this: b3613bf1abfd284d542bf7c753ec557a
721]
722[Remove src/allmydata/util/find_exe.py which is no longer used. fixes #1150
723david-sarah@jacaranda.org**20101226023206
724 Ignore-this: 7436c9b53bf210aed34a1a973cd9cace
725]
726[status_web_pages_review.darcs.patch
727freestorm77@gmail.com**20110102034214
728 Ignore-this: 29f1ecb36177f10f3f846b3d56b313b2
729 
730 I make some changes on status web pages
731 
732 status.xhtml:
733 - Delete unused webform_css link
734 - Align tables on the left
735 
736 tahoe-css:
737 - Do some minor changes on code synthax
738 - changes table.status-download-events style to look like other tables
739 
740 status.py:
741 - Align table on the left
742 - Changes table header
743 - Add heading tags
744 - Modify google api graph: add image border, calculate height to feet data
745 
746 signed-off-by: zooko@zooko.com
747 fixes #1219
748]
749[test_storage.py: fix a pyflakes unused import warning.
750david-sarah@jacaranda.org**20101231220756
751 Ignore-this: df08231540cb7dff9d2b038e47ab30ee
752]
753[test_storage.py: leave at least 512 MiB free when running test_large_share. refs #1195
754david-sarah@jacaranda.org**20101231203215
755 Ignore-this: b2144c0341c3452b5d4ba219e284ea0e
756]
757[storage: use fileutil's version of get_disk_stats() and get_available_space(), use mockery/fakery in tests, enable large share test on platforms with sparse files and if > 4 GiB of disk space is currently available
758zooko@zooko.com**20100910173629
759 Ignore-this: 1304f1164c661de6d5304f993eb9b27b
760]
761[fileutil: copy in the get_disk_stats() and get_available_space() functions from storage/server.py
762zooko@zooko.com**20100910173520
763 Ignore-this: 8b15569715f710f4fc5092f7ca109253
764]
765[Update foolscap version requirement to 0.6.0, to address http://foolscap.lothar.com/trac/ticket/167
766david-sarah@jacaranda.org**20101231060039
767 Ignore-this: 98d2b8086a1a500b9f4565bca5a3810
768]
769[docs/webapi.rst: typos.
770david-sarah@jacaranda.org**20101230034422
771 Ignore-this: d1f5166d72cc711f7e0d9981eac9105e
772]
773[docs/webapi.rst: capitalization, formatting of section on URL character encoding, and a correction about Internet Explorer.
774david-sarah@jacaranda.org**20101230034049
775 Ignore-this: b3b9819d2fb264b4cdc5c8afd4e8c48d
776]
777[docs: corrections and clarifications.
778david-sarah@jacaranda.org**20101227051056
779 Ignore-this: e33202858c7644c58f3f924b164294b6
780]
781[docs: more formatting cleanups and corrections. Spell webapi and wapi as web-API.
782david-sarah@jacaranda.org**20101227050533
783 Ignore-this: 18b23cbfb780df585d8a722a1ec63e94
784]
785[docs/debian.rst: bring description of building dependencies from source up-to-date, and change hostname from allmydata.com to tahoe-lafs.org.
786david-sarah@jacaranda.org**20101212222912
787 Ignore-this: f38462afc88b4475195610385a28391c
788]
789[docs/architecture.rst: correct rst syntax.
790david-sarah@jacaranda.org**20101212202003
791 Ignore-this: 3fbe12feb28bec6f1c63aedbc79aad21
792]
793[docs/architecture.rst: formatting.
794david-sarah@jacaranda.org**20101212201719
795 Ignore-this: 305fa5dfc2939355eaf6d0d2161eb1ff
796]
797[docs: linkification, wording improvements.
798david-sarah@jacaranda.org**20101212201234
799 Ignore-this: 4e67287f527a8bc728cfbd93255d2aae
800]
801[docs: formatting.
802david-sarah@jacaranda.org**20101212201115
803 Ignore-this: 2e0ed394ac7726651d3a4f2c4b0d3798
804]
805[docs/configuration.rst: more formatting tweaks; which -> that.
806david-sarah@jacaranda.org**20101212195522
807 Ignore-this: a7becb7021854ca5a90edd892b36fdd7
808]
809[docs/configuration.rst: more changes to formatting.
810david-sarah@jacaranda.org**20101212194511
811 Ignore-this: 491aac33e5f5268d224359f1447d10be
812]
813[docs/configuration.rst: changes to formatting (mainly putting commands and filenames in monospace).
814david-sarah@jacaranda.org**20101212181828
815 Ignore-this: 8a1480e2d5f43bee678476424615b50f
816]
817[scripts/backupdb.py: more accurate comment about path field.
818david-sarah@jacaranda.org**20101212170320
819 Ignore-this: 50e47a2228a85207bbcd188a78a0d4e6
820]
821[scripts/cli.py: fix missing 'put' in usage example for 'tahoe put'.
822david-sarah@jacaranda.org**20101212170207
823 Ignore-this: 2cbadf066fff611fc03d3c0ff97ce6ec
824]
825[docs/frontends/CLI.rst: changes to formatting (mainly putting commands and filenames in monospace), and to command syntax to reflect that DIRCAP/... is accepted. Clarify the syntax of 'tahoe put' and other minor corrections. Tahoe -> Tahoe-LAFS.
826david-sarah@jacaranda.org**20101212165800
827 Ignore-this: a123ef6b564aa8624d1e79c97068ea12
828]
829[docs/frontends/CLI.rst: Unicode arguments to 'tahoe' work on Windows as of v1.7.1.
830david-sarah@jacaranda.org**20101212063740
831 Ignore-this: 3977a99dfa86ac33a44171deaf43aaab
832]
833[docs/known_issues.rst: fix title and linkify another URL. refs #1225
834david-sarah@jacaranda.org**20101212062817
835 Ignore-this: cc91287f7fb51c23440b3d2fe79c449c
836]
837[docs/known_issues.rst: fix an external link. refs #1225
838david-sarah@jacaranda.org**20101212062435
839 Ignore-this: b8cbf12f353131756c358965c48060ec
840]
841[Fix a link from uri.rst to dirnodes.rst. refs #1225
842david-sarah@jacaranda.org**20101212054502
843 Ignore-this: af6205299f5c9a33229cab259c00f9d5
844]
845[Fix a link from webapi.rst to FTP-and-SFTP.rst. refs #1225
846david-sarah@jacaranda.org**20101212053435
847 Ignore-this: 2b9f88678c3447ea860d6b61e8799858
848]
849[More specific hyperlink to architecture.rst from helper.rst. refs #1225
850david-sarah@jacaranda.org**20101212052607
851 Ignore-this: 50424c768fca481252fabf58424852dc
852]
853[Update hyperlinks between docs, and linkify some external references. refs #1225
854david-sarah@jacaranda.org**20101212051459
855 Ignore-this: cd43a4c3d3de1f832abfa88d5fc4ace1
856]
857[docs/specifications/dirnodes.rst: fix references to mutable.rst. refs #1225
858david-sarah@jacaranda.org**20101212012720
859 Ignore-this: 6819b4b4e06e947ee48b365e840db37d
860]
861[docs/specifications/mutable.rst: correct the magic string for v1 mutable containers. refs #1225
862david-sarah@jacaranda.org**20101212011400
863 Ignore-this: 99a5fcdd40cef83dbb08f323f6cdaaca
864]
865[Move .txt files in docs/frontends and docs/specifications to .rst. refs #1225
866david-sarah@jacaranda.org**20101212010251
867 Ignore-this: 8796d35d928370f7dc6ad2dafdc1c0fe
868]
869[Convert docs/frontends and docs/specifications to reStructuredText format (not including file moves).
870david-sarah@jacaranda.org**20101212004632
871 Ignore-this: e3ceb2d832d73875abe48624ddbb5622
872]
873[scripts/cli.py: remove the disclaimer in the help for 'tahoe cp' that it does not handle non-ASCII filenames well. (At least, we intend to handle them.)
874david-sarah@jacaranda.org**20101130002145
875 Ignore-this: 94c003efaa20b9eb4a83503d79844ca
876]
877[relnotes.txt: fifth -> sixth labor-of-love release
878zooko@zooko.com**20101129045647
879 Ignore-this: 21c245015268b38916e3a138d256c09d
880]
881[Makefile: BB_BRANCH is set to the empty string for trunk, not the string 'trunk'.
882david-sarah@jacaranda.org**20101128233512
883 Ignore-this: 5a7ef8eb10475636d21b91e25b56c369
884]
885[relnotes.txt: eleventh -> twelfth release.
886david-sarah@jacaranda.org**20101128223321
887 Ignore-this: 1e26410156a665271c1170803dea2c0d
888]
889[relnotes.tst: point to known_issues.rst, not known_issues.txt.
890david-sarah@jacaranda.org**20101128222918
891 Ignore-this: 60194eb4544cac446fe4f60b3e34b887
892]
893[quickstart.html: fix link to point to allmydata-tahoe-1.8.1.zip.
894david-sarah@jacaranda.org**20101128221728
895 Ignore-this: 7b3ee86f8256aa12f5d862f689f3ee29
896]
897[TAG allmydata-tahoe-1.8.1
898david-sarah@jacaranda.org**20101128212336
899 Ignore-this: 9c18bdeaef4822f590d2a0d879e00621
900]
901Patch bundle hash:
90205e40d32c3646801698dd18a95af40dc02442706