[tahoe-lafs-trac-stream] [tahoe-lafs] #1355: bug in error path of cross_check_pkg_resources_versus_import (was: py2.7 incompatibility in cross_check_pkg_resources_versus_import?)

[tahoe-lafs]

#1355: bug in error path of cross_check_pkg_resources_versus_import
--------------------------+-------------------------------------------------
     Reporter:  warner    |       Owner:  somebody
         Type:  defect    |      Status:  new     
     Priority:  critical  |   Milestone:  1.9.0   
    Component:  code      |     Version:  1.8.2   
   Resolution:            |    Keywords:          
Launchpad Bug:            |  
--------------------------+-------------------------------------------------

Old description:

> Jimmy Tang [http://tahoe-lafs.org/pipermail/tahoe-
> dev/2011-February/006074.html wrote] to the mailing list with a crash on
> his Archlinux box. I suspect it's a python-2.7 incompatibility in the
> error-handling path in all the crazy what-version-of-each-dependency code
> inside {{{__init__.py}}}:
>
> {{{
>   File "/home/jtang/tmp/tahoe-lafs/src/allmydata-
> tahoe-1.8.2/src/allmydata/__init__.py",
> line 301, in cross_check_pkg_resources_versus_import
>     % (imp_ver, name, imp_loc, pr_ver, pr_loc, e.__class__.name, e))
> AttributeError: type object 'exceptions.TypeError' has no attribute
> 'name'
>     Aborting...
> }}}
>
> I suspect that {{{TypeError}}} moved from being implemented in Python (in
> py2.6) to being implemented in C (in py2.7), and that for some reason
> when you try to do the {{{.name}}} lookup, it explodes messily.
>
> I don't know if this is a real error path, or if it's just accumulating a
> bunch of warnings to print to stderr (i.e. if we just took out the
> offending line, would the program work anyways, or is the missing/old
> /unable-to-compute-version dependency actually important?).
>
> This sort of problem reinforces my feeling that our {{{__init__.py}}} is
> way too complex, and that all the version-checking logic may be doing
> more harm than good.
>
> This may provoke a 1.8.3, if it happens frequently enough. If the bad
> {{{e.__class__.name}}} is only triggered under weird combinations of
> installed dependencies, then we might be able to put it off until 1.9.0 .

New description:

 Jimmy Tang [http://tahoe-lafs.org/pipermail/tahoe-
 dev/2011-February/006074.html wrote] to the mailing list with a crash on
 his Archlinux box.

 {{{
   File "/home/jtang/tmp/tahoe-lafs/src/allmydata-
 tahoe-1.8.2/src/allmydata/__init__.py",
 line 301, in cross_check_pkg_resources_versus_import
     % (imp_ver, name, imp_loc, pr_ver, pr_loc, e.__class__.name, e))
 AttributeError: type object 'exceptions.TypeError' has no attribute 'name'
     Aborting...
 }}}

 I don't know if this is a real error path, or if it's just accumulating a
 bunch of warnings to print to stderr (i.e. if we just took out the
 offending line, would the program work anyways, or is the missing/old
 /unable-to-compute-version dependency actually important?).

 This sort of problem reinforces my feeling that our {{{__init__.py}}} is
 way too complex, and that all the version-checking logic may be doing more
 harm than good.

 This may provoke a 1.8.3, if it happens frequently enough. If the bad
 {{{e.__class__.name}}} is only triggered under weird combinations of
 installed dependencies, then we might be able to put it off until 1.9.0 .

--

Comment (by davidsarah):

 warner wrote in the original Description:
 > I suspect it's a python-2.7 incompatibility in the error-handling path
 in all the crazy what-version-of-each-dependency code inside
 {{{__init__.py}}}:
 >
 > ...
 >
 > I suspect that {{{TypeError}}} moved from being implemented in Python
 (in py2.6) to being implemented in C (in py2.7), and that for some reason
 when you try to do the {{{.name}}} lookup, it explodes messily.

 It should have been {{{e.__class__.__name__}}}. Sorry, my fault for not
 testing this error path adequately. It's not at all Python 2.7-specific.

 (For future reference: the Description field isn't the right place to
 speculate about the ''cause'' of a bug.)

 > This may provoke a 1.8.3, if it happens frequently enough. If the bad
 {{{e.__class__.name}}} is only triggered under weird combinations of
 installed dependencies, then we might be able to put it off until 1.9.0 .

 This error occurs on lines 287 and 301 of
 [source:src/allmydata/__init__.py at 4994#L279 __init__.py]. It happens when:
  * the {{{--version}}} or {{{--version-and-path}}} option to
 {{{bin/tahoe}}} is used, and
  * the version of a package obtained either from import or from
 pkg_resources, is not parseable at all by
 [source:src/allmydata/util/verlib.py]. (A non-normalized but parseable
 version is fine.)

 {{{--version-and-path}}} is passed to {{{bin/tahoe}}} by default when you
 run tests [source:setup.py at 4997#L258 via setup.py]. You can use the
 {{{--quiet}}} option (i.e. {{{setup.py test --quiet}}} or {{{setup.py
 trial --quiet}}}) to suppress this.

 > I don't know if this is a real error path, or if it's just accumulating
 a bunch of warnings to print to stderr (i.e. if we just took out the
 offending line, would the program work anyways, or is the missing/old
 /unable-to-compute-version dependency actually important?).

 We don't know whether the warning was indicative of a real error in this
 case, because we don't know what the version or package was. The best way
 to check that would be for Jimmy Tang to replace {{{.name}}} with
 {{{.__name__}}} on lines 287 and 301 of {{{__init__.py}}} and then see
 what is printed.

 It's not clear to me how common it is to have versions that
 {{{verlib.py}}} will raise an exception when trying to parse.

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1355#comment:1>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage