[tahoe-lafs-trac-stream] [tahoe-lafs] #1725: Tahoe reports catch-up incidents to a log gatherer with a Unicode filename, which results in them being dropped

[tahoe-lafs]

#1725: Tahoe reports catch-up incidents to a log gatherer with a Unicode filename,
which results in them being dropped
-------------------------------------------------+-------------------------
 Reporter:  davidsarah                           |          Owner:
     Type:  defect                               |  davidsarah
 Priority:  major                                |         Status:  new
Component:  code-nodeadmin                       |      Milestone:  1.9.2
 Keywords:  regression logging incident          |        Version:  1.9.1
  foolscap                                       |  Launchpad Bug:
-------------------------------------------------+-------------------------
 At [source:src/allmydata/node.py at 5469#L349], we have:
 {{{
         incident_dir = os.path.join(self.basedir, "logs", "incidents")
         # this doesn't quite work yet: unit tests fail
         foolscap.logging.log.setLogDir(incident_dir)
 }}}
 (ignore the comment; it's not relevant to this ticket).

 Since {{{self.basedir}}} is Unicode, so is {{{incident_dir}}}. foolscap
 mostly tolerates this, but sometimes ends up sending a Unicode filename to
 the log gatherer, which causes a type Violation, e.g.:
 {{{
 q2z53drs#188 17:29:18.675: Unhandled Error
 Traceback (most recent call last):
   File "/usr/local/lib/python2.6/dist-
 packages/foolscap-0.6.3-py2.6.egg/foolscap/eventual.py", line 26, in _turn
     cb(*args, **kwargs)
   File "/usr/local/lib/python2.6/dist-
 packages/foolscap-0.6.3-py2.6.egg/foolscap/logging/publish.py", line 106,
 in subscribe
     self.catch_up(since)
   File "/usr/local/lib/python2.6/dist-
 packages/foolscap-0.6.3-py2.6.egg/foolscap/logging/publish.py", line 114,
 in catch_up
     self.observer.callRemoteOnly("new_incident", name, trigger)
   File "/usr/local/lib/python2.6/dist-
 packages/foolscap-0.6.3-py2.6.egg/foolscap/referenceable.py", line 422, in
 callRemoteOnly
     *args, **kwargs)
 --- <exception caught here> ---
   File "/usr/local/lib/python2.6/dist-
 packages/Twisted-11.1.0-py2.6-linux-i686.egg/twisted/internet/defer.py",
 line 134, in maybeDeferred
     result = f(*args, **kw)
   File "/usr/local/lib/python2.6/dist-
 packages/foolscap-0.6.3-py2.6.egg/foolscap/referenceable.py", line 482, in
 _callRemote
     methodSchema.checkAllArgs(args, kwargs, False)
   File "/usr/local/lib/python2.6/dist-
 packages/foolscap-0.6.3-py2.6.egg/foolscap/remoteinterface.py", line 284,
 in checkAllArgs
     constraint.checkObject(argvalue, inbound)
   File "/usr/local/lib/python2.6/dist-
 packages/foolscap-0.6.3-py2.6.egg/foolscap/constraint.py", line 220, in
 checkObject
     raise Violation("'%r' is not a bytestring" % (obj,))
 foolscap.tokens.Violation: Violation
 (RILogObserver.foolscap.lothar.com.new_incident(name=)):
 ("'u'incident-2012-04-28--21-28-05Z-q3rwjdq'' is not a bytestring",)
 }}}

 The code in foolscap that creates the Unicode filenames is
 !LogPublisher.list_incident_names in foolscap/logging/publish.py. Due to
 Python 2.x's implicit unicode<->str conversions (booo!) and "do what I
 thought you wanted" behaviour of the filesystem APIs, there is no Python
 type error.

 The effect is that if a log-gatherer was down when incidents occurred and
 subsequently tries to catch up, those incidents will be dropped.

 This is a regression that was introduced with the Unicode basedir changes
 released in 1.8 (specifically [4605/trunk]).

-- 
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1725>
tahoe-lafs <https://tahoe-lafs.org>
secure decentralized storage