[tahoe-lafs-trac-stream] [Tahoe-LAFS] #1794: upload failed -- "I/O operation on closed file"

Tahoe-LAFS trac at tahoe-lafs.org
Fri Dec 19 02:09:06 UTC 2014 

#1794: upload failed -- "I/O operation on closed file"
------------------------------+----------------------------------
     Reporter:  zooko         |      Owner:
         Type:  defect        |     Status:  new
     Priority:  normal        |  Milestone:  undecided
    Component:  code-network  |    Version:  1.9.2
   Resolution:                |   Keywords:  upload error twisted
Launchpad Bug:                |
------------------------------+----------------------------------

Comment (by zooko):

 AB pressed the `Report An Incident` button, and mailed me the resulting
 incident report file. I'll attach it to this ticket, but here is the first
 evidence of weirdness in it:
 {{{
 local#12548 rx(2014-12-19_01:06:25.269490Z)
 emit(2014-12-19_00:59:39.491843Z): upload failed
  FAILURE:
  [CopiedFailure instance: Traceback from remote host -- Traceback (most
 recent call last):
    File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 295, in addCallbacks
      self._runCallbacks()
    File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 577, in _runCallbacks
      current.result = callback(current.result, *args, **kw)
    File "/usr/lib64/python2.7/site-
 packages/allmydata/immutable/upload.py", line 752, in <lambda>
      self._read_encrypted(length, ciphertext, hash_only, d2))
    File "/usr/lib64/python2.7/site-
 packages/allmydata/immutable/upload.py", line 767, in _read_encrypted
      d = defer.maybeDeferred(self.original.read, size)
  --- <exception caught here> ---
    File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 139, in maybeDeferred
      result = f(*args, **kw)
    File "/usr/lib64/python2.7/site-
 packages/allmydata/immutable/upload.py", line 1467, in read
      return defer.succeed([self._filehandle.read(length)])
  exceptions.ValueError: I/O operation on closed file
  ]
 local#12549 rx(2014-12-19_01:06:25.269490Z)
 emit(2014-12-19_00:59:39.608912Z): aborting shareholders
 local#12550 rx(2014-12-19_01:06:25.269490Z)
 emit(2014-12-19_00:59:39.618792Z): Exception rendering error page:
 local#12551 rx(2014-12-19_01:06:25.269490Z)
 emit(2014-12-19_00:59:39.646960Z): Unhandled Error
 Traceback (most recent call last):
   File "/usr/lib64/python2.7/site-packages/foolscap/eventual.py", line 26,
 in _turn
     cb(*args, **kwargs)
   File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 382, in callback
     self._startRunCallbacks(result)
   File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 490, in _startRunCallbacks
     self._runCallbacks()
   File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 577, in _runCallbacks
     current.result = callback(current.result, *args, **kw)
 --- <exception caught here> ---
   File "/usr/lib64/python2.7/site-packages/nevow/appserver.py", line 82,
 in processingFailed
     handler.renderHTTP_exception(ctx, reason)
   File "/usr/lib64/python2.7/site-packages/allmydata/web/common.py", line
 332, in renderHTTP_exception
     return self.simple(ctx, traceback, http.INTERNAL_SERVER_ERROR)
   File "/usr/lib64/python2.7/site-packages/allmydata/web/common.py", line
 302, in simple
     req.finishRequest(False)
   File "/usr/lib64/python2.7/site-packages/nevow/appserver.py", line 176,
 in finishRequest
     server.Request.finish(self)
   File "/usr/lib64/python2.7/site-packages/twisted/web/server.py", line
 228, in finish
     return http.Request.finish(self)
   File "/usr/lib64/python2.7/site-packages/twisted/web/http.py", line 930,
 in finish
     "Request.finish called on a request after its connection was lost; "
 exceptions.RuntimeError: Request.finish called on a request after its
 connection was lost; use Request.notifyFinish to keep track of this.

  FAILURE:
  [CopiedFailure instance: Traceback from remote host -- Traceback (most
 recent call last):
    File "/usr/lib64/python2.7/site-packages/foolscap/eventual.py", line
 26, in _turn
      cb(*args, **kwargs)
    File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 382, in callback
      self._startRunCallbacks(result)
    File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 490, in _startRunCallbacks
      self._runCallbacks()
    File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 577, in _runCallbacks
      current.result = callback(current.result, *args, **kw)
  --- <exception caught here> ---
    File "/usr/lib64/python2.7/site-packages/nevow/appserver.py", line 82,
 in processingFailed
      handler.renderHTTP_exception(ctx, reason)
    File "/usr/lib64/python2.7/site-packages/allmydata/web/common.py", line
 332, in renderHTTP_exception
      return self.simple(ctx, traceback, http.INTERNAL_SERVER_ERROR)
    File "/usr/lib64/python2.7/site-packages/allmydata/web/common.py", line
 302, in simple
      req.finishRequest(False)
    File "/usr/lib64/python2.7/site-packages/nevow/appserver.py", line 176,
 in finishRequest
      server.Request.finish(self)
    File "/usr/lib64/python2.7/site-packages/twisted/web/server.py", line
 228, in finish
      return http.Request.finish(self)
    File "/usr/lib64/python2.7/site-packages/twisted/web/http.py", line
 930, in finish
      "Request.finish called on a request after its connection was lost; "
  exceptions.RuntimeError: Request.finish called on a request after its
 connection was lost; use Request.notifyFinish to keep track of this.
  ]
 local#12552 rx(2014-12-19_01:06:25.269490Z)
 emit(2014-12-19_00:59:39.852801Z): 'Original exception:'
 local#12553 rx(2014-12-19_01:06:25.269490Z)
 emit(2014-12-19_00:59:39.853193Z): Unhandled Error
 Traceback (most recent call last):
   File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 295, in addCallbacks
     self._runCallbacks()
   File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 577, in _runCallbacks
     current.result = callback(current.result, *args, **kw)
   File "/usr/lib64/python2.7/site-packages/allmydata/immutable/upload.py",
 line 752, in <lambda>
     self._read_encrypted(length, ciphertext, hash_only, d2))
   File "/usr/lib64/python2.7/site-packages/allmydata/immutable/upload.py",
 line 767, in _read_encrypted
     d = defer.maybeDeferred(self.original.read, size)
 --- <exception caught here> ---
   File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 139, in maybeDeferred
     result = f(*args, **kw)
   File "/usr/lib64/python2.7/site-packages/allmydata/immutable/upload.py",
 line 1467, in read
     return defer.succeed([self._filehandle.read(length)])
 exceptions.ValueError: I/O operation on closed file

  FAILURE:
  [CopiedFailure instance: Traceback from remote host -- Traceback (most
 recent call last):
    File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 295, in addCallbacks
      self._runCallbacks()
    File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 577, in _runCallbacks
      current.result = callback(current.result, *args, **kw)
    File "/usr/lib64/python2.7/site-
 packages/allmydata/immutable/upload.py", line 752, in <lambda>
      self._read_encrypted(length, ciphertext, hash_only, d2))
    File "/usr/lib64/python2.7/site-
 packages/allmydata/immutable/upload.py", line 767, in _read_encrypted
      d = defer.maybeDeferred(self.original.read, size)
  --- <exception caught here> ---
    File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 139, in maybeDeferred
      result = f(*args, **kw)
    File "/usr/lib64/python2.7/site-
 packages/allmydata/immutable/upload.py", line 1467, in read
      return defer.succeed([self._filehandle.read(length)])
  exceptions.ValueError: I/O operation on closed file
  ]
 local#12554 rx(2014-12-19_01:06:25.269490Z)
 emit(2014-12-19_00:59:39.855494Z): Unhandled error in Deferred:
 local#12555 rx(2014-12-19_01:06:25.269490Z)
 emit(2014-12-19_00:59:39.855931Z): Unhandled Error
 Traceback (most recent call last):
   File "/usr/lib64/python2.7/site-packages/twisted/internet/base.py", line
 824, in runUntilCurrent
     call.func(*call.args, **call.kw)
   File "/usr/lib64/python2.7/site-packages/foolscap/eventual.py", line 26,
 in _turn
     cb(*args, **kwargs)
   File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 382, in callback
     self._startRunCallbacks(result)
   File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 490, in _startRunCallbacks
     self._runCallbacks()
 --- <exception caught here> ---
   File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 577, in _runCallbacks
     current.result = callback(current.result, *args, **kw)
   File "/usr/lib64/python2.7/site-packages/nevow/appserver.py", line 91,
 in processingFailed
     request.finishRequest( False )
   File "/usr/lib64/python2.7/site-packages/nevow/appserver.py", line 176,
 in finishRequest
     server.Request.finish(self)
   File "/usr/lib64/python2.7/site-packages/twisted/web/server.py", line
 228, in finish
     return http.Request.finish(self)
   File "/usr/lib64/python2.7/site-packages/twisted/web/http.py", line 930,
 in finish
     "Request.finish called on a request after its connection was lost; "
 exceptions.RuntimeError: Request.finish called on a request after its
 connection was lost; use Request.notifyFinish to keep track of this.

  FAILURE:
  [CopiedFailure instance: Traceback from remote host -- Traceback (most
 recent call last):
    File "/usr/lib64/python2.7/site-packages/twisted/internet/base.py",
 line 824, in runUntilCurrent
      call.func(*call.args, **call.kw)
    File "/usr/lib64/python2.7/site-packages/foolscap/eventual.py", line
 26, in _turn
      cb(*args, **kwargs)
    File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 382, in callback
      self._startRunCallbacks(result)
    File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 490, in _startRunCallbacks
      self._runCallbacks()
  --- <exception caught here> ---
    File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py",
 line 577, in _runCallbacks
      current.result = callback(current.result, *args, **kw)
    File "/usr/lib64/python2.7/site-packages/nevow/appserver.py", line 91,
 in processingFailed
      request.finishRequest( False )
    File "/usr/lib64/python2.7/site-packages/nevow/appserver.py", line 176,
 in finishRequest
      server.Request.finish(self)
    File "/usr/lib64/python2.7/site-packages/twisted/web/server.py", line
 228, in finish
      return http.Request.finish(self)
    File "/usr/lib64/python2.7/site-packages/twisted/web/http.py", line
 930, in finish
      "Request.finish called on a request after its connection was lost; "
  exceptions.RuntimeError: Request.finish called on a request after its
 connection was lost; use Request.notifyFinish to keep track of this.
  ]

 }}}

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1794#comment:6>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage

More information about the tahoe-lafs-trac-stream mailing list