[tahoe-lafs-trac-stream] [Tahoe-LAFS] #3887: HTTPConnectionPool briefly leaks reactor state in a way that makes resource cleanup more complicated

[Tahoe-LAFS]

#3887: HTTPConnectionPool briefly leaks reactor state in a way that makes resource
cleanup more complicated
---------------------+---------------------------
 Reporter:  exarkun  |          Owner:
     Type:  defect   |         Status:  new
 Priority:  normal   |      Milestone:  undecided
Component:  unknown  |        Version:  n/a
 Keywords:           |  Launchpad Bug:
---------------------+---------------------------
 `twisted.web.client.HTTPConnectionPool`, used with `Agent`, does not offer
 a way to wait until all of the connections it created have been closed.

 It has a `closedCachedConnections` method which can be used to wait for
 all of its idle, cached connections (which I think you can already ensure
 will be an empty set if you set `persistent=False`).  However, this will
 not wait for connections that are not idle.

 The almost guaranteed series of events, then, is:

 1. You make a request which opens a connection and does some network stuff
 2. The response to the request comes in and your request Deferred fires
 3. Your application code processes the response and when that is over, you
 believe you are done
 4. Finally, the HTTP client implementation gets to clean itself up, either
 closing the connection or returning it to the pool.

 After (3) you usually assume everything is done but there is still an open
 connection until (4).  You'll most easily notice this in a test when you
 think the test can end after (3) and then trial complains about the open
 connection.  Conceivably this could be an issue under real world use,
 though it seems less common that a single connection existing for slightly
 longer than expected causes issues there.

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3887>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage