[tahoe-lafs-trac-stream] [Tahoe-LAFS] #3038: Many Python 2.7 macOS builds fail with "Too many open files"

[Tahoe-LAFS]

#3038: Many Python 2.7 macOS builds fail with "Too many open files"
-------------------------+-----------------------
     Reporter:  exarkun  |      Owner:  exarkun
         Type:  defect   |     Status:  new
     Priority:  normal   |  Milestone:  undecided
    Component:  unknown  |    Version:  1.12.1
   Resolution:           |   Keywords:
Launchpad Bug:           |
-------------------------+-----------------------

Comment (by exarkun):

 I tried making a wrapper for the endpoint with a __del__ that closes the
 file descriptor if it needs to be closed (if listen hasn't been called).
 This didn't help because apparently the garbage collector doesn't run
 frequently enough to clean anything up before file descriptors run out
 (this makes sense, you only have to leak a hundred or so file descriptors
 to cause a problem and the GC only wants to run after a lot more has
 happened than that, I expect (CPython GC is driven by
 allication/deallocation from Python memory arenas, not file descriptors)).

 I expect a weakref-based solution to have the same problem since weakrefs
 are driven by the CPython GC.

 Possibly this issue can be worked around by inserting an explicit
 gc.collect() call into this port allocation code path.  This would tie
 collection frequency to port allocation frequency which should let us keep
 on top of the stale resources.

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3038#comment:12>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage