[tahoe-lafs-trac-stream] [tahoe-lafs] #2015: leasedb: high file descriptor usage exposed by tests

[tahoe-lafs]

#2015: leasedb: high file descriptor usage exposed by tests
-------------------------+-------------------------------------------------
     Reporter:  daira    |      Owner:  markberger
         Type:  defect   |     Status:  new
     Priority:  major    |  Milestone:  1.11.0
    Component:  code-    |    Version:  1.10.0
  storage                |   Keywords:  leasedb tests reliability file-
   Resolution:           |  descriptor leak
Launchpad Bug:           |
-------------------------+-------------------------------------------------

Comment (by zooko):

 Replying to [comment:2 daira]:
 > Replying to [comment:1 zooko]:
 > > fdleakfinder definitely reports that the sqlite files opened during
 test cases are left open instead of closed at the end of this test case.
 >
 > That's understandable, because there is no code to close them! (This
 shouldn't matter for a node which only has one DB connection open; it only
 matters for tests.)

 I would have assumed that the fds would be closed automatically when the
 Python objects were garbage-collected.

 markberger: it sounds like there might be a bug in the Python sqlite
 module such that it leaks fds. Or at least a misunderstanding on my part.
 You could investigate this hypothesis by writing a program that opens a
 million sqlite databases in a row while losing its reference to teh
 previous one. Something like this (untested, written by cutting and
 pasting a bit of code from dbutil.py):

 {{{
 for i in range(1000000):
     sqlite3.connect("db-test-file-%d" % i)
 }}}

 If that script runs out of fds, then indeed the underlying database files
 are not getting closed when the accompanying Python objects get garbage
 collected. (Or the Python objects are not getting garbage-collected in a
 timely way, due to reference cycles preventing their de-allocation by
 reference-counting…)

-- 
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2015#comment:3>
tahoe-lafs <https://tahoe-lafs.org>
secure decentralized storage