Opened at 2012-11-20T01:37:31Z

Last modified at 2021-03-30T18:40:19Z

#1870 new defect

leasedb: performance regression — at Version 7

Reported by: davidsarah Owned by: markberger
Priority: normal Milestone: soon
Component: code-storage Version: 1.9.2
Keywords: leasedb performance regression sqlite Cc:
Launchpad Bug:

Description (last modified by daira)

The ​1818-leasedb branch has a performance regression that shows up when running the test suite -- in fact, the test suite is not able to pass at the moment purely due to timeouts.

Since the regression does not show up when using make tmpfstest (which uses a memory-based tmpfs filesystem rather than disk), my tentative conclusion is that it is due to the latency of leasedb database syncs. There are currently many redundant syncs due to every SQL query/update being in a separate transaction, and due to there being more SQL queries and updates than necessary per storage API request. We could also use a more relaxed consistency mode, if that is safe.

Change History (7)

comment:1 Changed at 2012-11-20T01:38:50Z by davidsarah

  • Description modified (diff)
  • Status changed from new to assigned

comment:2 Changed at 2012-11-20T01:39:56Z by davidsarah

  • Milestone changed from undecided to 1.11.0

comment:3 Changed at 2012-11-20T01:40:11Z by davidsarah

  • Component changed from unknown to code-storage

comment:4 Changed at 2013-02-28T00:02:30Z by zooko

Here are my notes about this:

​https://tahoe-lafs.org/pipermail/tahoe-dev/2012-December/007877.html

Bottom line: I believe we should turn on sqlite's synchronous = NORMAL, journal mode = WAL.

Also, that we should fix ticket #1893, which would reduce this load.

comment:5 Changed at 2013-02-28T04:41:17Z by davidsarah

I thought that mode caused a problem with file handle leakage? Or am I misremembering?

comment:6 Changed at 2013-02-28T04:43:53Z by davidsarah

No, I remembered correctly. In current leasedb.py: 

# synchronous = OFF is necessary for leasedb to pass tests for the time being,
# since using synchronous = NORMAL causes failures that are apparently due to
# a file descriptor leak, and the default synchronous = FULL causes the tests
# to time out.

comment:7 Changed at 2013-07-04T19:19:25Z by daira

  • Description modified (diff)
  • Keywords blocks-cloud-merge added
  • Owner changed from davidsarah to markberger
  • Status changed from assigned to new
Note: See TracTickets for help on using tickets.