[tahoe-lafs-trac-stream] [tahoe-lafs] #2034: Mutable update tests are sensitive to the number of servers

[tahoe-lafs]

#2034: Mutable update tests are sensitive to the number of servers
------------------------------+-----------------------
     Reporter:  markberger    |      Owner:
         Type:  defect        |     Status:  new
     Priority:  normal        |  Milestone:  undecided
    Component:  code-mutable  |    Version:  1.10.0
   Resolution:                |   Keywords:
Launchpad Bug:                |
------------------------------+-----------------------
Description changed by markberger:

Old description:

> When the number of servers used in allmydata.test.test_mutable.Update is
> changed from the default value to ten to something greater than ten,
> almost all of the tests fail due to key error.
>
> {{{
> ===============================================================================
> [FAIL]
> Traceback (most recent call last):
>   File "/Users/markberger/Code/tahoe-
> lafs/src/allmydata/test/test_mutable.py", line 3663, in <lambda>
>     self.failUnlessEqual(results, new_data))
>   File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
> /site-packages/twisted/trial/_synctest.py", line 356, in assertEqual
>     % (msg, pformat(first), pformat(second)))
> twisted.trial.unittest.FailTest: not equal:
> a = 'test datatest datatest datatest datatest datatest datatest datatest
> datatest datatest data'
> b = 'test datatest datatest datatest datatest datatest datatest datatest
> datatest datatest dataappended'
>

> allmydata.test.test_mutable.Update.test_update_sdmf
> ===============================================================================
> [ERROR]
> Traceback (most recent call last):
> Failure: allmydata.mutable.common.NotEnoughServersError: ("Publish ran
> out of good servers, last failure was: [Failure instance: Traceback:
> <type 'exceptions.KeyError'>: 0\n/Users/markberger/Code/tahoe-
> lafs/src/allmydata/mutable/publish.py:642:_push\n/Users/markberger/Code
> /tahoe-
> lafs/src/allmydata/mutable/publish.py:662:push_segment\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
> /site-
> packages/twisted/internet/defer.py:304:addCallback\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
> /site-packages/twisted/internet/defer.py:293:addCallbacks\n--- <exception
> caught here>
> ---\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
> /site-
> packages/twisted/internet/defer.py:575:_runCallbacks\n/Users/markberger/Code
> /tahoe-lafs/src/allmydata/mutable/publish.py:767:_push_segment\n]", None)
>
> allmydata.test.test_mutable.Update.test_append
> allmydata.test.test_mutable.Update.test_append_power_of_two
> allmydata.test.test_mutable.Update.test_multiple_segment_replace
> allmydata.test.test_mutable.Update.test_replace_beginning
> allmydata.test.test_mutable.Update.test_replace_segstart1
> ===============================================================================
> [ERROR]
> Traceback (most recent call last):
> Failure: allmydata.mutable.common.NotEnoughServersError: ("Publish ran
> out of good servers, last failure was: [Failure instance: Traceback:
> <type 'exceptions.KeyError'>: 1\n/Users/markberger/Code/tahoe-
> lafs/src/allmydata/mutable/publish.py:642:_push\n/Users/markberger/Code
> /tahoe-
> lafs/src/allmydata/mutable/publish.py:662:push_segment\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
> /site-
> packages/twisted/internet/defer.py:304:addCallback\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
> /site-packages/twisted/internet/defer.py:293:addCallbacks\n--- <exception
> caught here>
> ---\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
> /site-
> packages/twisted/internet/defer.py:575:_runCallbacks\n/Users/markberger/Code
> /tahoe-lafs/src/allmydata/mutable/publish.py:767:_push_segment\n]", None)
>
> allmydata.test.test_mutable.Update.test_replace_and_extend
> allmydata.test.test_mutable.Update.test_replace_in_last_segment
> allmydata.test.test_mutable.Update.test_replace_locations
> allmydata.test.test_mutable.Update.test_replace_middle
> allmydata.test.test_mutable.Update.test_replace_zero_length_middle
> ===============================================================================
> [ERROR]
> Traceback (most recent call last):
>   File "/Users/markberger/Code/tahoe-
> lafs/src/allmydata/mutable/filenode.py", line 1177, in
> _build_uploadable_and_finish
>     return p.update(u, offset, segments_and_bht[2], self._version)
>   File "/Users/markberger/Code/tahoe-
> lafs/src/allmydata/mutable/publish.py", line 341, in update
>     self._push()
>   File "/Users/markberger/Code/tahoe-
> lafs/src/allmydata/mutable/publish.py", line 642, in _push
>     return self.push_segment(self._current_segment)
>   File "/Users/markberger/Code/tahoe-
> lafs/src/allmydata/mutable/publish.py", line 659, in push_segment
>     return self._push()
>   File "/Users/markberger/Code/tahoe-
> lafs/src/allmydata/mutable/publish.py", line 645, in _push
>     return self.push_everything_else()
>   File "/Users/markberger/Code/tahoe-
> lafs/src/allmydata/mutable/publish.py", line 778, in push_everything_else
>     self.push_blockhashes()
>   File "/Users/markberger/Code/tahoe-
> lafs/src/allmydata/mutable/publish.py", line 806, in push_blockhashes
>     writers = self.writers[shnum]
> exceptions.KeyError: 0
>
> allmydata.test.test_mutable.Update.test_replace_zero_length_beginning
> allmydata.test.test_mutable.Update.test_replace_zero_length_segstart1
> -------------------------------------------------------------------------------
> Ran 14 tests in 58.570s
> }}}

New description:

 When the number of servers used in allmydata.test.test_mutable.Update is
 changed from the default value of ten to something greater than ten,
 almost all of the tests fail due to key error.

 {{{
 ===============================================================================
 [FAIL]
 Traceback (most recent call last):
   File "/Users/markberger/Code/tahoe-
 lafs/src/allmydata/test/test_mutable.py", line 3663, in <lambda>
     self.failUnlessEqual(results, new_data))
   File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
 /site-packages/twisted/trial/_synctest.py", line 356, in assertEqual
     % (msg, pformat(first), pformat(second)))
 twisted.trial.unittest.FailTest: not equal:
 a = 'test datatest datatest datatest datatest datatest datatest datatest
 datatest datatest data'
 b = 'test datatest datatest datatest datatest datatest datatest datatest
 datatest datatest dataappended'


 allmydata.test.test_mutable.Update.test_update_sdmf
 ===============================================================================
 [ERROR]
 Traceback (most recent call last):
 Failure: allmydata.mutable.common.NotEnoughServersError: ("Publish ran out
 of good servers, last failure was: [Failure instance: Traceback: <type
 'exceptions.KeyError'>: 0\n/Users/markberger/Code/tahoe-
 lafs/src/allmydata/mutable/publish.py:642:_push\n/Users/markberger/Code
 /tahoe-
 lafs/src/allmydata/mutable/publish.py:662:push_segment\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
 /site-
 packages/twisted/internet/defer.py:304:addCallback\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
 /site-packages/twisted/internet/defer.py:293:addCallbacks\n--- <exception
 caught here>
 ---\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
 packages/twisted/internet/defer.py:575:_runCallbacks\n/Users/markberger/Code
 /tahoe-lafs/src/allmydata/mutable/publish.py:767:_push_segment\n]", None)

 allmydata.test.test_mutable.Update.test_append
 allmydata.test.test_mutable.Update.test_append_power_of_two
 allmydata.test.test_mutable.Update.test_multiple_segment_replace
 allmydata.test.test_mutable.Update.test_replace_beginning
 allmydata.test.test_mutable.Update.test_replace_segstart1
 ===============================================================================
 [ERROR]
 Traceback (most recent call last):
 Failure: allmydata.mutable.common.NotEnoughServersError: ("Publish ran out
 of good servers, last failure was: [Failure instance: Traceback: <type
 'exceptions.KeyError'>: 1\n/Users/markberger/Code/tahoe-
 lafs/src/allmydata/mutable/publish.py:642:_push\n/Users/markberger/Code
 /tahoe-
 lafs/src/allmydata/mutable/publish.py:662:push_segment\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
 /site-
 packages/twisted/internet/defer.py:304:addCallback\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
 /site-packages/twisted/internet/defer.py:293:addCallbacks\n--- <exception
 caught here>
 ---\n/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
 packages/twisted/internet/defer.py:575:_runCallbacks\n/Users/markberger/Code
 /tahoe-lafs/src/allmydata/mutable/publish.py:767:_push_segment\n]", None)

 allmydata.test.test_mutable.Update.test_replace_and_extend
 allmydata.test.test_mutable.Update.test_replace_in_last_segment
 allmydata.test.test_mutable.Update.test_replace_locations
 allmydata.test.test_mutable.Update.test_replace_middle
 allmydata.test.test_mutable.Update.test_replace_zero_length_middle
 ===============================================================================
 [ERROR]
 Traceback (most recent call last):
   File "/Users/markberger/Code/tahoe-
 lafs/src/allmydata/mutable/filenode.py", line 1177, in
 _build_uploadable_and_finish
     return p.update(u, offset, segments_and_bht[2], self._version)
   File "/Users/markberger/Code/tahoe-
 lafs/src/allmydata/mutable/publish.py", line 341, in update
     self._push()
   File "/Users/markberger/Code/tahoe-
 lafs/src/allmydata/mutable/publish.py", line 642, in _push
     return self.push_segment(self._current_segment)
   File "/Users/markberger/Code/tahoe-
 lafs/src/allmydata/mutable/publish.py", line 659, in push_segment
     return self._push()
   File "/Users/markberger/Code/tahoe-
 lafs/src/allmydata/mutable/publish.py", line 645, in _push
     return self.push_everything_else()
   File "/Users/markberger/Code/tahoe-
 lafs/src/allmydata/mutable/publish.py", line 778, in push_everything_else
     self.push_blockhashes()
   File "/Users/markberger/Code/tahoe-
 lafs/src/allmydata/mutable/publish.py", line 806, in push_blockhashes
     writers = self.writers[shnum]
 exceptions.KeyError: 0

 allmydata.test.test_mutable.Update.test_replace_zero_length_beginning
 allmydata.test.test_mutable.Update.test_replace_zero_length_segstart1
 -------------------------------------------------------------------------------
 Ran 14 tests in 58.570s
 }}}

--

-- 
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2034#comment:1>
tahoe-lafs <https://tahoe-lafs.org>
secure decentralized storage