Opened at 2012-03-21T13:47:30Z
Last modified at 2012-04-05T20:18:13Z
#1689 closed defect
assertion failure — at Initial Version
| Reported by: | jg71 | Owned by: | nobody |
|---|---|---|---|
| Priority: | critical | Milestone: | 1.9.2 |
| Component: | code-mutable | Version: | 1.9.1 |
| Keywords: | tahoe-check verify regression assertion error mutable review-needed | Cc: | |
| Launchpad Bug: |
Description
tahoe deep-check -v --repair --add-lease tahoe:
'<root>': healthy done: 1 objects checked pre-repair: 1 healthy, 0 unhealthy 0 repairs attempted, 0 successful, 0 failed post-repair: 1 healthy, 0 unhealthy
when 9 out of 9 storage servers are available.
The error I have found is reproducable by just stopping one storage server and run once more: tahoe deep-check -v --repair --add-lease tahoe:
ERROR: AssertionError() "[Failure instance: Traceback: <type 'exceptions.AssertionError'>: " /usr/lib64/python2.6/site-packages/allmydata/mutable/filenode.py:563:upload /usr/lib64/python2.6/site-packages/allmydata/mutable/filenode.py:661:_do_serialized /usr/lib64/python2.6/site-packages/twisted/internet/defer.py:298:addCallback /usr/lib64/python2.6/site-packages/twisted/internet/defer.py:287:addCallbacks --- <exception caught here> --- /usr/lib64/python2.6/site-packages/twisted/internet/defer.py:545:_runCallbacks /usr/lib64/python2.6/site-packages/allmydata/mutable/filenode.py:661:<lambda> /usr/lib64/python2.6/site-packages/allmydata/mutable/filenode.py:689:_upload /usr/lib64/python2.6/site-packages/allmydata/mutable/publish.py:404:publish
I can see from the second capture that tahoe connects exactly to that storage server that has been stopped: connectTCP to ('256.256.256.256', 66666)
Versions used locally:
Nevow: 0.10.0, foolscap: 0.6.3, setuptools: 0.6c11, Twisted: 11.1.0, zfec: 1.4.22, zbase32: 1.1.3, pyOpenSSL: 0.13, simplejson: 2.3.2, mock: 0.7.2, argparse: 1.2.1, pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958, pyutil: 1.8.4, zope.interface: 3.8.0, allmydata-tahoe: allmydata-tahoe-1.9.0-94-gcef646c, pycrypto: 2.5, pyasn1: 0.0.13
all 9 storage servers use these: Nevow: 0.10.0, foolscap: 0.6.3, setuptools: 0.6c11, Twisted: 11.1.0, zfec: 1.4.22, pycrypto: 2.4.1, zbase32: 1.1.3, pyOpenSSL: 0.13, simplejson: 2.3.2, mock: 0.7.2, argparse: 1.2.1, pyutil: 1.8.4, zope.interface: 3.8.0, allmydata-tahoe: 1.9.1, pyasn1: 0.0.13, pycryptopp: 0.5.29
Might this be related to https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1648 ?
