[tahoe-lafs-trac-stream] [tahoe-lafs] #1669: ValueError (need more than 8 values to unpack) during mutable MDMF repair

[tahoe-lafs]

#1669: ValueError (need more than 8 values to unpack) during mutable MDMF repair
-------------------------+-------------------------------------------------
     Reporter:  vikarti  |      Owner:  zooko
         Type:  defect   |     Status:  assigned
     Priority:           |  Milestone:  1.9.2
  critical               |    Version:  1.9.1
    Component:  code-    |   Keywords:  mutable mdmf repair error
  mutable                |  regression preservation
   Resolution:           |
Launchpad Bug:           |
-------------------------+-------------------------------------------------

Comment (by zooko):

 Update: I hate it when I post partial results of my investigations and
 then someone (usually Brian) goes and finishes debugging it and fixing it
 before I do. But, I figure that's not a good reason to withhold
 information, so here's my partial results.

 • The stack trace in the original post shows a tuple with 8 elements being
 returned in place of "verinfo", but "verinfos" are supposed to have 9
 elements.

 • The elements shown in the stack trace in the original post are not the
 elements of a verinfo. A verinfo is (according to
 [source:git/src/allmydata/mutable/layout.py?annotate=blame&rev=87ca4fc7055faaea7e54cbab4584132b021e42e1#L433
 mutable/layout.py]) is (int -- seqnum, cryptoval -- hash, cryptoval --
 salt or hash, int -- segsize, int -- datalen, int -- k, int -- n, str --
 prefix for signing, tuple -- offsets). The stack trace in the original
 post shows something that is (int -- seqnum, cryptoval -- hash?, int -- k,
 int -- n, int -- segsize, int -- datalen, str -- prefix for signing?,
 tuple -- offsets).

 • The docstring of
 [source:git/src/allmydata/mutable/layout.py?annotate=blame&rev=87ca4fc7055faaea7e54cbab4584132b021e42e1#L433
 SDMFSlotWriteProxy.get_verinfo] is wrong about the 2-indexed item -- it is
 a hash or a salt, not "a blank (nothing)"

 •
 [source:git/src/allmydata/mutable/layout.py?annotate=blame&rev=87ca4fc7055faaea7e54cbab4584132b021e42e1#L433
 SDMFSlotWriteProxy.get_verinfo] and
 [source:git/src/allmydata/mutable/layout.py?annotate=blame&rev=87ca4fc7055faaea7e54cbab4584132b021e42e1#L1692
 MDMFSlotReadProxy.get_verinfo] return the same shape of tuple, but
 [source:git/src/allmydata/mutable/layout.py?annotate=blame&rev=87ca4fc7055faaea7e54cbab4584132b021e42e1#L1102
 MDMFSlotWriteProxy.get_verinfo] returns a different shape. Aha! And the
 latter matches the data shown in the stack trace in the original post, so
 it is one of those that is ending up getting parsed as a "verinfo" and
 causing this exception.

 • I wrote a type-checking function that asserts the shape of a verinfo
 tuple and sprinkled it everywhere and ran unit tests and got unit tests
 failures.

 • I don't know how this code that says "d2 = reader.get_verinfo();
 d2.addErrback(…)" in
 [source:git/src/allmydata/mutable/servermap.py?annotate=blame&rev=470acbf1e1d0a525cb8af9fb7a82ebe09033e8ca#L705
 mutable/servermap.py] ever worked! None of the functions named
 "get_verinfo" that I see return a Deferred. But, if that function raises
 an !AttributeError every time (saying "tuple has no such attribute
 'addErrback'"), wouldn't we have noticed before now? That line of code was
 added in [bb10d685ed86eb0870bbd09469f912043e405711/git] 11 months ago and
 has been in 1.9.x all along. Hm... but I see that any !AttributeError
 raised from there would get handled by
 [source:git/src/allmydata/mutable/servermap.py?annotate=blame&rev=470acbf1e1d0a525cb8af9fb7a82ebe09033e8ca#L981
 _query_failed]...

 Okay, I'm taking a brief break. but DON'T GO AND FIX THIS WITHOUT ME AND
 HAVE ALL THE FUN.

-- 
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1669#comment:12>
tahoe-lafs <https://tahoe-lafs.org>
secure decentralized storage