Opened at 2009-03-24T23:55:00Z
Last modified at 2013-05-03T01:01:44Z
#667 closed defect
KeyError in mutable download — at Version 9
| Reported by: | warner | Owned by: | |
|---|---|---|---|
| Priority: | major | Milestone: | soon |
| Component: | code-mutable | Version: | 1.3.0 |
| Keywords: | mutable download reliability availability | Cc: | |
| Launchpad Bug: |
Description (last modified by davidsarah)
Zooko observed a KeyError exception raised during mutable-file retrieval, in #651 (specifically at git/src/allmydata/mutable/retrieve.py?rev=4061258c#L157). This is a bug in the retrieval code, probably triggered by running out of valid shares. Either the self.remaining_sharemap mapping should not be modified while download() is iterating over it, or more likely download() must be prepared for remaining_sharemap to change during the iteration.
Change History (9)
comment:1 Changed at 2009-12-30T01:45:33Z by davidsarah
- Keywords download availability added
comment:2 Changed at 2009-12-30T01:46:00Z by davidsarah
- Summary changed from KeyError in mutable publish to KeyError in mutable download
comment:3 Changed at 2010-01-26T16:56:11Z by zooko
- Milestone changed from 1.6.0 to eventually
comment:4 Changed at 2010-05-27T22:10:26Z by zooko
- Milestone changed from eventually to 1.8.0
comment:5 Changed at 2010-08-10T04:14:17Z by davidsarah
- Keywords mutable reliability added
- Milestone changed from 1.8.0 to 1.9.0
comment:6 Changed at 2010-08-10T04:40:29Z by zooko
If you like this ticket, you're probably the kind of weirdo who would like #474 (uncaught exception in mutable-retrieve: UCW between mapupdate and retrieve), #548 (mutable publish sends queries to servers that have already been asked), #540 (inappropriate "uncoordinated write error" after handling a server failure), #546 (mutable-file surprise shares raise inappropriate UCWE), #547 (mapupdate(MODE_WRITE) triggers on a false boundary), #548 (mutable publish sends queries to servers that have already been asked), #549 (MODE_WRITE mapupdate: maybe increase epsilon to handle large batches of new servers better). And we need more weirdos like you!
comment:7 Changed at 2011-07-23T20:31:03Z by davidsarah
- Milestone changed from 1.9.0 to soon
This seems unlikely to be fixed for 1.9.
comment:8 follow-up: ↓ 9 Changed at 2012-12-06T22:15:31Z by davidsarah
The description of this ticket probably refers to the code as it was before the start of Kevan's MDMF patches, i.e. up to and including darcs revision 4772 / git revision 4061258c. The code changes a lot for MDMF and this bug might have been incidentally fixed -- in particular, there is no longer any indexing into self.remaining_sharemap, and all of the loops over its keys are done synchronously with the keys having been copied beforehand.
If that's correct, this ticket can be closed as "cannot reproduce".
(There was a vaguely similar bug in git/src/allmydata/mutable/publish.py that is easy to confuse with this one, but that was fixed in #1749.)
comment:9 in reply to: ↑ 8 Changed at 2012-12-06T22:19:23Z by davidsarah
- Description modified (diff)
Editing description to refer to the right revision.
It's really bothering me that mutable file upload and download behavior is so finicky, buggy, inefficient, hard to understand, different from immutable file upload and download behavior, etc. So I'm putting a bunch of tickets into the "1.8" Milestone. I am not, however, at this time, volunteering to work on these tickets, so it might be a mistake to put them into the 1.8 Milestone, but I really hope that someone else will volunteer or that I will decide to do it myself. :-)