[tahoe-lafs-trac-stream] [tahoe-lafs] #1505: Error in Latency Report on MDMF Retrieve

[tahoe-lafs]

#1505: Error in Latency Report on MDMF Retrieve
-------------------------+-------------------------------------------------
     Reporter:  zancas   |      Owner:  kevan
         Type:  defect   |     Status:  assigned
     Priority:  major    |  Milestone:  undecided
    Component:  unknown  |    Version:  1.9.0a1
   Resolution:           |   Keywords:  mdmf latency statistics response-
Launchpad Bug:           |  time
-------------------------+-------------------------------------------------

Comment (by kevan):

 This specific issue is due to a bug around line 806 of {{{retrieve.py}}}:

 {{{
          # validate this block, then generate the block hash root.
          self.log("validating share %d for segment %d" % (reader.shnum,
                                                               segnum))
          self._status.add_fetch_timing(reader.peerid, started)
          self._status.set_status("Valdiating blocks for segment %d" %
 segnum)

 }}}

 {{{started}}} is the time at which the fetch was started.
 {{{RetrieveStatus.add_fetch_timing}}} wants the elapsed time, though, so
 what we really want is {{{time.time() - started}}}. I'll attach a patch
 with this change.

 While investigating this bug, I noticed several other rough spots in how
 the retrieve process interacts with its status object. In particular,

   * The progress field is set to 0.0 in {{{Retrieve.__init__}}}, then
 never touched again, so it says 0.0% even though the retrieve is finished.
   * The decrypt and decode timing doesn't take into account multiple
 segments, so later segments clobber the timing data for earlier segments
 and the report page only shows decrypt and decode timing data for the last
 segment.
   * The status field says "decrypting" even though the retrieval process
 has finished.

 It's easy enough to be attentive about updating the status and progress
 fields. We can measure cumulative decryption and decoding time to address
 the second point; this requires minor modifications to the retrieve code
 and status code, and a wording tweak on the status template.

 (Do these issues deserve their own ticket? I'll open one if so.)

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1505#comment:2>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage