[tahoe-lafs-trac-stream] [Tahoe-LAFS] #2459: webapi doesn't handle Range header correctly

[Tahoe-LAFS]

#2459: webapi doesn't handle Range header correctly
-------------------------+-------------------------------------------------
     Reporter:           |      Owner:  daira
  spreitzer              |     Status:  assigned
         Type:  defect   |  Milestone:  1.10.2
     Priority:  major    |    Version:  1.10.0
    Component:  code-    |   Keywords:  webapi reliability availability
  frontend-web           |  mutable retrieve Range http standards
   Resolution:           |
Launchpad Bug:           |
-------------------------+-------------------------------------------------

Comment (by warner):

 So, it's useful to know that SDMF files, even though they only have a
 single segment, still round up their recorded `segsize` value to be a
 multiple of `shares.needed`. So if you upload a 2-byte file, and your
 `tahoe.cfg` holds the default `k` of 3, then you'll wind up with
 `segsize=3`. If you've changed `k=N=1`, you'll wind up with `segsize=2`.

 `segsize` is used by `mutable/retrieve.py` to decide which segments we're
 going to download. This only really makes sense for MDMF (which can have
 multiple segments), but when MDMF landed, SDMF got the same logic. It is
 also used in `_set_segment()` to figure out how much of each segment
 should be delivered to the consumer. This last function had several bugs,
 and one failure case was to read with offset=0 and size=(some multiple of
 the segsize). In this case, if you're only reading one segment, the data
 would be truncated completely, and nothing would be written to the
 consumer. `web/filenode.py` has already returned a Content-Length header
 by this point, so the HTTP client is expecting to see all the data it
 asked for. If the client is using a persistent connection, then they won't
 notice that the request has finished, and the client will hang.

 It looks like `_set_segment()` would also have had problems if you set the
 offset= to something non-zero: I think it would have returned the wrong
 number of bytes. The problem didn't show up in the two-byte file when it
 was uploaded with `k=3`, because then the two-byte read wasn't a multiple
 of k, and the modulo bug wasn't triggered.

 We rewrote `_set_segment()`, and I think it should now handle all inputs
 correctly.

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2459#comment:9>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage