improve HTTP/1.1 byterange handling

[davidsarah]

The attached patch by Jeremy Fitzhardinge improves byterange handling as follows.

Fix parsing of a Range: header to support:

• multiple ranges (parsed, but not returned)
• suffix byte ranges ("-2139")
• correct handling of incorrectly formatted range headers (correct behaviour is to ignore the header and return the full file)
• return appropriate error for ranges outside the file

Multiple ranges are parsed, but only the first range is returned. Returning multiple ranges requires using the multipart/byterange content type.

[davidsarah]

Improve HTTP/1.1 byterange handling

[zooko]

• return appropriate error for ranges outside the file

Is it disallowed by the standard to go ahead and return results in a case like this? For example, if the user requests bytes 99 through 1000 and the file is only 200 bytes long, can we return bytes 99 through 200?

[davidsarah]

Replying to zooko:

• return appropriate error for ranges outside the file

Is it disallowed by the standard to go ahead and return results in a case like this? For example, if the user requests bytes 99 through 1000 and the file is only 200 bytes long, can we return bytes 99 through 200?

​RFC 2616 section 10.4.17:

   A server SHOULD return a response with this status code if a request
   included a Range request-header field (section 14.35), and none of
   the range-specifier values in this field overlap the current extent
   of the selected resource, and the request did not include an If-Range
   request-header field. (For byte-ranges, this means that the first-
   byte-pos of all of the byte-range-spec values were greater than the
   current length of the selected resource.)

In other words, yes, we should return bytes 99 through 200 in your example. We should only respond with a 416 error if all byte requested ranges start after the end of the resource.

[davidsarah]

Replying to davidsarah:

In other words, yes, we should return bytes 99 through 200 in your example. We should only respond with a 416 error if all byte requested ranges start after the end of the resource.

It looks to me as though the current patch gets this right.

[davidsarah]

The default set of whitespace characters stripped by .strip() is undocumented, so ​use an explicit argument at web/filenode.py line 145, i.e. .strip(' \t') assuming header lines have already been unfolded.

Otherwise looks good. Jeremy: Do you intend this to be applied for 1.7, or are you going to implement multipart/byterange?

[davidsarah]

Assuming that this should be applied for 1.7.

[davidsarah]

Applied in 63b28d707b12202f and a67e745b26130e78. However, I forgot to address comment:4. Jeremy, is it correct to only strip space and tab?

[zooko]

Okay, I'm still waiting for jsgf to answer the question, but the patch is in and the release is going out, so marking this as fixed.