[tahoe-lafs-trac-stream] [Tahoe-LAFS] #2459: webapi doesnt handle Range header correctly
Tahoe-LAFS
trac at tahoe-lafs.org
Tue Jul 7 20:39:06 UTC 2015
#2459: webapi doesnt handle Range header correctly
-----------------------------------+-----------------------
Reporter: spreitzer | Owner:
Type: defect | Status: new
Priority: major | Milestone: undecided
Component: code-frontend-web | Version: 1.10.0
Resolution: | Keywords:
Launchpad Bug: |
-----------------------------------+-----------------------
Old description:
> The webapi hangs if a GET request with Range header is sent and the Range
> is equal the size or size-1 if cap is a unterminated string.
>
> given that filecap contains 'hi':
> {{{
> 130 sspreitz at sspreitz:~/workspace/tahoefuse/src$ curl -v -r 0-0
> http://127.0.0.1:3456/uri/URI:SSK:rei5f5wqbqycmq2mwtjygglyii:ho4xwwvgbvhwp6xyleafclcjikszuvilb2yzyap4slwgsnmz6joa
> * Hostname was NOT found in DNS cache
> * Trying 127.0.0.1...
> * Connected to 127.0.0.1 (127.0.0.1) port 3456 (#0)
> > GET
> /uri/URI:SSK:rei5f5wqbqycmq2mwtjygglyii:ho4xwwvgbvhwp6xyleafclcjikszuvilb2yzyap4slwgsnmz6joa
> HTTP/1.1
> > Range: bytes=0-0
> > User-Agent: curl/7.37.0
> > Host: 127.0.0.1:3456
> > Accept: */*
> >
> < HTTP/1.1 206 Partial Content
> < Content-Length: 1
> < Accept-Ranges: bytes
> * Server TwistedWeb/14.0.2 is not blacklisted
> < Server: TwistedWeb/14.0.2
> < Content-Range: bytes 0-0/2
> < Date: Sat, 04 Jul 2015 15:02:39 GMT
> < Content-Type: text/plain
> <
> * Connection #0 to host 127.0.0.1 left intact
> hsspreitz at sspreitz:~/workspace/tahoefuse/src$ curl -v -r 0-1
> http://127.0.0.1:3456/uri/URI:SSK:rei5f5wqbqycmq2mwtjygglyii:ho4xwwvgbvhwp6xyleafclcjikszuvilb2yzyap4slwgsnmz6joa
> * Hostname was NOT found in DNS cache
> * Trying 127.0.0.1...
> * Connected to 127.0.0.1 (127.0.0.1) port 3456 (#0)
> > GET
> /uri/URI:SSK:rei5f5wqbqycmq2mwtjygglyii:ho4xwwvgbvhwp6xyleafclcjikszuvilb2yzyap4slwgsnmz6joa
> HTTP/1.1
> > Range: bytes=0-1
> > User-Agent: curl/7.37.0
> > Host: 127.0.0.1:3456
> > Accept: */*
> >
> < HTTP/1.1 206 Partial Content
> < Content-Length: 2
> < Accept-Ranges: bytes
> * Server TwistedWeb/14.0.2 is not blacklisted
> < Server: TwistedWeb/14.0.2
> < Content-Range: bytes 0-1/2
> < Date: Sat, 04 Jul 2015 15:02:50 GMT
> < Content-Type: text/plain
> <
> ^C
> 130 sspreitz at sspreitz:~/workspace/tahoefuse/src$ curl -v -r 0-2
> http://127.0.0.1:3456/uri/URI:SSK:rei5f5wqbqycmq2mwtjygglyii:ho4xwwvgbvhwp6xyleafclcjikszuvilb2yzyap4slwgsnmz6joa
> * Hostname was NOT found in DNS cache
> * Trying 127.0.0.1...
> * Connected to 127.0.0.1 (127.0.0.1) port 3456 (#0)
> > GET
> /uri/URI:SSK:rei5f5wqbqycmq2mwtjygglyii:ho4xwwvgbvhwp6xyleafclcjikszuvilb2yzyap4slwgsnmz6joa
> HTTP/1.1
> > Range: bytes=0-2
> > User-Agent: curl/7.37.0
> > Host: 127.0.0.1:3456
> > Accept: */*
> >
> < HTTP/1.1 206 Partial Content
> < Content-Length: 2
> < Accept-Ranges: bytes
> * Server TwistedWeb/14.0.2 is not blacklisted
> < Server: TwistedWeb/14.0.2
> < Content-Range: bytes 0-1/2
> < Date: Sat, 04 Jul 2015 15:02:56 GMT
> < Content-Type: text/plain
> <
> ^C
> }}}
>
> Apache httpd returns:
> {{{
> sspreitz at sspreitz:~/workspace/tahoefuse/src$ curl -v -r 0-0
> http://people.redhat.com/sspreitz/hi
> * Hostname was NOT found in DNS cache
> * Trying 10.5.19.28...
> * Connected to people.redhat.com (10.5.19.28) port 80 (#0)
> > GET /sspreitz/hi HTTP/1.1
> > Range: bytes=0-0
> > User-Agent: curl/7.37.0
> > Host: people.redhat.com
> > Accept: */*
> >
> < HTTP/1.1 206 Partial Content
> < Date: Sat, 04 Jul 2015 15:06:32 GMT
> * Server Apache is not blacklisted
> < Server: Apache
> < Last-Modified: Sat, 04 Jul 2015 15:06:06 GMT
> < ETag: "2-51a0e030688e9"
> < Accept-Ranges: bytes
> < Content-Length: 1
> < Content-Range: bytes 0-0/2
> < Connection: close
> <
> * Closing connection 0
> hsspreitz at sspreitz:~/workspace/tahoefuse/src$ curl -v -r 0-1
> http://people.redhat.com/sspreitz/hi
> * Hostname was NOT found in DNS cache
> * Trying 10.5.19.28...
> * Connected to people.redhat.com (10.5.19.28) port 80 (#0)
> > GET /sspreitz/hi HTTP/1.1
> > Range: bytes=0-1
> > User-Agent: curl/7.37.0
> > Host: people.redhat.com
> > Accept: */*
> >
> < HTTP/1.1 206 Partial Content
> < Date: Sat, 04 Jul 2015 15:06:36 GMT
> * Server Apache is not blacklisted
> < Server: Apache
> < Last-Modified: Sat, 04 Jul 2015 15:06:06 GMT
> < ETag: "2-51a0e030688e9"
> < Accept-Ranges: bytes
> < Content-Length: 2
> < Content-Range: bytes 0-1/2
> < Connection: close
> <
> * Closing connection 0
> hisspreitz at sspreitz:~/workspace/tahoefuse/src$ curl -v -r 0-2
> http://people.redhat.com/sspreitz/hi
> * Hostname was NOT found in DNS cache
> * Trying 10.5.19.28...
> * Connected to people.redhat.com (10.5.19.28) port 80 (#0)
> > GET /sspreitz/hi HTTP/1.1
> > Range: bytes=0-2
> > User-Agent: curl/7.37.0
> > Host: people.redhat.com
> > Accept: */*
> >
> < HTTP/1.1 206 Partial Content
> < Date: Sat, 04 Jul 2015 15:06:42 GMT
> * Server Apache is not blacklisted
> < Server: Apache
> < Last-Modified: Sat, 04 Jul 2015 15:06:06 GMT
> < ETag: "2-51a0e030688e9"
> < Accept-Ranges: bytes
> < Content-Length: 2
> < Content-Range: bytes 0-1/2
> < Connection: close
> <
> * Closing connection 0
> }}}
New description:
The web-API hangs if a `GET` request with `Range` header is sent and the
end byteRange is equal the size or size-1 if cap is a unterminated string.
given that filecap contains 'hi':
{{{
130 sspreitz at sspreitz:~/workspace/tahoefuse/src$ curl -v -r 0-0
http://127.0.0.1:3456/uri/URI:SSK:rei5f5wqbqycmq2mwtjygglyii:ho4xwwvgbvhwp6xyleafclcjikszuvilb2yzyap4slwgsnmz6joa
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 3456 (#0)
> GET
/uri/URI:SSK:rei5f5wqbqycmq2mwtjygglyii:ho4xwwvgbvhwp6xyleafclcjikszuvilb2yzyap4slwgsnmz6joa
HTTP/1.1
> Range: bytes=0-0
> User-Agent: curl/7.37.0
> Host: 127.0.0.1:3456
> Accept: */*
>
< HTTP/1.1 206 Partial Content
< Content-Length: 1
< Accept-Ranges: bytes
* Server TwistedWeb/14.0.2 is not blacklisted
< Server: TwistedWeb/14.0.2
< Content-Range: bytes 0-0/2
< Date: Sat, 04 Jul 2015 15:02:39 GMT
< Content-Type: text/plain
<
* Connection #0 to host 127.0.0.1 left intact
hsspreitz at sspreitz:~/workspace/tahoefuse/src$ curl -v -r 0-1
http://127.0.0.1:3456/uri/URI:SSK:rei5f5wqbqycmq2mwtjygglyii:ho4xwwvgbvhwp6xyleafclcjikszuvilb2yzyap4slwgsnmz6joa
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 3456 (#0)
> GET
/uri/URI:SSK:rei5f5wqbqycmq2mwtjygglyii:ho4xwwvgbvhwp6xyleafclcjikszuvilb2yzyap4slwgsnmz6joa
HTTP/1.1
> Range: bytes=0-1
> User-Agent: curl/7.37.0
> Host: 127.0.0.1:3456
> Accept: */*
>
< HTTP/1.1 206 Partial Content
< Content-Length: 2
< Accept-Ranges: bytes
* Server TwistedWeb/14.0.2 is not blacklisted
< Server: TwistedWeb/14.0.2
< Content-Range: bytes 0-1/2
< Date: Sat, 04 Jul 2015 15:02:50 GMT
< Content-Type: text/plain
<
^C
130 sspreitz at sspreitz:~/workspace/tahoefuse/src$ curl -v -r 0-2
http://127.0.0.1:3456/uri/URI:SSK:rei5f5wqbqycmq2mwtjygglyii:ho4xwwvgbvhwp6xyleafclcjikszuvilb2yzyap4slwgsnmz6joa
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 3456 (#0)
> GET
/uri/URI:SSK:rei5f5wqbqycmq2mwtjygglyii:ho4xwwvgbvhwp6xyleafclcjikszuvilb2yzyap4slwgsnmz6joa
HTTP/1.1
> Range: bytes=0-2
> User-Agent: curl/7.37.0
> Host: 127.0.0.1:3456
> Accept: */*
>
< HTTP/1.1 206 Partial Content
< Content-Length: 2
< Accept-Ranges: bytes
* Server TwistedWeb/14.0.2 is not blacklisted
< Server: TwistedWeb/14.0.2
< Content-Range: bytes 0-1/2
< Date: Sat, 04 Jul 2015 15:02:56 GMT
< Content-Type: text/plain
<
^C
}}}
Apache httpd returns:
{{{
sspreitz at sspreitz:~/workspace/tahoefuse/src$ curl -v -r 0-0
http://people.redhat.com/sspreitz/hi
* Hostname was NOT found in DNS cache
* Trying 10.5.19.28...
* Connected to people.redhat.com (10.5.19.28) port 80 (#0)
> GET /sspreitz/hi HTTP/1.1
> Range: bytes=0-0
> User-Agent: curl/7.37.0
> Host: people.redhat.com
> Accept: */*
>
< HTTP/1.1 206 Partial Content
< Date: Sat, 04 Jul 2015 15:06:32 GMT
* Server Apache is not blacklisted
< Server: Apache
< Last-Modified: Sat, 04 Jul 2015 15:06:06 GMT
< ETag: "2-51a0e030688e9"
< Accept-Ranges: bytes
< Content-Length: 1
< Content-Range: bytes 0-0/2
< Connection: close
<
* Closing connection 0
hsspreitz at sspreitz:~/workspace/tahoefuse/src$ curl -v -r 0-1
http://people.redhat.com/sspreitz/hi
* Hostname was NOT found in DNS cache
* Trying 10.5.19.28...
* Connected to people.redhat.com (10.5.19.28) port 80 (#0)
> GET /sspreitz/hi HTTP/1.1
> Range: bytes=0-1
> User-Agent: curl/7.37.0
> Host: people.redhat.com
> Accept: */*
>
< HTTP/1.1 206 Partial Content
< Date: Sat, 04 Jul 2015 15:06:36 GMT
* Server Apache is not blacklisted
< Server: Apache
< Last-Modified: Sat, 04 Jul 2015 15:06:06 GMT
< ETag: "2-51a0e030688e9"
< Accept-Ranges: bytes
< Content-Length: 2
< Content-Range: bytes 0-1/2
< Connection: close
<
* Closing connection 0
hisspreitz at sspreitz:~/workspace/tahoefuse/src$ curl -v -r 0-2
http://people.redhat.com/sspreitz/hi
* Hostname was NOT found in DNS cache
* Trying 10.5.19.28...
* Connected to people.redhat.com (10.5.19.28) port 80 (#0)
> GET /sspreitz/hi HTTP/1.1
> Range: bytes=0-2
> User-Agent: curl/7.37.0
> Host: people.redhat.com
> Accept: */*
>
< HTTP/1.1 206 Partial Content
< Date: Sat, 04 Jul 2015 15:06:42 GMT
* Server Apache is not blacklisted
< Server: Apache
< Last-Modified: Sat, 04 Jul 2015 15:06:06 GMT
< ETag: "2-51a0e030688e9"
< Accept-Ranges: bytes
< Content-Length: 2
< Content-Range: bytes 0-1/2
< Connection: close
<
* Closing connection 0
}}}
--
Comment (by daira):
See [https://tools.ietf.org/html/rfc2616#section-14.16] for the HTTP/1.1
spec. The correct response for the `Range: bytes=0-2` case is the whole
file. (A '416 Requested Range not satisfiable' error should not be
returned in that case, because the requested range ''does'' have an
overlap with the file contents. The web-API seems to correctly return a
416 error if the starting byte offset is past the end of the file.)
There is currently no test for Range requests for SDMF in
`test_mutable.py`; only for MDMF. I have added one for SDMF on the
[https://github.com/tahoe-lafs/tahoe-lafs/commits/2659.test-sdmf-version-
partial-read.0] branch.
The behaviour of this test (`Version.test_partial_read_sdmf_*`) is
confusing; it works when the data is 100 bytes or 2 bytes, but not if it
is 90 bytes. Perhaps there is an off-by-one error that only triggers when
the size of the data is a multiple of `k` bytes? (See
[ticket:2462#comment:3] for why that might happen.) But the case that
hangs in this ticket is 2 bytes, which suggests that the test is not
finding the same bug.
--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2459#comment:3>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage
More information about the tahoe-lafs-trac-stream
mailing list