id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc	launchpad_bug
822	Web API should use a more reliable, out-of-band means of reporting errors (such as a server connection being lost) during a download	davidsarah		"The discussion of bug #698 (which turned out to be a Firefox bug) turned up a potential integrity problem that can occur if a server connection is lost in the middle of a download via the WUI:

http://allmydata.org/trac/tahoe/ticket/698#comment:1

> The first thing that comes to mind is that a server connection could have been lost in the middle of the download (in this case, after we've retrieved the UEB and some of the hashes, but before we've retrieved the first data block). The web server has to commit to success (200) or failure (404 or 500 or something) before it starts sending any of the plaintext, but it doesn't want to store the entire file either. So it bases the HTTP response code upon the initial availability of k servers, and hopes they'll stick around for the whole download.

> When we get a ""late failure"" (i.e. one of the servers disconnects in the middle), the webapi doesn't have a lot of choices. At the moment, it emits a brief error message (attached to whatever partial content has already been written out), then drops the HTTP connection, and hopes that the client is observant enough to notice that the number of received bytes does not match the previously-sent Content-Length header, and then announce an error on the client side.

> If the application doing the fetch (perhaps the browser, perhaps tiddywiki itself?) doesn't strictly check the Content-Length header, then it could get partial content without an error message.

> There are two directions to fix this:

>    * change the webapi to use ""Chunked Encoding"", basically delivering data one segment at a time, possibly giving the server a chance to emit an error header in between segments: this would let us respond better to these errors

>    * fix the other download-should-be-better tickets (#193, #287) to tolerate lost servers better, which might reduce the rate at which these errors occur 

As pointed out in http://allmydata.org/pipermail/tahoe-dev/2009-May/001724.html , it is possible that the length so far plus the length of the error message, coincidentally equals the expected file length. So even for a web client that diligently checks the Content-Length, there might not be enough information to detect an error. An attacker might try to force this situation (I don't know what their chance of success would be, but probably much higher than trying to attack the crypto).

In any case, the WUI is currently using in-band error reporting, which is problematic because the error message will be treated as data of whatever format the client thinks the content has. This is an integrity issue because the download from the gateway to the client has no cryptographic integrity checking.

To close this bug, find and implement some way to make typical web clients reliably report an error when a download fails part-way through. Alternatively, prove that it isn't possible, and document this as an inherent limitation of the WUI."	defect	new	major	soon	code-frontend-web	1.5.0		integrity error http download