[tahoe-dev] [tahoe-lafs] #822: WUI should use a more reliable, out-of-band means of reporting errors when a server connection is lost during a download

[tahoe-lafs]

#822: WUI should use a more reliable, out-of-band means of reporting errors when
a server connection is lost during a download
-------------------------------+--------------------------------------------
 Reporter:  davidsarah         |           Owner:           
     Type:  defect             |          Status:  new      
 Priority:  major              |       Milestone:  undecided
Component:  code-frontend-web  |         Version:  1.5.0    
 Keywords:  integrity          |   Launchpad_bug:           
-------------------------------+--------------------------------------------
 The discussion of bug #698 (which turned out to be a Firefox bug) turned
 up a potential integrity problem that can occur if a server connection is
 lost in the middle of a download via the WUI:

 http://allmydata.org/trac/tahoe/ticket/698#comment:1

 > The first thing that comes to mind is that a server connection could
 have been lost in the middle of the download (in this case, after we've
 retrieved the UEB and some of the hashes, but before we've retrieved the
 first data block). The web server has to commit to success (200) or
 failure (404 or 500 or something) before it starts sending any of the
 plaintext, but it doesn't want to store the entire file either. So it
 bases the HTTP response code upon the initial availability of k servers,
 and hopes they'll stick around for the whole download.

 > When we get a "late failure" (i.e. one of the servers disconnects in the
 middle), the webapi doesn't have a lot of choices. At the moment, it emits
 a brief error message (attached to whatever partial content has already
 been written out), then drops the HTTP connection, and hopes that the
 client is observant enough to notice that the number of received bytes
 does not match the previously-sent Content-Length header, and then
 announce an error on the client side.

 > If the application doing the fetch (perhaps the browser, perhaps
 tiddywiki itself?) doesn't strictly check the Content-Length header, then
 it could get partial content without an error message.

 > There are two directions to fix this:

 >    * change the webapi to use "Chunked Encoding", basically delivering
 data one segment at a time, possibly giving the server a chance to emit an
 error header in between segments: this would let us respond better to
 these errors

 >    * fix the other download-should-be-better tickets (#193, #287) to
 tolerate lost servers better, which might reduce the rate at which these
 errors occur

 As pointed out in http://allmydata.org/pipermail/tahoe-
 dev/2009-May/001724.html , it is possible that the length so far plus the
 length of the error message, coincidentally equals the expected file
 length. So even for a web client that diligently checks the Content-
 Length, there might not be enough information to detect an error. An
 attacker might try to force this situation (I don't know what their chance
 of success would be, but probably much higher than trying to attack the
 crypto).

 In any case, the WUI is currently using in-band error reporting, which is
 problematic because the error message will be treated as data of whatever
 format the client thinks the content has. This is an integrity issue
 because the download from the gateway to the client has no cryptographic
 integrity checking.

 To close this bug, find and implement some way to make typical web clients
 reliably report an error when a download fails part-way through.
 Alternatively, prove that it isn't possible, and document this as an
 inherent limitation of the WUI.

-- 
Ticket URL: <http://allmydata.org/trac/tahoe/ticket/822>
tahoe-lafs <http://allmydata.org>
secure decentralized file storage grid