[tahoe-dev] Bringing Tahoe ideas to HTTP

[Brian Warner]

Nicolas Williams wrote:
> One possible problem: streaming [real-time] content.

Yeah, that's a very different problem space. You need the low-alacrity
stuff from tahoe, but also you don't generally know the full contents in
advance. So you're talking about a mutable stream rather than an
immutable file. If Tahoe had such a thing, I imagine the format would
involve proof that each segment was signed by a privkey whose pubkey was
securely identified in the URL/filecap.

> Suppose that given a non-hashed URL you could get a hashed URL variant
> by asking the server (e.g., via an HTTP response header). Then an
> author would keep originals with non-hashed URLs, then run a tool over
> the originals to fix all URLs, then publish the output of that tool.
> Now add public key signatures to verify authenticity at URL fixing
> time, and you're golden.

Hmm, "golden", you say. So, which public key should the end-user use,
eh? Where should they store it? How did they learn it? :-)

You might have a scheme in mind, but I can't guess what it is from that
brief description, so I can't tell if it would provide the sort of
properties that I'm looking for. If the answer is that the pubkey is
provided by the same server, and that the URL doesn't contain its hash,
then you could easily find yourself right back in the SSL+CA pit.

To meet the same goals that we have in Tahoe, the protocol must
guarantee that downloading a given immutable reference (URL) will result
in the exact same bytes every time, no matter who is doing the
downloading, when they do it, or what malicious entities are camped out
on their network or the server. Similarly, for mutable references, it
must guarantee that the downloader will get some series of bytes that
the holder of the writecap (the private key) intended for them to get.

This is the mode-of-thinking that I'm trying to get across. The user's
reference, the URL, *must* contain integrity-checking information. You
can have layers of indirection, as long as there's a verifyable path
from the URL to the final downloaded document, which doesn't rely upon
any third parties (or second parties, for that matter: the web server is
not allowed to make modifications to immutable files either). But the
URL must contain the root, and the protocol must show how you securely
get from this root, through each download step, to the final document.

If the URL contained a site-wide pubkey identifier (or, if like
yurls/web-keys, these were embedded in the DNS name), and used that in
conjunction with SSL, then you could get a coarser version of the
mutable-reference guarantee. In such a scheme, you'd be guaranteed to be
talking to a server that the URL-creator intended you to reach. Anything
further would be up to the server and the people who are able to modify
its files and configuration. (this is what Foolscap does). You'd only
have to update people's URLs when the SSL cert changes. But you remain
vulnerable to server compromise, you can't take advantage of mirrors
(unless they all know the private key, which limits who you'd be willing
to use as a mirror), and instead of hiding the ugly over on the right
edge of the statusbar (where it might be invisible), it would be on the
far left edge, sure to make at least one user ask why they should
download something from http://6a4gu64uacvrzrkbxzyvqqa2sq.lothar.com .

Tahoe must do better than that, since in Tahoe we aren't allowed to rely
upon the servers for anything more than best-effort share storage. And
I'd really like e.g. the firefox upgrade process to do better than that:
upgrade bundles are produced by a very strict process, and are rigidly
immutable (once 3.5.0 is shipped, they'll never publish a different file
also named "3.5.0"). So they're good candidates for a URL which has
immutable integrity guarantees. Furthermore, the channel over which this
URL is normally delivered (i.e. the firefox auto-upgrade mechanism,
which polls their server each day) isn't even read by humans, so
ugly-looking URLs won't even offend anybody's statusbar.

If you had that sort of checking on the client side, the end user's
integrity wouldn't depend upon the behavior of the distribution mirrors.
Tahoe uses this to enable a greater range of servers, letting you safely
store your files on other people's computers instead of being limited to
just your own. For software upgrades, it would reduce the attack surface
significantly.

> Business process transitions take a long time to effect. But yes, this
> would be a very nice one to see happen.

Yeah, in a lot of ways this is all academic. But I think it's always a
good exercise to define and study a solution that we know would
*technically* work, even if there are political or economic reasons that
might preclude an immediate transition. In many cases, the protocol
changes can be applied incrementally, or we can use the design criteria
as requirements on new systems being developed that aren't constrained
by those same limitations. This is why I spent so much time on Petmail
back in 2004, not because I thought anyone would actually use it, but
rather because I wanted people who were designing new messaging systems
to say "it must have security properties and spam-resistance that are at
least as good as what that crazy Petmail thing would have had".

thanks!
 -Brian