[tahoe-dev] some questions, oh and hi :)

[Brian Warner]

Hi!

> On Tue, 8 Jan 2008 13:37:04 -0800
> "Joshua Teitelbaum" <joshuat at gmail.com> wrote:

> From my cursory understanding, the VDrive is just an aggregation of URI's in
> a flat file on the user's disk.

As Zooko pointed out, it's actually a lot more than that. The simple answer
is that there are multiple directories, and each directory is represented by
a serialized dictionary that maps child name to the URI of the child.
Starting with today's 0.7.0 release, these "dirnodes" are distributed through
the grid just like normal data files. The only thing stored on the user's
disk is their private root directory URI.

> Is this flat file protected by any means?

The URIs contained in each directory are encrypted, such that you have to go
through a parent directory to read them. You're responsible for remembering
the root directory URI on your own (which is why the tahoe node writes it to
your local disk, with the file permissions set to prevent anyone else from
reading it). This root directory URI is the "handle" by which you get to
reach the rest of your filesystem.

The more complex answer:

Some of the confusion surrounding the term "vdrive" stems from early versions
of the code which took a very different approach. So let's speak in terms of
"directory nodes" and "file nodes" instead.

The basic Tahoe architecture consists of three layers. The bottommost layer
can be treated as your usual DHT: you put in a key, and you get out a value.
In this case, there are two kinds of slots: mutable and immutable. Most of
the files are stored in immutable slots, using so-called "CHK" identifiers
(also known as URIs, but we're now calling them "read-caps"). These
identifiers represent a specific sequence of bytes. There is a hash of those
bytes in the CHK identifier, so the slot can't be changed once it's been
created and filled. This same property lets us use CHK identifiers to protect
the integrity of the file it represents: if I give you a CHK read-cap, you
can retrieve the bytes and compare the hashes, and nobody else can cause you
to believe that some other sequence of bytes were the ones that I intended
for you to retrieve. Without the read-cap, you can't decrypt the contents of
the file. So access control is easy: just don't give somebody the read-cap.

The other kind of slot is mutable, so you can modify them after creation.
These currently use RSA public keys to provide integrity[1], and have two
kinds of identifiers: the "write-cap" that allows the holder to modify the
slot, and the "read-cap" which only allows the holder to read the contents of
the slot. The read-cap is derived from the write-cap (by hashing), so if you
can write to the slot, you can also read from it, but not vice-versa. If you
have neither the write-cap nor the read-cap, you can't decrypt the contents.
Not even the storage servers which hold these distributed slots can see the
plaintext.

We use mutable slots to hold directory information, so you can
add/remove/rename something in a directory without changing the identifier of
the directory itself (and thus forcing you to modify the parent directory,
etc). There's a special kind of directory-write-cap identifier, and the
related (but weaker) directory-read-cap identifier, both of which just wrap a
mutable-slot write-cap or read-cap and tell the holder to treat the contents
as a serialized dictionary.

Now the part you care about: the "filesystem" (which we've sometimes referred
to as a "vdrive") is a directed graph of directory nodes and file nodes. The
filenodes are terminal nodes (or leaves): they represent a sequence of bytes,
and do not point to anything else. The directory nodes (or "dirnodes") are
just mappings from child name to some other node (either a dirnode or a
filenode). Each node is represented by an identifier (a CHK read-cap or a
directory-write-cap or a directory-read-cap), so the dirnode is just a
mutable slot filled with a serialized dictionary. (it's a bit more
complicated than that, since we want to provide *transitive* readonlyness,
basically we record the full-strength child URI under one key, and the
read-only form of the child URI under a different key, and you can only get
the first key if you have the directory-write-cap. I was about to point you
at the docs, but it turns out that docs/dirnodes.txt in the recent 0.7.0
release is completely out of date, so I'll be updating that tomorrow).

Any metadata associated with the directory entries[2] is encrypted along with
the read-cap of the child, so it is protected in the same way as the actual
files.


So, anyways, the security paradigm is that each dirnode gives you access to
its children, but doesn't give access to anyone else. You can read everything
that you can reach through this graph. The fact that it's a *directed* graph
(i.e. neither files nor directories include a reference to their parent)
means that you can give away access to a single file or directory and not
expose everything else. The storage servers only see ciphertext and don't get
to see the plaintext of your files or directories, so they can't access
anything.


> Like does the user have to present any kind of credentials in order to
> "unlock" this VDrive, or is it stored in plaintext on the user's hard drive?
> From my understanding, yes, the contents of the URIs are indeed encrypted
> (in convergent fashion, or not), but I think having some kind of assurance
> and or protection against unwanted eyes on the metadata would be good and
> interesting.

The big thing about the 0.7.0 release was to move the directory information
off of the old "vdrive server" (which was a single, centrally-located server)
and distribute them into the grid with the same high-availability properties
as the file data itself. Directories in 0.6.1 were vulnerable to the central
server going away: directories in 0.7.0 are highly available. But we retain
the same security properties as before: the only way to see the contents of a
directory is to have the read-cap for it, and that is protected just like the
file contents are.


Hope that helps clear up some questions. The docs/ directory has a lot of
notes on the architecture, but the new dirnode layout is out-of-date, so
ignore anything you read in docs/dirnodes.txt until you see a version that
stops mentioning the old "vdrive server".


cheers,
 -Brian


[1]: we plan to move to DSA keys with a system-wide shared modulus to achieve
     faster dirnode creation and shorter caps, see ticket #217 for details
     and a picture of the design
[2]: we haven't implemented metadata yet, see ticket #117 for our plans