[tahoe-dev] #467 static-server-selection UI

[Brian Warner]

On 1/24/11 8:36 PM, Chris Palmer wrote:
> Brian Warner writes:
> 
>> Our assumption is that server reliability (and the user's opinion of
>> it) is a fuzzy concept that the Tahoe client node cannot figure out
>> on its own.
> 
> As usual, I must disagree. :)

Awesome! Just the sort of disagreement I was hoping for! :)

> The problem is not that reliability is too fuzzily-defined, but that
> it is hyper-precisely defined.

Yeah, the user wants "reliable" but doesn't want to think too hard about
what that means. The agent doesn't have enough information to predict
the future. Something has to compromise.

If the agent had a lot of good historical information about server
behavior, or a crystal ball, then it could do some math and know when
the file was sufficiently distributed. The idea of getting the user
involved was to tolerate the lack of such information.

> Why should our reliability depend on the nature or behavior of the
> server operator or on our relationship to her or on our
> bandwidth/latency to her? We seek to be independent of that.

Ah, that gets into the difference in use cases, and the simplifications
that distinguish Tahoe from say MojoNation, and the differences between
cooperating with friends and cooperating with strangers.

 * "One-True-Grid" world-net: a single grid for the whole world,
   designed to scale to millions of nodes. You have no personal
   relationship with any of them: purely economic incentives are
   necessary to overcome the Tragedy Of The Commons and general malice

 * friendnet: participation limited to people you already know, social
   incentives can be used to overcome allocation problems and malice

I'd really like to have a protocol that works well without relying upon
anything as messy as social relationships, favors, or personal
reputation. I think it's possible to build that, but I also think it's a
lot harder than building something that will work for smaller
pre-established groups. In addition to connection-management and
share-placement algorithms that scale way better than what we have now,
a world-scale anti-social grid would need ways to keep people from
consuming storage without paying for it, ways to prevent abuse, ways to
force servers to return the data we gave them earlier. There are cool
economic approaches to many of these, but the effort spent in developing
them (and thus not available to spend elsewhere) was part of the reason
that MojoNation failed.

So yeah, I'd like to not depend upon the behavior of the server
operator, but I suspect it will be a lot easier to make our most common
use-case work if we do.

> Always start with the user story, preferably including UI mockups, and
> nail down the technical bits last.

Yeah, totally.

> ...

> Now, we have a situtation that normal users, expert users, and
> implementers can all understand. All the words have their colloquial
> meanings: status, green, yellow, red, 1, 5, overkill, cost,
> reliability. Advanced users can look up "asynchronous" in the
> dictionary and think, "Ohh, that's why it's always Red for a moment
> after I save something. Cool. One time it stayed Red for like 10
> minutes and I was like whoa! Then it turned Yellow and I was like
> whew."

I *really* like that. It's like how the Dropbox UI works: as long as the
icon is still spinning, it's still doing stuff, and I shouldn't close
the laptop yet.

Octavia's k=1 (i.e. replication) has the nice property that there's no
significant overhead to peeling off new shares. Specifically, you could
upload 4 shares all at the same time, or 2 now and 2 later, and the
overall CPU cost would be about the same. Hm, you know, zfec and sha256
are really fast, Tahoe could probably generate extra shares later
without a lot of overhead, especially if we kept the hash trees around.
We'd have to spend more local disk to keep a whole copy of the file, but
we could conceivably switch to a non-streaming less-parallelized model
in which we prioritize getting k shares uploaded first, and don't spend
bandwidth on the next k shares until after those were in place.

Hm, and also, we could pick a larger default value for N and then just
not bother uploading those shares unless somebody wanted extra
reliability for the file. The nature of zfec is such that you have to
generate the share data, but you discard it right away, and the overhead
probably isn't very big. So say we split our "N" into "Nzfec" and
"Nupload", and we only bother pushing "Nupload" shares (but we could
make new ones, up to Nzfec, at any time).

Then we have a mode where files that you've asked Tahoe to upload are
stored locally until Nupload shares have been placed. And we get a web
UI status light that goes red-yellow-green according to how many shares
have really been placed.

k=1 also makes repair trivial, so improving reliability later (after
you've dropped the local full copy) is cheap.

> Yes, we've pessimized storage efficiency and made fully Green writes
> slow (because many copies), so the design is "dumb" or "wasteful" or
> "embarrassing". But we've improved understandability (for all classes
> of person), read/write performance, and provider-independent
> reliability.

Yeah, I see what you mean. The understandability win is really big. The
k>1 case has a couple of benefits (the 8GB file that Michael Coppola is
trying to upload could take advantage of servers with less than 8GB
free, and for good-uptime servers I think I was able to show that e.g.
4-of-8 is much safer than 1-of-2), but maybe it's possible to achieve
these simplifications without completely giving up on erasure-coding.

But, I'm still not convinced that it improves the provider-independent
reliability aspect. What if the replica you just pushed went to a guy
who only installed a node because he saw the project on slashdot and
will delete it in an hour when he gets bored? Or it went to somebody's
laptop and now it isn't there anymore, except then tomorrow it appears
again? If I only know you as an anonymous NodeID, what incentive do I
have to keep my server running and not just delete your shares when my
drive gets full and I want to rip another CD? Repair is easy, but once
the grid gets larger, the churn will swamp your bandwidth budget. I
don't know how to deal with that without becoming selective about share
placement, and I don't see how to do *that* without injecting some
information into the system, either automatically gleaned from the
future, or somehow solicited from the user, in the guise of social
connections and Ostrom-ish feedback.


great stuff! let's keep it going!
 -Brian