[tahoe-dev] split brain? how handled in tahoe -- docs?

[Tony Arcieri]

On Tue, Aug 7, 2012 at 8:04 PM, Two Spirit <twospirit6905 at gmail.com> wrote:

> in split brain scenario, wouldn't both halves have write capabilities?
> unless you made some requirement that more than half of the nodes needed to
> be up in order to write.


Hi everyone,

This is very much a formalized topic of computer science. If you have a
horse in this race I suggest you divest yourself of that horse and spend
your time immersing yourself in the literature on the topic.

What the OP refers to as "split brain" is formally known as "partition
tolerance", and the properties of partition tolerant systems are formally
described by CAP theorem, otherwise known as Brewer's Theorem:

http://lpd.epfl.ch/sgilbert/pubs/BrewersConjecture-SigAct.pdf

If you are not familiar with CAP theorem, I suggest you spend some time
reading about it. If you find the above paper too challenging, you might
consider reading this one instead:
http://ksat.me/a-plain-english-introduction-to-cap-theorem/

CAP theorem is a triangle ala Zooko's triangle: we may only consider two of
the three vertices at a time. No distributed system can satisfy all three
at once. In the context of this debate, we are considering Tahoe to be a
partition tolerant system, and I wouldn't suggest otherwise. The question
becomes: what does Tahoe do in the face of a network partition?

There are only two options:

- Available: Tahoe still accepts writes (and all other operations, but
writes are the hardest) in the middle of a network partition, like it does
today (provided sufficient nodes are available). The current mechanism is
"last writer wins" although there are more sophisticated mechanisms
available if this property is desirable.
- Consistent: Tahoe sacrifices write availability to ensure strong
consistency in the event of a network partition. This can take the form of
several algorithms including a singular master with two phase commit or
Paxos.

These are the only two options. For those who desire "reliability", these
are the only buckets that reliability can be segmented into. So far I have
not heard *any* good arguments towards strong consistency. Instead I have
heard that Tahoe shouldn't "lose data", but a filesystem loses data if you
save over an important file with a junk version. There is nothing specific
to Tahoe as a distributed system that can prevent this: even a strongly
consistent distributed filesystem can't prevent someone with the
appropriate write capability from saving over an important file with
garbage.

So please consider that there are only two options here: consistent or
highly available. Choose wisely.

-- 
Tony Arcieri
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20120808/814e8a67/attachment.html>