<div dir="ltr">Here's a set of trade-offs that I currently find intriguing:<div><br></div><div style>Non-Global Use Case: The One Grid technology is opt in. By default tahoe-lafs behaves similar to its current incarnation. The One Grid implementation could conceivably even be a separate code base, interface, process, etc...</div>
<div style><br></div><div style>Storage Management Policy: The design is "hands off" and does not alter share placement, accounting, garbage collection, etc...</div><div style><br></div><div style>Efficiency / Latency / Reliability: We sacrifice all of these in favor of the other trade-offs.</div>
<div style><br></div><div style>Incentives: Grid Universalist fanatics run extra software / infrastructure to facilitate The One Grid because of their ideological brainwashing in the education camps.</div><div style><br>
</div>
<div style>Mental Models: "It's just like normal Tahoe-LAFS, except whenever you create a new cap, after doing so, the One Grid technology publishes details of how to find appropriate storage servers to the magic global lookup mechanism in the sky; when you request a cap, the One Grid technology finds the appropriate storage servers, then it feeds the result to normal Tahoe-LAFS to fetch or update the content."</div>
<div style><br></div><div style>Implementation Cost: The design is "hands off" so that it can be developed semi-independently from Tahoe-LAFS. Failure to release will not affect mainstream Tahoe-LAFS users. Its bugs and runtime costs, etc... will not affect mainstream Tahoe-LAFS users.</div>
<div style><br></div><div style><br></div><div style>Given those trade-offs, it sounds like the implementation:</div><div style><br></div><div style>a. Lives in a separate codebase from tahoe trunk. One option is a fork, but I'm kind of inclined to have a separate codebase and separate process and separate web interface which then talks to a stable version of mainline Tahoe. We might advocate for minimal changes to mainline Tahoe to facilitate this, such as a new web api request of the form:</div>
<div style><br></div><div style>"Please give me the list of storage servers currently storing $CAP on your grid."</div><div style><br></div><div style>-and:<br></div><div style><br></div><div style>"Please fetch/update $CAP using this list of storage servers: [...]"</div>
<div style><br></div><div style>b. Implements a DHT which maps $CAP to [list of storage servers].</div><div style><br></div><div style>c. Provides a webapi that looks just like mainstream Tahoe-LAFS, but whenever a new cap is created (by delegating to a mainline gateway), it publishes the cap into the DHT, and whenever retrieving a cap, it looks up storage servers from the DHT, then delegates to a mainline gateway.</div>
<div style><br></div><div style><br></div><div style>Thoughts?</div><div style><br></div><div style>Regards,</div><div style>Nathan</div><div style><br></div><div style>"""</div><div style>One Grid to rule them all, One DHT to find them,</div>
<div style>One Grid to transfer all data and in the Local Node erasure-decode and decrypt them.</div><div style>"""</div><div style><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Thu, Jun 27, 2013 at 8:41 PM, Callme Whatiwant <span dir="ltr"><<a href="mailto:nejucomo@gmail.com" target="_blank">nejucomo@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Dear Distributed Secure Storage fans,<div><br></div><div>The time has come to shed our conspiratorial pretense of being nothing but small disparate bands of neighborly do gooders sharing storage with their friends. It is time to reveal to the world our true conquest of world domination and announce our intent to create The One Grid to Rule Them All!</div>
<div><br></div><div>What on earth is he talking about, you may be asking? I'm talking about extending the Boring Old Web (BOW) with the ossm-sauce that is Tahoe-LAFS so that we can spring our despotic vision of provider independent security on the unsuspecting subjects of our new world order!</div>
<div><br></div><div>Capabilities should be universally shareable in the same contexts as URLs in general! This is our vision! Dissent on this manner is heretical (but we will still openly accept mailing list posts, ticket submission and herding, documentation help, patches, and any other contributions from such counter-revolutionaries).</div>
<div><br></div><div><br></div><div>Ok, enough thespianism:</div><div><br></div><div>I personally want to be able to email or tweet or inscribe on papyrus a URL containing a read cap, and anyone who sees that and has Tahoe-LAFS version Glorious Future installed should have a reasonable chance to retrieve the content.</div>
<div><br></div><div>How could this be designed and implemented? There are myriad trade-offs to consider:</div><div><br></div><div>Non-global use case: A fair number of users probably want a *non global grid* such as for their own enterprise or collective, so it would be nice to avoid dumping more complexity on them. On the other hand, if the features were opt in, that adds configuration complexity.</div>
<div><br></div><div>Storage Management Policy: Some schemes would automate share placement using some fancy DHT related technology, but that would interfere with individual users and storage operators from deciding where their data lives.</div>
<div><br></div><div>Efficiency / Latency / Reliability: Some schemes would add a separate global resolution system, but this adds round trips (latency), reliability.</div><div><br></div><div>Incentives: A non-global grid often has "natural incentives" (same company, same friend group, etc..) A global system has different incentive issues. See [1].</div>
<div><br></div><div>Mental Models: Some schemes may be complex to understand, reducing the ability of many users to anticipate the effects of their choices or whom or what they are relying upon and for what features.</div>
<div><br></div><div>Implementation Cost: Some schemes may be complex to implement, increasing the time to implement, the chance of bugs, etc...</div><div><br></div><div><br></div><div>There are probably many other tradeoffs I fail to account for, I just want to get the ball rolling on this. Let's be really clear about the costs of various approaches.</div>
<div><br></div><div>As a concrete step, I propose a new ticket keyword: "globalcaps" for any ticket related to making capabilities globally usable.<br></div><div><br></div><div><br></div>
<div>Regards,</div><div>Comrade Nathan</div><div>Grid Universalist</div><div><br></div><div>References:</div><div><br></div><div>[1] The Tahoe-LAFS community has a great awareness of incentive issues. This is a good starting page:</div>
<div><br></div><div><a href="https://tahoe-lafs.org/trac/tahoe-lafs/wiki/Ostrom" target="_blank">https://tahoe-lafs.org/trac/tahoe-lafs/wiki/Ostrom</a></div></div>
</blockquote></div><br></div>