[tahoe-dev] storage-club URLs

[Greg Troxel]

Brian Warner <warner at lothar.com> writes:

> I've been thinking and talking a while about moving to "Storage Clubs"
> in tahoe as a grid-membership management system. The idea is use an
> invitation scheme: when you start your Tahoe node, you can either start
> a new grid or accept an invitation to join an existing one. Grids would
> be kept small (below Dunbar's Number[1], maybe 150 participants) to let
> social pressures work to keep the grid healthy, in conjunction with the
> Ostrom's Principles accounting work we're doing.

I understand your point about how grids might be organized, but I don't
follow "moving to".   The pubgrid is anomalous, and there are
volunteergrid, volunteergrid2, plus numerous unadvertised grids.  So it
seems like we're already there.

If you mean have code to make all this work reasonably, as in 

  tahoe invite user at example.com

which packages up the grid params and sends an OpenPGP signed and
encrypted mail with the data, then that sounds cool; now you have to do
that by hand.

> To get files from somebody else's grid, you'd need a gateway of some
> sort. What I've been thinking about recently is how to reference files
> through those gateways with URLs that could also be used by regular
> non-Tahoe HTTP clients, and how they could be used to publish data with
> various tradeoffs between security and legibility.

Here I was initially very skeptical, as I have never really understood
the tahoe community's conflation of storage and publishing.  Perhaps
that's because 

  * I view local storage as essentially free, and reliable and
    survivable storage as hard.

  * I associate the "publishing in tahoe" approach with using a
    particular pubgrid gateway, so it isn't any more reliable than a
    traditiional server

  * I have the impression the publish-in-tahoe activities to date are as
    much a tahoe marketing activity as they are genuinely useful.


After reading the rest of your message, I think what you're proposing
makes more sense, and it needs situations with one of two properties:

  government suppression of publication

  massive datasets with infrequent access, such that putting them on a
  regular server is infeasible.

I think the first use case is more compelling, and then the entire
system has to be designed against that threat model.

I wonder if first addressing distributed introducers is in order; it
would seem that some DHT-type scheme for within-grid discovery might
also work for the publication scenario, and the tahoe-lafs.org dyndns
server becomes a single point of failure -- your scheme would not have
worked in Egypt.

There's another thorny problem lurking, which is that storage accounting
isn't sufficient.  I've been talking to people as I head towards a
private grid, and one person was concerned about network usage.  Someone
here recently expressed concern about a 60G/month usage cap, and this is
an obvious issue when shopping for a VPS provider.
So one really needs accounting for total data transfers.

I wonder if your proposal has somewhat the same properties as a regular
website as a tor hidden service.  Instead, the server is distributed and
it's not imediately obvious who put the bits there.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20110222/e47147b5/attachment.pgp>