[tahoe-dev] Tahoe-LAFS as web server file backend?

[Alfonso Montero López]

Hi all.

I'm watching and playing with tahoe to use it as a family/personal
backup solution. Nothing working yet, just playing by now :) I tried
once with the code but too difficult, too little time by then. At
least, I added some novice notes to the docs, along the way. But, now
I'm at it, I would like to say that I think Tahoe-LAFS is a brilliant
piece of software with great ideas in it worth watching evolve. Thanks
for the awesome work!

Now I'm at another different thing. Thinking in ditributed/clustered
web serving, I wonder what would be the best way to use Tahoe-LAFS as
the file backend, if possible. I mean, you throw a bunch of webservers
at the front, say Apache or nginx and point their webroots to a
locally stored tahoe cap and serve/run files and scripts from there
(PHP, for instance). Let's leave MySQL for another story :)

Mounting would need to be read/write and performant enough for running
apps such as CMS and other complex scripts. I still don't have a true
sense/measurement of its performance by my current experience, and I'm
not sure of it being possible to be handled.

I know tahoe has its webapi but seems not easily pluggable into
apache, without much coding (too far for me). I need to do it by
gluing some pieces, and don't know where to look next. I suppose it
has to be mounted in the filesystem r/w. In my experience don't think
the FTP frontend being stable and current enough to handle it, let
alone the complexity and layer performance hit. I recall seeing it
somewhere being used as some web app backend, but the app was
tahoe-specifically coded, I think.
Inbetween of those some Apache-plugged reverse proxying module + WebAV
trick could be the way, but my knowledge in that area is still
limited. Or maybe WebDAV is currently working well enough to be used
with davfs fuse.
Any tried and tested stable mounting solution anybody can recommend?
Any WebDAV/fuse/whatever layer (the lighter, the better) anyone can
point to? Creating a package mounting a tahoe root in the appropiate
place in the filesystem for the webserver makes it a tempting
low-hanging fruit :)
The file usage would be more reads than writes, since lots of software
depend on DBs for really frequently used data and (perhaps?) file
writes will be majority a single object with less frequent updates.
There will be updates, anyway. But usage-wise, maybe I'm too CMS
biased, anyway. Maybe it's not that relevant, but just for
completeness.
The write performance/consistency/concurrency/name-your-issue of
several web servers has to be taken in account the first. I don't have
any clue about its overhead and implications. But at least, it may be
good enough to having a hot-standby or point-in-time secondary web
server, anyway. Or maybe there is a better/easier way of doing this
without tahoe-LAFS that I just don't know about. But if finally it
makes sense for me, it will have a lot of sense to discuss it in
public, too. So, pardon my verbosity.

BTW, I should confess that about the hosted apps I'm a bit more biased
to the Drupal CMS, since with its pluggable storage backends, even in
a reduced version, tahoe might have sense for it as a file storage.
And this would be a big pool of developers to attract their interest,
the least. Might make for a howto. But I prefer to keep it general if
possible.

So, before exploring any further route, I would like to ask. How the
bright minds I've seen here by lurking for some time would address
this scenario? Since there an overwhemling number of moving parts and
possibilities here for me, nobody with better knowledge than people in
this list can provide feedback about the whole use case.

It might be achievable maybe in a bunch of config files or scripts?
(grid-updates smartness comes to mind). I would be happy in
collaborating/sharing my work in a repo to make it a valid use case,
when the time comes (no python by now, just bash scripting). But at
least, if it is feasible, it makes for sure worth seeding some docs in
the wiki to open discussion about it, and who knows, compiling some
repos if some passerby decides it's worth going for it. Combined with
some spice such as already existing Puppet manifests could make it a
trully awesome tahoe-LAFS based solution, IMO.

Many thanks in advance.
Regards,

--
Alfonso M. L.