webdav frontend

[warner]

I suspect that the most convenient front-end in the long run is going to be a WEBDAV server. Most operating systems know how to mount DAV shares, we wouldn't need any special per-platform code, and our webapi server is pretty close to being a DAV server anyways.

OTOH there is some funky locking that must be done, and there might be some impedance mismatches that need to be resolved.

[warner]

I looked a little bit at this, and I think it's feasible. Our existing RESTful interface is pretty close to what a webdav client appears to use. The main difference is that directory listing uses a special PROPsomething method (instead of a special GET), and servers can implement locking. The PROPFIND/PROPSET methods use a bizarre XML syntax (as befits a microsoft-inspired protocol :-).

The best way to pursue this will start by building the apple calendar server (​http://trac.calendarserver.org/), which includes a webdav server written in twisted (using web2). If we could get this server running inside a tahoe node, I think the rest would be pretty easy. Getting twisted.web2 into a tahoe process when it isn't in the regular twisted distribution is a hassle, but should be possible. (web2 was recently removed from twisted SVN, so it could spend some time maturing. The challenge is to manage the namespaces properly). web2 doesn't play well with Nevow, but I think we might want a separate port/server for the webdav stuff anyways, so that wouldn't be a problem.

Once that's running, the question is how people should use it. One possibility that might make regular users a bit more comfortable would be to give the tahoe node a table of username/password/rootcap, and have the webdav server implement HTTP Digest authentication, and teach the twisted.vfs layer that when a request shows up for /foo/bar.txt via username Bob, that means it should turn that into a GET for /uri/[BOB's-rootcap]/foo/bar.txt . Another possibility is that users could tell their clients to mount http://localhost:8123/dav/ROOTCAP, which would be more capabilitiesish.

After we build this, we might want to look into caching dirnodes for some limited period of time (5 seconds?), since at least Mac OS-X likes to look for a lot of hidden OS-X -specific files in each directory (to decide how to arrange the icons, where to open the window, etc).

The unix tool 'cadaver' is a DAV client that acts a lot like the old FTP client, and is useful for running tests. The easiest way to get a server running is to enable mod_dav in apache.

[davidsarah]

See wiki:GSoCIdeas#WebDAVSupport

[zooko]

There exists a very old project to add webdav to twisted: ​http://akadav.sourceforge.net/

And a branch within twisted: ​http://www.mail-archive.com/twisted-web@twistedmatrix.com/msg02169.html

[davidsarah]

Replying to zooko:

There exists a very old project to add webdav to twisted: ​http://akadav.sourceforge.net/

This has a dependency on ​PyXML, which is unmaintained.

And a branch within twisted: ​http://www.mail-archive.com/twisted-web@twistedmatrix.com/msg02169.html

This code depended on Twisted web2, which is dead. ​Progress on reanimating this branch seems slow.

I think that ​wsgidav is probably a better choice, because it ​seems to be actively maintained.

[warner]

Yay! A new option for WebDAV!

But, doesn't WSGI usually means synchronous+blocking? twisted.web can call out to WSGI applications, but usually it's hard for a WSGI server to call out to Deferred-returning APIs. But, maybe there's a way to hack it in. It'd be awesome to get DAV working.

[davidsarah]

Replying to warner:

Yay! A new option for WebDAV!

But, doesn't WSGI usually means synchronous+blocking?

Yes. However ​this appears to say that someone got wsgidav working with Twisted's wsgi-container (see code in comment 10).

As you say, there might be problems calling back into Tahoe's APIs, since wsgi-container works by using deferToThread. But maybe you can have a WSGI thread block until the main Tahoe thread has responded to a message from it.

[davidsarah]

​Zope2 apparently has a WebDAV server, ​as does Zope3 (​source of Zope3's implementation).

It appears to be fairly tightly dependent on the rest of Zope (​example for COPY/MOVE), but is still worth looking at for inspiration, and for how it has solved client-compatibility issues.

[zooko]

I just discovered ​pyWebDAV. I heard about it because it had a security vulnerability announced on ​http://lwn.net .

[amontero]

Call to Twisted/WSGI/webserver wizards: Could not #2144 help achieve WebDAV, just using an out-of-the-box web server?