replace nevow with twisted.web.template

[zooko]

Users still occasionally have problems with the nevow dependency:

​https://github.com/twisted/nevow/issues/7 (see #2032)

Also, Nevow is unmaintained -- there has been no release since 2009. But it turns out (thanks to Daira for pointing this out to me) that twisted.web.template is the successor to nevow!

​http://twistedmatrix.com/documents/current/web/howto/twisted-templates.html

So, we can reduce our dependencies by one, by using twisted.web.template instead of Nevow!

[daira]

[corrected the module name]

[nejucomo]

I've filed #2032 specifically for the installation issue, which I consider separate from this dependency issue.

[daira]

I made a start on a branch to replace Nevow with twisted.web[.template] at ​https://github.com/tahoe-lafs/tahoe-lafs/commits/1963-replace-nevow.

[remyroy]

I'll have a look at this.

[daira]

The idea behind the branch is to first perform a refactoring that means we still use Nevow, but we import its code indirectly via allmydata.web.common.

I also added a dummy @renderer decorator that will be needed for twisted.web (see ​this doc), but currently does nothing.

Once it's been made to work that way, we can switch the imports in allmydata.web.common to the equivalent twisted.web.template ones in cases where Twisted has a direct equivalent, and add new classes that inherit from or delegate to the Twisted ones to replace any Nevow-specific functionality.

The current state of the branch is that most of the imports in non-test code have been changed, and the @renderer decorator has been added in some of the cases where twisted.web would need it.

[daira]

BTW, please feel free to approach this differently if you see a better way of doing it, @remyroy.

[daira]

Incidentally, pyflakes is your friend when checking consistency of imports.

[remyroy]

From my understanding, Nevow is more than just templating. Replacing Nevow with twisted.web.template is not going to work if the goal is to totally remove Nevow. It would not make much sense to replace just the templating part of Nevow and keep the rest of it.

The big question is what part of Nevow are we using that is not templating and can twisted.web fulfill this in an easy and straightforward way?

This is what we are using from Nevow that is not templating related:

• nevow.appserver
• nevow.appserver.NevowRequest
• nevow.appserver.NevowSite
• nevow.resource_filename

and whatever is under those including idioms and conventions from Nevow. There seems to be a lot of Nevow code under those from my quick reading of Nevow source code.

I've been reading the twisted.web documentation and it feels quite different from how it's done with Nevow. twisted.web.template might have been inspired by the templating system from Nevow but the rest of twisted.web not so much. I also feels like twisted.web is a third world "web framework" coming from something like Flask.

If we are to replace Nevow with something quite different, I would feel uncomfortable to replace it with twisted.web . I'd prefer to use a battle tested modern minimalist WSGI based "web framework" with twisted.web.wsgi.WSGIResource . I like Flask but I know there are also some nice alternatives. We could still keep using the twisted.web.template to minimize the amount of changes with templates if it makes sense.

I'll keep investigating how twisted.web can be *plugged in* and check out if there are other *big projects* who made the switch from Nevow.

Some feedback on this would be appreciated.

Sorry for the long comment.

[zooko]

A big desideratum would be to not add new dependencies to Tahoe-LAFS, and in fact to remove dependencies if possible. Twisted is the one major dependency that we know we can't live without, so it is okay to depend on all features of Twisted, but it is a big issue to add a dependency on any other library.

Someone who knows a lot about this topic is dreid! I'll see if I can get him to comment.

[daira]

Replying to remyroy:

The big question is what part of Nevow are we using that is not templating and can twisted.web fulfill this in an easy and straightforward way?

Yes.

This is what we are using from Nevow that is not templating related:

• nevow.appserver
• nevow.appserver.NevowRequest?
• nevow.appserver.NevowSite?
• nevow.resource_filename

and whatever is under those including idioms and conventions from Nevow. There seems to be a lot of Nevow code under those from my quick reading of Nevow source code.

I've been reading the twisted.web documentation and it feels quite different from how it's done with Nevow.

I think you're probably overestimating the differences. We actually only use a small subset of Nevow, and that subset has almost direct equivalents in twisted.web.

[remyroy]

Replying to daira:

I think you're probably overestimating the differences. We actually only use a small subset of Nevow, and that subset has almost direct equivalents in twisted.web.

Great, I'll keep digging deeper.

[dreid]

Last I checked tahoe-lafs made extensive use of nevow:pattern a concept which is not supported by twisted.web.template.

The other differences, like locateChild vs getChild and render_* being able to return Deferreds are relatively minor.

[remyroy]

I'm sorry but this ticket is overwhelming me. I'll pass it back to Daira.

I wish I could be of more help.

[daira]

Replying to remyroy:

I'm sorry but this ticket is overwhelming me. I'll pass it back to Daira.

No problem; it's better to pass it to someone else than to be overwhelmed. (It was quite a complicated ticket to start with.)

[daira]

Replying to dreid:

Last I checked tahoe-lafs made extensive use of nevow:pattern a concept which is not supported by twisted.web.template.

Hmm, can't nevow:pattern be replaced with nevow:render (with some code refactoring obviously)? Or is that also unsupported?

[Edit: yes `render` is supported.]

[daira]

Or nevow:slot maybe (which is also supported)?

[daira]

​http://twistedmatrix.com/trac/ticket/5040 seems relevant.

[daira]

This is the doc for the feature that was added to twisted.web.template in place of nevow:pattern: ​http://twistedmatrix.com/trac/changeset/32840

[zooko]

The two motivations for this ticket in the initial comment were ​https://github.com/twisted/nevow/issues/7 and that Nevow hasn't had a new release since 2009. As of recently, those two things are no longer true: ​https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2032#comment:18

Should we close this ticket now, or is there another reason to switch from Nevow to twisted.web.template?

[daira]

Even though Nevow has a new release, it still isn't being actively developed. I would prefer to remove it as a dependency.

[daira]

Also we can't actually use the Nevow 0.11.1 release on Windows due to it requiring Twisted >= 13.0, which triggers #2028.

[daira]

​https://github.com/tahoe-lafs/tahoe-lafs/commits/1963-replace-nevow is rebased to master, although it still needs a lot more work.

[sajith]

This seems to be a little outdated by now. There's more recent work on replacing nevow: see the top-level ticket #3244.