mini-Summit report, day 2

[Callme Whatiwant]

Responses inline:


On Wed, Jul 2, 2014 at 1:05 PM, Brian Warner <warner at lothar.com> wrote:
>
> Summary of the second day. Brian, Daira, and Nathan met in a coffee
> shop, and we:
>
> * built a branch that replaces Tahoe's build process with the "create a
>   virtualenv, peep-install everything into it, make bin/tahoe run
>   venv/bin/tahoe" technique that Brian used in Petmail. Modulo
>   bugs/limitations in Peep, this is "safe", in that it checks
>   pre-specified hashes on all tarballs before running any code from
>   them. If you managed to get a "good" copy of tahoe's source code, this
>   would ensure that you only use "good" code for tahoe's dependencies.
>
>   We actually wrote two separate versions of this (racing against each
>   other, dueling-laptops style). My (Brian's) version is published at
>   https://github.com/warner/tahoe-lafs/tree/venv , and also strips out a
>   lot of code that was annoying me (version string management, almost
>   everything in __init__.py and setup.py, setup.cfg, _auto_deps.py).

For contrast, the other implementation took a "change as little as
possible" approach rather than a "rip out all the stuff this will
supercede" approach.  The "change as little as possible" approach
effectively just bolts on new functionality, leaving older redundant
functionality sitting there, so it's certainly not clean, but
hopefully doesn't break any existing packaging uses.

We filed a ticket to follow up on this approach:

https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2255

>
>   (basically, tahoe's current "install stuff into support/" trick was
>   something I came up with 7 years ago before virtualenv was a thing,
>   and these days we should just use virtualenv. It doesn't even require
>   that you have setuptools installed first, since it brings its own copy
>   of virtualenv, which includes a copy of setuptools and pip)
>
>
> * discussed what a ideal green-field programming language would use for
>   dependency management. The actual runtime code would identify imported
>   modules by hashes, and modules would be deep-frozen, so importing the
>   module should be indistinguishable from interpolating the source code
>   of the module. Developers would have one phase where they select their
>   dependencies (using local petnames for convenience, but their
>   development environment knows how to map those petnames to specific
>   module hashes), then a separate step where the code gets compiled or
>   translated into non-petname hash-based modules, for execution.
>
>   Module authors (and others) would publish signed "mod XYZ would be a
>   good replacement for mod ABC" links (edges in a graph, where the nodes
>   are module hashes) to indicate newer versions or forks, but these
>   would not be followed automatically. Ideally the developer would look
>   at the changes (or the recommendations) to make decisions, and for
>   not-entirely-compatible changes, the edges would include information
>   about how you need to update your calling code to match. This could
>   guide a tool (like python's 2to3) that search through your codebase
>   for things that need changing. Imagine a screen that says "to update
>   from dependency v1 to v2, you must changes the following 4 call
>   sites".
>
>   The runtime tool should accept overrides from the ops-folks/admins to
>   say "I know you wanted hash 123, but you should accept hash 456
>   instead", to deploy security updates faster than the upstream author
>   can change their locally-declared dependencies. In general, all
>   modules declare their dependencies with strong references (hashes).
>
>   (Brian did some work in this space back in the Jetpack days:
>   http://people.mozilla.org/~bwarner/jetpack/components/ )
>
> cheers,
>  -Brian
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at tahoe-lafs.org
> https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev