mini-Summit report, day 2

[Brian Warner]

Summary of the second day. Brian, Daira, and Nathan met in a coffee
shop, and we:

* built a branch that replaces Tahoe's build process with the "create a
  virtualenv, peep-install everything into it, make bin/tahoe run
  venv/bin/tahoe" technique that Brian used in Petmail. Modulo
  bugs/limitations in Peep, this is "safe", in that it checks
  pre-specified hashes on all tarballs before running any code from
  them. If you managed to get a "good" copy of tahoe's source code, this
  would ensure that you only use "good" code for tahoe's dependencies.

  We actually wrote two separate versions of this (racing against each
  other, dueling-laptops style). My (Brian's) version is published at
  https://github.com/warner/tahoe-lafs/tree/venv , and also strips out a
  lot of code that was annoying me (version string management, almost
  everything in __init__.py and setup.py, setup.cfg, _auto_deps.py).

  (basically, tahoe's current "install stuff into support/" trick was
  something I came up with 7 years ago before virtualenv was a thing,
  and these days we should just use virtualenv. It doesn't even require
  that you have setuptools installed first, since it brings its own copy
  of virtualenv, which includes a copy of setuptools and pip)


* discussed what a ideal green-field programming language would use for
  dependency management. The actual runtime code would identify imported
  modules by hashes, and modules would be deep-frozen, so importing the
  module should be indistinguishable from interpolating the source code
  of the module. Developers would have one phase where they select their
  dependencies (using local petnames for convenience, but their
  development environment knows how to map those petnames to specific
  module hashes), then a separate step where the code gets compiled or
  translated into non-petname hash-based modules, for execution.

  Module authors (and others) would publish signed "mod XYZ would be a
  good replacement for mod ABC" links (edges in a graph, where the nodes
  are module hashes) to indicate newer versions or forks, but these
  would not be followed automatically. Ideally the developer would look
  at the changes (or the recommendations) to make decisions, and for
  not-entirely-compatible changes, the edges would include information
  about how you need to update your calling code to match. This could
  guide a tool (like python's 2to3) that search through your codebase
  for things that need changing. Imagine a screen that says "to update
  from dependency v1 to v2, you must changes the following 4 call
  sites".

  The runtime tool should accept overrides from the ops-folks/admins to
  say "I know you wanted hash 123, but you should accept hash 456
  instead", to deploy security updates faster than the upstream author
  can change their locally-declared dependencies. In general, all
  modules declare their dependencies with strong references (hashes).

  (Brian did some work in this space back in the Jetpack days:
  http://people.mozilla.org/~bwarner/jetpack/components/ )

cheers,
 -Brian