Snapshots / magic-folder discussion

[meejah]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Magic-Folder and Snapshots
- --------------------------

May 29, 2020. Attendees: exarkun, vu3rdd, nonzen, meejah

Discussing Snapshots ("the Leif design") as sketched out in:
https://github.com/LeastAuthority/magic-folder/pull/133/files

  - "what are we signing?"
    - for initial pass, can we have some kind of stub thing?
    - ...in the interests of getting all other stuff going
  - "stash area / offline-first / etc"
    - how big are the files
    - do we need to snapshot multiple files at once? (e.g. files that "match" from the application)
    - exarkun sounds skeptical about local-snapshots etc
    - what about big files that have small amounts of data changing in a big file == lots of wasted space
    - meejah: we can DESIGN this now, but not actually do the offline-first etc stuff until later
  - exarkun: what about representing things that are common between LocalSnapshot and RemoteSnapshot
  - Author class: split (LocalAuthor vs Remoteauthor), or no?
    - exarkun: PrivateSnapshotAuthor vs PublicSnapshotAuthor ("names are terrible but")
    - could always have the public version of yourself, and only sometimes the private version
  - ram: maybe Authors have no name at all -- just public-keys
    - exarkun: .. and so maybe some other part of the system assigns names to keys
    - meejah: at invite-time the owner of the key could suggest one
  - meejah: what if Author just always has public-key, and we make
    you have a signing-key at upload-time only
  - exarkun: promote "signature" to be higher-level (not "metadata" in RemoteSnapshot)
  - ram: need a version-number somewhere
    - exarkun: the "folder itself" could have a version? (oh, he means the magic-folder)
    - doesn't matter where the version is for Snapshots (if we only version Snapshots)
    - ^ cuts down on "number of versions of things there are"
    - what persistent things do we have?
       - Snapshots
       - folder-DMD
       - collective-DMD
    - both things can have versions, of course
    - can't re-write history easily (because signatures)
    - sounds like we're converging on "folders have versions, but also so do Snapshots"
    - maybe EVERYTHING has a version
      - collective has a version
      - individual DMDs have a version
      - Snapshots have a version
      - where to put that?
    - MUST support reading older versions, SHOULD write latest version
  - DECISION: everything has a version (where to represent this TBD)

  - DECISION: we've decided that having magic-folder try to pre-compute the
    capability-string is bad

  - open question: is there an easier / better way to do testing?

  - the testing we have here is sort of a "twisted way" of doing
    stuff where you use most of the "real" code except IO
    - gets rid of "can't bind port XX" etc
    - "real" http client (e.g. if headers are wrong etc)
    - downside that remains: unit-tests are still "dragging in a lot of code"
    - everything filters through http-land, so if there's server-code
      problems, maybe you get a traceback in the response-body, etc
  - does "adding another layer of abstraction" get around this?
    - if Tahoe provided a Python API, then we could fake that instead
  - "add Python API to Tahoe" is too much pre-requisite for us
  - could do sans-io approach: instead of accepting a "tahoe_client"
    we return some structure that says what IO we want to do .. so
    there's no HTTP client involved in the test because you just have
    objects that describe what you wanted to do...
  - probably this strategy is as good as anything else we can invent
  - WOULD BE NICE to have a sans-io and more-proper HTTP/REST/JSON
    API in Tahoe (that also has fakes etc)

  - before more work on test-suite, exarkun wants to talk about
   Hypothesis + matchers, etc wrt. magic-folder testin

  - ACTION: take out Author names (only public-key)
  - ACTION: timestamps (just piggy-back on existing timestamp metadata in tahoe)
  - ACTION: take out metadata
  - ACTION: need to put the version-number somewhere
  - ACTION: meet on Monday re: magic-folder testin



-----BEGIN PGP SIGNATURE-----

iQFFBAEBCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAl7RMmMRHG1lZWphaEBt
ZWVqYWguY2EACgkQwmAoAxKAaacHYggAsazJETVeITDGZ8U1Bn1URyYH00x3u8tg
VnyNZMulyZ7l43ly+eJ5l0H5Jow+5z9FFkBVfFP93E3YojfAkLpkJx3unGnSrLqB
QJ25sF6ZWVoNG78sCdic2WIibDfB+5l4hANCgwdoYZZUdqGpTytxfcbxcg0F99Yk
utXe0BcInrdDrUoZJrWAj7B79U67yAZpv9pNpisUWq4KvEDCdf6P/VFMgo2bog0V
U5uEZ8w+YMbTq0f6YZUjzXv7XJTMrffmxUpDDVrTKj8hMyG5xEAZeehq6SGIsOlz
jUbUONWJMGVg4tzs9v7bNWR5Ga/56KH4CpzN50I9YnzLJiuvdOHqKA==
=1po0
-----END PGP SIGNATURE-----