[tahoe-lafs-trac-stream] [Tahoe-LAFS] #2788: cache server information, use later (with overrides)

[Tahoe-LAFS]

#2788: cache server information, use later (with overrides)
------------------------------+-----------------------
     Reporter:  warner        |      Owner:
         Type:  enhancement   |     Status:  new
     Priority:  normal        |  Milestone:  undecided
    Component:  code-network  |    Version:  1.11.0
   Resolution:                |   Keywords:
Launchpad Bug:                |
------------------------------+-----------------------

Comment (by warner):

 In testing out PR281, I noticed that the `yaml.dump()` is emitting some
 pretty horrible-looking YAML, because of the preponderance of unicode
 strings in our announcements (which comes from the fact that we're
 shipping around JSON, in which everything is unicode). One test node I
 looked at had a `NODEDIR/private/introducer_cache.yaml` with entries like:

 {{{
 - ann:
     !!python/unicode 'anonymous-storage-FURL': !!python/unicode
 'pb://w2hqnbaa25yw4qgcvghl5psa3srpfgw3@tcp:127.0.0.1:51309,tcp:10.0.1.43:51309,tcp:10.0.1.57:51309/vucto2z4fxment3vfxbqecblbf6zyp6x'
     !!python/unicode 'app-versions': {!!python/unicode 'Nevow':
 !!python/unicode '0.14.0',
       !!python/unicode 'OpenSSL': !!python/unicode '1.0.2h',
 !!python/unicode 'PyYAML': !!python/unicode '3.11',
       !!python/unicode 'Twisted': !!python/unicode '16.1.1',
 !!python/unicode 'attrs': !!python/unicode '15.2.0',
       !!python/unicode 'cffi': !!python/unicode '1.6.0', !!python/unicode
 'characteristic': !!python/unicode '14.3.0',
       !!python/unicode 'cryptography': !!python/unicode '1.3.2',
 !!python/unicode 'enum34': !!python/unicode '1.1.5',
       !!python/unicode 'foolscap': !!python/unicode '0.11.0',
 !!python/unicode 'idna': !!python/unicode '2.1',
       !!python/unicode 'ipaddress': !!python/unicode '1.0.16',
 !!python/unicode 'platform': !!python/unicode
 'Darwin-15.4.0-x86_64-i386-64bit',
       !!python/unicode 'pyOpenSSL': !!python/unicode '16.0.0',
 !!python/unicode 'pyasn1': !!python/unicode '0.1.9',
       !!python/unicode 'pyasn1-modules': !!python/unicode '0.0.8',
 !!python/unicode 'pycparser': !!python/unicode '2.14',
       !!python/unicode 'pycrypto': !!python/unicode '2.6.1',
 !!python/unicode 'pycryptopp': !!python/unicode
 '0.7.1.869544967005693312591928092448767568728501330214',
       !!python/unicode 'python': !!python/unicode '2.7.11',
 !!python/unicode 'service-identity': !!python/unicode '16.0.0',
       !!python/unicode 'setuptools': !!python/unicode '21.0.0',
 !!python/unicode 'simplejson': !!python/unicode '3.8.2',
       !!python/unicode 'six': !!python/unicode '1.10.0', !!python/unicode
 'tahoe-lafs': !!python/unicode '1.11.0.post96.dev0',
       !!python/unicode 'twisted': !!python/unicode '16.1.1',
 !!python/unicode 'zfec': !!python/unicode '1.4.24',
       !!python/unicode 'zope.interface': !!python/unicode 'unknown'}
     !!python/unicode 'my-version': !!python/unicode 'tahoe-
 lafs/1.11.0.post96.dev0'
     !!python/unicode 'nickname': !!python/unicode 'node-4'
     !!python/unicode 'nonce': !!python/unicode
 'ig7y65ufqcopdq3pul5efixhtf7kqw2i33sp3pxsifryw5gvm27a'
     !!python/unicode 'oldest-supported': !!python/unicode '1.0.0'
     !!python/unicode 'permutation-seed-base32': !!python/unicode
 'w2hqnbaa25yw4qgcvghl5psa3srpfgw3'
     !!python/unicode 'seqnum': 53
     !!python/unicode 'service-name': !!python/unicode 'storage'
     !!python/unicode 'version': 0
   key_s: v0-o5vaij77fgdsj5npijjq3gydm6cbblspirkjqcxh44qvrtyp2moa
 }}}

 That's.. pretty ugly, and I'm not sure I'd want to ask someone to
 copy/edit/paste it into an override file.

 I'm wondering if we should move to JSON instead (with `indent=1` on the
 output so we get newlines and indentation). It's not entirely human-
 friendly, but it wouldn't treat unicode as something magical.

 I noticed that if we used `yaml.safe_dump()` to write the file, intead of
 `yaml.dump()`, then both unicode and bytes get written out as normal
 strings (without the `!!python/unicode` mess). When parsed with
 `yaml.safe_load()` (or unsafe `yaml.load()`), these come back as either
 bytes or unicode (probably depending upon whether it can be represented as
 ASCII or not). That's not ideal (I wish it would be consistently unicode),
 but it'd be easier to work with than the `!!python/unicode` things.

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2788#comment:2>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage