allmydata.test.test_system.Connections.test_rref can fail with "Address already in use."

[exarkun]

On c-locale CI job on CircleCI:

Traceback (most recent call last):
  File "/tmp/tahoe-lafs.tox/py27-coverage/lib/python2.7/site-packages/twisted/internet/defer.py", line 122, in execute
    result = callable(*args, **kw)
  File "/tmp/tahoe-lafs.tox/py27-coverage/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 495, in listenTCP
    p.startListening()
  File "/tmp/tahoe-lafs.tox/py27-coverage/lib/python2.7/site-packages/twisted/internet/tcp.py", line 1363, in startListening
    raise CannotListenError(self.interface, self.port, le)
twisted.internet.error.CannotListenError: Couldn't listen on any:38205: [Errno 98] Address already in use.

​https://circleci.com/gh/tahoe-lafs/tahoe-lafs/18657

This is intermittent and unpredictable. Presumably it is a problem similar to the one described in ​https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3026

[exarkun]

One possibly cause for this is the "main" Tub. If the node configuration gives no explicit listen address, the initialization code for this Tub falls back to allmydata.util.iputil.allocate_tcp_port which is known to result in this kind of failure.

From discussion with meejah on IRC, here's one possible solution.

Add methods to _Config for getting the configured tub location/endpoint. Use these methods in _tub_portlocation instead of going directly to get_config. Implement these in a way that makes it easy to substitute alternate values for the result.

Add a new create_client_for_test helper which is similar to allmydata.client.create_client but which loads the configuration and then pokes test-friendly results for tub location/endpoint into the _Config object. Then it calls create_client_from_config with this modified _Config object.

Last, change most or all of the test suite callers of create_client to use create_client_for_test instead. All tests modified this way will switch from using _tub_portlocation's failure-prone allocate_tcp_port to using the less failure-prone port allocation scheme that is available to the test suite, SameProcessStreamEndpointAssigner.