storage_client.py should use HTTP if it's in the announcement

[itamarst]

Part of #3901. Blocked on #3902. Probably implement in the same branch as #3912?

IServer has a bunch of Foolscap-specific assumptions that need fixing. Do we need IServer, or is it really just NativeStorageServer? and the interface can go away?

More broadly there's some basic impedance mismatch between connection-oriented foolscap and request-oriented GBS. E.g. the webui wants to show users how many servers they're "connected to". There may be some test code that tries to wait until "enough" servers are "connected to".

The share placement algorithm (which servers to put a share on) wants to operate on a set of "connected" servers.

So step 1 is... investigate that and write up design. Or something

[itamarst]

And of course it needs to support different IStorageServer implemetnations.

[itamarst]

As a substitute for the concept of connection, perhaps the client could send an occasional request to the HTTP server. If that happens frequently enough (below the threshold of various timeouts) in practice the HTTPS connection will stay open and the checks will be extra cheap and fast.

[itamarst]

Looking at the code again, seems like we actually want to:

1. Rename NativeStorageServer to FoolscapNativeStorageServer.
2. Create a new HTTPNativeStorageServer which also implements IServer. It would implement the concept of being "connected" by sending HTTP requests once a minute or something.
3. When an announcement arrives, it is checked for NURLs, and that determines which of the two IServer implementations are used. This is done in StorageFarmBroker._make_storage_server.

[itamarst]

Should probably do this together with #3912.

[itamarst]

Another question: how to deal with the fact there are multiple NURLs?

[itamarst]

Reading further, the above design is... insufficient, but maybe still workable. NativeStorageServer has a bunch of extra methods not listed in IServer, and in practice StorageFarmBroker relies on them. So also need to add them to IServer, or maybe figure out how to get rid of them, or get rid of IServer.

[itamarst]

Thinking about implementation some more... there'd be a lot of duplicated functionality if you had two IServer implementations, especially given the need for backwards compatibility. So perhaps a better approach is making NativeStorageServer choose an HTTP IStorageServer client if there are NURLs.

Edit: I tried this. It was not a success, it doesn't actually reduce code duplication, nor does it simplify hings meaningfully, too much of NativeStorageServer is tied to Foolscap. So having a separate IServer implementation for HTTP is probably the way to go.

[GitHub <noreply@…>]

In ac524a3/trunk:

Merge pull request #1225 from tahoe-lafs/3783-storage-client-http

Make storage_client support HTTP

Fixes ticket:3783