improve new-downloader logging

[zooko]

Updating my blog, using tahoe-lafs v1.8.0b2, I got this incident report file, attached, which includes this line:

13:59:48.590 [1513]: WEIRD Error in Segmentation FAILURE:
[CopiedFailure instance: Traceback from remote host -- Traceback (most recent call last):
Failure: allmydata.interfaces.NoSharesError: no shares (need 3). Last failure: None
]
[INCIDENT-TRIGGER]

[warner]

Talking with Zooko, it sounds like this was a filecap from one grid that was linked into a directory from a different grid. The logs show an ordinary NoSharesError exception. So there's nothing critical to fix here.

The log events could be better, though.. here are some notes for post-1.8.0 about how they should be improved:

• the ShareFinder loop messages show a build-up of received responses, followed by 750ms of successive removals from the pending= list. I think that pending= should be removed as the responses are received, and shouldn't take so long
• The umid="wAlnHQ" in node.py:DownloadNode._start_new_segment needs a logparent
• the umid="EYlXBg" in segmentation.py:Segmentation._error doesn't need to be log.WEIRD when it's due to NoSharesError: i.e. we don't need to make an Incident out of NoSharesError, using log.UNUSUAL is probably more appropriate
• it'd be nice if the Node.read/_fetch_next/get_segment() logparent tree that included the Segmentation._error message were part of the same log tree as the main _Node/ShareFinder tree. Ideally the top-level call to FileNode.read would create a single log tree.

We should check that the NoSharesError made it out to the WUI or CLI, displayed properly (as something human-readable).

[warner]

fix maybe half the logging issues in this ticket

[warner]

that patch adds logparents and cleans up some missing umids. It does not yet fix the log.WEIRD on NoSharesError. Also, the ShareFinder loop messages I mentioned are actually ok: I was misreading the events (if I'd looked at the dump in chronological order, instead of nested, it would have been obvious that the pending= list shrinks upon each receipt of a DYHB response).

So the log.WEIRD issue is the main one left. My patch rearranges some code, so it will definitely wait for after the 1.8.0 release before going in.

[warner]

The logging changes actually landed, with some other improvements, in 00e9e4e. Closing this out.