add happiness count to check reports and fix the definition of needs-rebalancing

[davidsarah]

I have a patch for this to go in 1.10 1.11. Note that the description of #1115 said "add servers-of-happiness to reports", and we agreed to do that, but the patch applied for #1115 only fixed a bug in computing count-good-share-hosts.

[david-sarah@…]

In 5532/1.9.2:

[1.9.2 branch] Add comments and a caveat in webapi.rst indicating that the needs-rebalancing field may be computed incorrectly. refs #1115 refs #1784

[david-sarah@…]

In 5886/cloud-backend:

[1.9.2 branch] Add comments and a caveat in webapi.rst indicating that the needs-rebalancing field may be computed incorrectly. refs #1115 refs #1784

[Daira Hopwood <david-sarah@…>]

In b06f8cd8d03a6239:

Add comments and a caveat in webapi.rst indicating that
the needs-rebalancing field may be computed incorrectly. refs #1115, #1784, #1477

Signed-off-by: Daira Hopwood <david-sarah@…>

[daira]

Copying this from ticket:1115#comment:27, with minor updates:

... the value of needs-rebalancing is computed inconsistently between checker.py and filenode.py. In checker.py it is computed as:

# The file needs rebalancing if the set of servers that have at least
# one share is less than the number of uniquely-numbered shares
# available.
# TODO: this may be wrong, see ticket #1115 comment:27 and ticket #1784.
needs_rebalancing = bool(good_share_hosts < len(verifiedshares))

In filenode.py it is computed as

# TODO: this may be wrong, see ticket #1115 comment:27 and ticket #1784.
needs_rebalancing = bool(len(sm) >= verifycap.total_shares)

where len(sm) is equal to count-shares-good. I don't understand this latter definition at all, it looks completely wrong. The definition in checker.py is more subtly wrong because it credits servers that only have duplicated shares as contributing to existing balance. The correct definition should be something like 'iff the happiness count is less than the number of uniquely-numbered good shares available'.

I propose to change filenode.py and checker.py to be consistent, and to both use the happiness-based definition above.