id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc	launchpad_bug
485	server incident reporting	warner	somebody	"The current version of Foolscap now has code to report ""incidents"", which are logs of the events that led up to some high-severity event. There is also an API to subscribe to hear about these events.

We need to build a gathering mechanism for these events. The storage servers on a commercial grid should report Incidents to this gatherer, and the gatherer can then summarize and deliver them via email, or an RSS feed.

See also #484, which addresses a similar issue on the client side.

Things to be wary of: overloading the gatherer, bounding the sender's queue size, thundering herds if many servers experience problems at the same time.

The gatherer's interface should have a way to manage display of incidents: human operators should be able to say ""yes, I know about that one"", and not be distracted by well-known problems for which a fix is in progress. This kind of implies a table of incident disposition (new, still-troublesome, ignored), and maybe eventually a mechanism to automatically classify new incidents as being in a known category (""another 42 instances of Bug!#123 were seen today"").
"	task	closed	major	1.3.0	operational	1.1.0	fixed