New Visualizer is insufficiently labelled/documented (plus layout problem)

[zooko]

I'm looking at the output of Brian's New Visualizer, which output François attached to #1264. Attached is a screenshot of what I'm looking at. It is insufficiently labelled and documented, so I can't tell what it means.

(Also there is a problem with layout so that things are scribbled on top of one another into illegibility.)

As far as I know, the only way to learn what this graph means is to read this comment from Brian on ticket #1170:

​http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1170#comment:90

[zooko]

Now that we've finally committed #1200 (package up Brian's New Visualization of immutable download) to trunk and it is destined to be included in Tahoe-LAFS v1.9.0, we should (re-)visit this ticket. Probably the next step is for me to re-evaluate and make my complaints more precise, so I'm assigning this to me and putting it into 1.9.0 Milestone.

[drewp]

This is a type of thing I like to hack on. I even did a layout-lots-of-overlapping-labels project recently which might be relevant. Adding myself to the list in hopes I will be able to contribute.

[zooko]

drewp: Awesome! If you do it soon then it can go into v1.9.0 (being the first release with Brian's New Visalizer). I'm hoping v1.9.0 will go into Ubuntu Oneiric Ocelot 11.10 -- deadline coming up soon. I'll post to the tahoe-dev list about this.

[zooko]

drewp: if you are waiting for me to post my ideas/complaints about the current trunk Brian's New Visualizer, then leave this ticket assigned to me. If you are intending to start editing it yourself without waiting for me, then please accept this ticket so that it will be assigned to you.

[zooko]

drewp: we're probably going to freeze tahoe-lafs trunk and stop accepting new non-bugfix patches for v1.9.0 in about a week.

[drewp]

Here is my plan:

1. Extend the event_json payload to include all the strings that DownloadStatusTimelinePage? is generating for its template, so that the /timeline page doesn't have to be a template at all. This makes the page render use fewer technologies, and it simplifies the other things I'm going to try.

2. Make another /timeline page that doesn't use protovis. I believe that with plain old divs and some jquery magic, I can render a similar timeline but have an easier time doing more elaborate layout. I intend to model it after the firebug 'net' tab, where each event is on its own row (which may be subdivided into the "phases" of the event as appropriate). The rows can be expanded to show details underneath. I also intend to mimic the mouseover behavior of the firebug timeline bars. Firebug has immediate, multi-line tooltips that should be more usable than simple 'title'-attribute tooltips.

3. Depending on the success of #2, I'll see about folding the good parts into the protovis timeline, replacing it, providing links to both, etc.

4. I hope for my all-javascript normal-divs timeline page to be a model for people who want to try other displays, such as a node-centric one showing all the blocks I'm giving out, or a super-compact one that can be superimposed on normal directory pages as a fancy status bar.

I'm trying not to be NIH about this, but after doing a project with D3 (protovis successor) and many other charts with divs, I think I can be much more effective if I port the protovis piece. As a small bonus, the divs layout will work on more browsers.

[drewp]

Here is how far I got: ​http://bigasterisk.com/post/tahoe/timeline2.jpg

The first 'read' line has been expanded to show more detail. Mousing over any timeline row currently shows the whole JSON payload I got for that row.

My row sorting is still weird, but that's because I don't understand data like this:

segment: start: 0, length: 131073, start_time: 1310960867.0037, finish_time: 1310960867.694093

block: start: 87418, length: 43691, start_time: 1310960868.006524, finish_time: 1310960868.03249

The block is read completely after the segment is done? Furthermore, my biggest block offset is only 264047(+170) for a 700kB file. The segment start/length spans look fine.

[drewp]

darcs pull ​http://darcs.bigasterisk.com/tahoe-lafs

will now get my patches, after which you should be able to go to the old /timeline page and append a '2' to the URL. I'd like more feedback about the block issue and also on what else should be shown. Should the 'segment' rows expand to show an explanation of the blocks they used? Can someone write a verbose story template for a DYHB request (e.g. "We contacted _ and asked if it had _; it said it has _ which means we can _")?

[zooko]

drewp: I like the plan from comment:6. Still hoping to get time to review this patch soon.

[zooko]

Brian: what do you think of this ticket -- is it worth putting in an additional visualizer for 1.9? Personally I definitely like the idea, but your Release Manager and deadlines are upon us...

[zooko]

Per ​Brian's mail to tahoe-dev, this ticket didn't make the cut for 1.9. I really hope that drewp works on it some more soon though, so that it can be one of the tickets which kicks off the Tahoe-LAFS v1.10 project. :-)

Having interesting new features in trunk early on in a development cycle really helps, IMO, with making the next release happen sooner and easier.

[zooko]

I opened #1550 to track drewp's new visualizer.

The remaining parts of this ticket are to add documentation and labels to the current visualizer. That seems like it should be in scope for v1.9.0 to me, so I'm moving this ticket back into 1.9.0 from 1.10.0 (but I'm not the Release Manager for v1.9.0).

See also #1169 which is about updating the accompanying text document docs/frontends/download-status.rst​.

[warner]

this is not making it into 1.9 . If anyone feels like adding docs, prove me wrong! :)

[warner]

timeline view on current trunk

[warner]

I've just attached a screen-capture of the timeline view displayed by current trunk. It has seconds along the bottom, and labels on each section. No layout problems on Firefox or Chrome. All comments on this ticket (except for zooko's original description) are about drewp's work, so if this ticket is about my visualizer, then I'm going to kick this back to zooko with the question: is attachment:timeline.png​ sufficiently labeled for you?

[zooko]

attachment:timeline.png​​ is broken, probably due to #1581. I'll try it out myself from trunk (again) and report back.