Time to make a final choice...

I feel I've neglected and ignored this project over the past few months. The pressures of four classes, all with significant programming coursework has left me with little spare time. But now the semester is done, and I've taken a couple of days for myself to relax (by which I of course mean upgrading from Kubuntu Hardy to Karmic), I can begin to make moves with the project.

The most vital thing now, and this is echoed by the feedback I received from my second marker, Richard Connor, is that I've got to make a decision on what to begin implementing.

Although I've not made a final decision, I have made a decision on one of my earlier brainstorms, in that I definitely won't be implementing it. Sorry Triage mode, you're out.

I didn't realise at the time, but the current version of Findbugs (1.3.9-rc) already has a triage mode of sorts. Running against a "social networking backdrop", users can designate a priority which is submitted to the "Cloud". Although I feel there could be improvements to the UI, it's not going to provide a decent enough return on investment.

I can't remember if I had read either of the two papers[1][2] when I suggested a triaging mode. My suggestion was based on a hypothetical improvement which could shave off X amount of seconds from each bug classification. Using the data collected in [1], which found bug classification took less than two minutes on average, this approach may still have been a possibility. However, the analysis in [2] suggested that the average triage took 8 minutes. The former collects its data from experiments, the latter from the use of Findbugs in production (at Google).

For me, the analysis using Findbugs on production software, with industry engineers rather than students/academic staff, is more significant. And when it takes around 8 minutes to triage a bug, there can be a reasonable assumption made that the gains from improving the UI (maybe a few mouse clicks or key presses) is not going to put much of a dent in that time. Even if I was being wildly generous to the concept of an optimized UI, and said it could shave off one minute from that time, which is very unlikely, I'd still say the ROI wasn't high enough.

No, the goal, as alluded to in another post is that the system should be able to get a better idea of what bugs to report. In the cases where it can make a pretty good guess that the bug report is irrelevant, it could shave 8 minutes off an 8 minute triage time.

That's the kind of ROI I'm aiming for.


[1] A Report on a Survey and Study of Static Analysis Users - Nat Aweyah, William Pugh
[2] Predicting Accurate and Actionable Static Analysis Warnings: An Experimental Approach - Joseph R. Ruthruff, John Penix, J. David Morgenthaler, Sebastian Elbaum, and Gregg Rothermel