Using the Cloud to Paint a Picture of the Project

In a previous post I talked about how a system for reducing false positives should get to know the project. From some of the reading listed here there are several descriptions of the methods used to reduce false positives. Some, techniques, such as Heckman's[1] use developer feedback to directly influence the models used to rank alerts. Others, such as Kim and Ernst[2] mine the change history of a project to increase warning precision.

With regards to Findbugs, there may be a way to combine both developer feedback and change history. In the latest version, Findbugs provides a socal-networking backdrop to static analysis with its introduction of "The Cloud". Through the Swing UI, users of Findbugs can categorise alerts and have them persisted and shared through the Cloud. This functionality could provide the basis for a supervised learning session which could be used to rank alerts, as described by Heckman. Perhaps most appealingly, it could be used retrospectively. Findbugs' analysis also provides mechanisms[3] to recognise when a bug was first introduced, and when it last appeared. Accessing this information, which provides both the bug type and location, could be used as a change history to apply some of the techniques described by Kim and Earnst.

There are two ways I would like to see this information being used. First and most obviously, the UI could be adapted to show the highest ranked alerts first. Users could be told that alerts are ranked, and once they are consistently seeing false positives, they can stop analysing. A "skimming the cream" approach.

The second way, is to use the model from a Findbugs analysis to realise when a newly introduced bug should be fixed, and alert the developer as soon as possible. This was a point raised by Nat Aweyah* in an email correspondence: that the most cost-effective way to fix bugs was to raise it with the person who wrote the code, as soon after they wrote it as possible. In a continuous integration setting, armed with the knowledge that a bug alert will be acted upon, the system could "break the build". The developer who checked in the breaking code would then be alerted to the error shortly after writing it, keeping the context fresh in their mind. This is a double-edged sword though. While raising alerts so visibly could be the most cost-effective way to fix them, the negative effect of false positives would be amplified hugely. As far as I can tell, The Cloud already allows insertion into bug tracking systems, breaking the build would be a step up from that, and assuming enough confidence in the analysis, I doubt it would be much hassle to adapt the system to break the build, for instance with a plugin for ant/Maven/Hudson.

I think I'm going to try to prototype some of these analysis models. The Cloud idea which is intended for Findbugs 2.0 looks like a deep well for this type of integration.

* A PhD student at Maryland who has coauthored papers on Findbugs with Bill Pugh.
[1] Adaptively Ranking Alerts Generated From Automated Static Analysis - Sarah Smith Heckman, 2007
[2] Which Warnings Should I Fix First? - Sunghun Kim, Michael D. Earnst
[3] Tracking Defect Warning Across Versions - Jaime Spacco, David Hovermeyer, William Pugh