Reading Review

I thought it would be useful to review some of the reading that I have done for the project so far. This will allow me to express the related works which I know about, it will also help me when it comes time to write the report (i.e. "Who was it that said that?").


Finding Bugs Is Easy - David Hovermeyer, William Pugh, 2004
The original paper introducing the Findbugs static analysis tool. This paper describes the concept of bug patterns, describes some of the first detectors written for Findbugs, discusses why errors occur, and compare Findbugs to PMD, another static analysis tool.

Finding More Null Pointer Bugs, But Not Too Many - David Hovermeyer, William Pugh, 2007
Report on the challenge laid down to the Findbugs project to find more null pointer analysis.

Using Findbugs On Production Software - Nathaniel Ayewah, William Pugh, J. David Morgenthaler, John Penix, YuQuian Zhou, 2007
Short report on the experience of using Findbugs at Google. Quantifies the issues review, reported and fixed. Also discusses the idea of "True but low impact defects" - quite relevant to this project.

Evaluating Static Analysis Defect Warnings On Production Software - Nathaniel Ayewah, William Pugh, J. David Morgenthaler, John Penix, YuQuian Zhou, 2007
Longer version of the previous report. Includes more detail on the use of Findbugs at Google. Also described the results of applying Findbugs to Sun's JDK and Glassfish. Introduces the concept of 'triaging' the bug reports.

A Brief Introduction Into Machine Learning - Gunnar Ratsch
Brief overview of several machine learning techniques including supervised and unsupervised learning and classification.

Adaptively Ranking Alerts Generated From Automated Static Analysis - Sarah Smith Heckman, 2007 (Published in Crossroads Volume 14, Issue 1)
Very interesting article which uses ranking techniques to classify issues reported by static analysis tools on a spectrum from false positive to true positive. Ranking is based on several factors, including code locality, developer suppression of alerts and alert type. Use of these techniques provided a significant benefit when used to order the alerts shown to developers. Using the techniques 81% of true positive alerts were discovered within the first 20% of inspections, compared to 22% in the first 20% of inspections found with a random ordering of alerts.

Because the techniques address the issue of false positives, and learn from alert suppression, this report is very relevant to the project.

A Report on a Survey and Study of Static Analysis Users - Nathaniel Ayewah, William Pugh, 2008
Reports on the findings of a controlled experiment used to determine the use of Findbugs and another static analysis tool, Fortify SCA. This study found classification of bugs took less than two minutes on average.

Predicting Accurate and Actionable Static Analysis Warnings: An Experimental Approach - Joseph R. Ruthruff, John Penix, J. David Morgenthaler, Sebastian Elbaum and Gregg Rothermel, 2008
A paper which uses statistical techniques to reduce the false positives generated. This case study also uses Findbugs at Google. I think I initially read this paper on an early morning bus journey, and I'm not sure how much of it stuck. Definitely requires a re-read.

Reordering Violations by Analyzing Software History - Sunghun Kim, Michael D. Ernst, 2007
Like the paper described earlier "Adaptively Ranking Alerts Generated From Automated Static Analysis" this paper discusses techniques used to rank error reports. In this case however, the techniques do not adapt to a developer's use of a static analysis tool, instead, it analyses the lifetime of error reports within a codebase and calculates a model for ranking based on how long the error persisted. I.e. Short lived error reports were interpreted to indicate than an error was important, thus fixed quickly, likewise a long lived error was interpreted to be unimportant, hence it would remain.

Tracking Defect Warning Across Versions - Jaime Spacco, David Hovermeyer, William Pugh
Paper describing the techniques used by Findbugs to correlate warnings across different version of software (by different versions it means from build to build, milestone to milestone etc, rather than across major release versions). The instance hash mechanism was found to be an effective way to match persistent error reports.

Which Warnings Should I Fix First? - Sunghun Kim, Michael D. Earnst
Similar to "Reordering Violations by Analyzing Software History", this paper uses software history to classify error reports as false positives and goes on to describe an algorithm to increase the warning precision of Findbugs (and other tools). Using bug-fix information, the techniques described correlate changes which are part of a bug fix to static analysis alerts based on matching location. If a bug fix alters a line which an alert is generated for, the alert is considered to be a true positive. This information can then be used to re-prioritize the warnings.



This list is not exhaustive - exhausting maybe. Now that I've wrote them down it looks like I've read a lot less than I feel as though I have.