Brainstorming possible techniques for Enhancing Findbugs - Supervised Learning

One possible technique for reducing the false positives (FPs) would be to have developers train a machine learning system to recognise occurrences. Under this scheme, the machine would recognise a user action which marked a reported bug as a FP. At this point, there are several pieces of information which would be candidates for use as training data:

- location of the FP
- the type(s) involved
- the type of error reported by Findbugs


At this point I'm only considering a scenario where a single developer works on the codebase. This list is also not comprehensive, just what I can think of on a Friday morning travel into uni.

Location of the FP
The location of the FP includes the complete range of scope, e.g. from project to package to class to method (though there are other locations to be considered, such as constructors or static initialisation blocks). At this point, the system would look for FPs marked in a similar location. Consider a hypothetical FP which reports a potential null dereference of a parameter in a package-private constructor . The calls to the constructor already check for nullness of the parameter in question, and construction is limited to the package. At this point the system could learn that this constructor, within this package, is checked for nullness. If another package private constructor is added, it could potentially use the learned information to remove errors relating to null dereferencing.

class Foo {
    ...
    Foo(Object potentiallyNull) {
        this.bar = potentiallyNull.getBar(); // Error: potential null dereference
    }
}

class FooCreator {
    ...
    void createFoo() {
        ...
        if(potentiallyNull == null) throw new NullPointerException();
        this.foo = new Foo(potentiallyNull); // Null dereference inside Foo constructor is not possible*
    }
}

* ignoring thread safety for this example.

This is example is not the greatest, as there is the possibility of overlooking true errors (who says a new call to the constructor is doing the null checks previous callers were?) but it does demonstrate how location can be taken into account to learn about the codebase.

It may be likely that Findbugs can't always trace back to the calls to check as it may be too expensive. But if this type of learning could trigger an expensive check only in cases where it was likely to remove an FP, it would be ensure true errors were not neglected. The original trigger would be the developer performing an action which said "It's safe to trigger this check in this situation". The problem with this approach is that the code to run such an expensive check would depend on the bug pattern detector, and must be explicitly written for each case - which is perhaps anaethema to machine learning.

The (Java) Types Involved
Re-using the example from above, but reversed (a method call has an error reported that it could be passing in a potential null value to a constructor), the system could learn about the type involved. The code below shows the scenario:

class Foo {
    ...
    Foo(Object potentiallyNull) {
        if(potentiallyNull != null)
            this.bar = potentiallyNull.getBar(); // Null dereference is not possible
    }
}

class FooCreator {
    ...
    void createFoo() {
        ...
        this.foo = new Foo(potentiallyNull); // Error: passing potentially null value to method
    }
}

In this case, the error is an FP - the type Foo checks its parameters for nullness. This again is similar to the previous point, it might be too expensive for Findbugs to follow method calls to check if nullness is checked for. But if the system can learn that the type Foo always checks for nullness, it can stop reporting errors. Or, when the Foo constructor is called, the system can learn to trigger a specific bug detector that will follow the control, knowing that checking a call to Foo is not overly expensive.


Type of Error Reported by Findbugs
The last possibility for supervised learning is slightly simpler. An example of how a machine could learn how to handle certain errors is to do with a class of errors reported by Findbugs: 'bad style' errors. But a developer may have a justifiable reason for ignoring these errors in certain situations. Although a blanket ignore-list for error types is available in Findbugs (I think, note to self: double check), the developer may not want to ignore it everywhere. The system could be used to recognise where a certain error won't be accepted by the developer and begin to remove these.

Learning to reduce FPs by only one of these factors is likely to be quite dangerous. In order to do learn effectively, without removing true errors, it's likely a combination of the location, the code and the type of error would need to be combined.


There will almost certainly be more information which is both available and useful at the point where a system is learning, and these will hopefully become apparent soon. Also, the hypothetical examples here are not necessarily representative of the errors Findbugs actually reports, as I just made them up. Further usage and inspection of Findbugs is will hopefully shed more light on this.

It is my uninformed opinion that supervised learning of the kind described here is likely to be frought with peril, but maybe, unlike the Hoff, I'm just hiding in the darkness, afraid to come into the light.