Jetpack Add-On to Predict Likelihood of Bug Fix in Bugzilla

Written by: Diederik van Liere on July 7, 2009.

Last week I published a blog post where I demonstrate that it is possible to make a prediction how likely it is that a particular bug will be solved. Using an extensive data set from the Firefox project I sh0wed that bug reporter experience, bug reporter past success rate, the presence of a stack trace and whether the bug reporter is a Mozilla affiliate are strong and positive predictors whether a bug will be fixed. The full blog post is available here.

Gerv Markham suggested, in a follow-up discussion based on that blog post, to make a GreaseMonkey extension to augment the Mozilla Bugzilla site. I have never worked with GreaseMonkey but I have a little bit of experience with Jetpack. So, I decided to give it a shot. If you do not have Jetpack yet, please install it now!

The Jetpack Add-on calculates the probability that a bug report will result in a bugfix. A bug fix is defined as a bug that receives the “FIXED” status in Bugzilla. WORKSFORME, INVALID, DUPLICATE, INCOMPLETE, WONTFIX are not defined as a bug fix. The idea is that bug triagers and Mozilla developers will spend less time on ‘poor’ bug reports and instead focus their scarce resources on bug reports that are ‘better’. The predicted probabilities range approximately between 10 and 70%. As a rule of thumb, I would take bug reports with a probability of 35% and higher serious. Hopefully, I will be able to make better predictions in the future and I am looking forward on feedback on this topic in particular!

The “Predict Likelihood of Bug Fix Jetpack Add-On” does the following:

Extract email address of bug reporter and send it to www.network-labs.org
Lookup past success rate and number of bug reports by bug reporter from a simple sqlite database
Return json answers to Jetpack
Search if bug report contains stacktrace
Determine if bug reporter is Mozilla affiliate
Calculate probability of bug fix
Display results

The dashboard with the predicted likelihood of a bug fix will only be displayed when the url contains ‘bugzilla.mozilla.org/show_bug.cgi’ in the string. If everything works fine, then you should see something like this when browsing a bug report:

Predict Bug Fix in Action

I would love to receive feedback! One disclaimer about my code: I am a mediocre developer, I am sure there are 1000 ways to make the code more elegant and efficient. If you have suggestions to improve the code, please let me know so I / we can improve the quality of the code!

Download

Version 0.1 (070709) bugdashboard.js can be be found here (make sure that you have installed Jetpack)

Share and Enjoy:

Bugzilla, Firefox, open source, software

This entry was posted on July 7, 2009, 3:11 am and is filed under Firefox. You can follow any responses to this entry through RSS 2.0. You can leave a response, or trackback from your own site.

Lee

Just FYI, this doesn't seem to work with the Dusk theme for bugzilla.

Lech Deregowski

This is very cool but also at the same time it's hilarious as you're attempting to calculate a positive prediction for one bug among thousands (that's brilliant!). But there's one issue when calculating the possible outcome when a reporter (lets use me for this example) is not a coder but in this case just a lowly reporter. At the moment it appears that it's making the assessment purely on the reporters stats without using any other available (valuable) data (I know, I know, it's still version 1 :]) which is kind of funny because if you look at a bug that has been fixed the percentage is still a lowball guess which is not 100%.

Some thoughts on this would be to include some more data and base the initial average on not only the basic history of the reporter but also including how many of the bugs reported are duplicates, invalid/nofixes or have been fixed. I'm not sure if you can mine that deep for historical data, but if it's possible then awesome.

Then rebuild an estimate based on that. Scoring positive for fixed, negative for invalid and duplicates and building a percentage from total tickets found and maybe recalculate again based on the bug status. If you want to go further factor in the averages for users participating, the number of responses by them and attached files as patches or test cases. With the right kind of scoring, you could certainly build a better bugzilla user to bugs fixed profile and have better prediction in the process.

Just my two cents...

NetworkLabs