How Mozilla uses AI to manage Firefox bug reports
The company created a homegrown artificial intelligence tool dubbed BugBug to classify and categorize each bug report.
Some 71% of businesses plan to use AI and machine learning in their security tools this year, though over half aren’t sure what that tech really does, according to Webroot.
Sifting through the hundreds of bug reports and feature requests that Mozilla receives about Firefox each day would be a challenging and time-consuming task for developers. That’s why the company uses artificial intelligence (AI) to triage those bugs so the right people can more quickly see them and fix them.
To tackle the influx of bug reports, Mozilla devised BugBug, a machine learning tool that automatically assigns a product and component to each new reported and untriaged bug, as described in a Tuesday blog post. The product designation classifies the bug based on the actual application, such as Firefox for Android or Thunderbird, while the component breaks it down to a specific feature or subset, such as the PDF Viewer for Firefox.
In the past, the triaging of bug reports was a job assigned mostly to volunteers and some developers, according to Mozilla’s Marco Castelluccio and Sylvestre Ledru, who authored the blog post. But as the number of bugs has grown, that effort became less manageable. Hence, BugBug.
As with any AI, BugBug needed the necessary training and information to learn how do its job properly. From the past two decades worth of bugs, BugBug’s programmers narrowed the focus to just the past two years. But bugs that had already been tagged as resolved wouldn’t do the trick, so the programmers had to roll back each bug to the time it was originally filed. Further, BugBug was fed only components with a number of bugs that was at least 1% of the number of bugs of the largest component. That still resulted in around 100,000 bugs that BugBug had to assimilate as part of its education.
Training BugBug on the two years worth of data requires around 40 minutes on a 6-core computer with 32GB of RAM. But the time it takes BugBug to evaluate the data is in the order of milliseconds, according to Castelluccio and Ledru. That compares with the week it would take to manually assign bugs to the responsible parties.
Since BugBug went into production at the end of February, the AI has triaged around 350 bugs, according to the blog post. The median amount of time for a developer to respond to a triaged bug this way is two days.
Mozilla also has bigger plans for BugBug’s future. One trick the AI needs to learn is how to better identify duplicate bugs. Duplicates are eventually caught by the triage process, but finding them more quickly can help developers who need to resolve them. Another goal is to provide developers with more automated help from BugBug to better detect the type of bug reported, or to detect bugs in which the steps needed to reproduce them are missing. A third objective for BugBug is finding important bugs for a specific release of Firefox as early as possible.
Further, BugBug’s repertoire is currently limited to Firefox-related products. Castelluccio and Ledru said they’d like to extend the AI’s prowess to handle components for other Mozilla products.
Finally, Mozilla is extending a helping hand to other companies who’d like to use AI to track bugs.