The talk was well received and there was a lot of interest from various parties in getting their hands on the slides. Along side the release of my presentation I prepared this blog entry as the slides are terse and contain some eye charts that, without having heard the talk, could be easily misinterpreted. The slides were presented through Google docs and are now available at:
Mostrame la Guita! Adventures in Buying Vulnerabilities
Please read on for notes on specific slides of interest. If you have any questions or comments feel free to post them as a comment here or contact me directly via e-mail.
Market PricesFor the purposes of my talk I broke the market down into four categories:
- Vendor market: bug bounties offered by Mozilla and Ghostscript for example.
- "White" market: purchase programs where the researcher knows where the information they are selling is going to end up. This includes the TippingPoint ZDI and iDEFENSE VCP who both disclose all purchases to the affected vendor.
- "Grey" market: programs where the final destination of the information sold is unclear. This includes direct / indirect sales to the government as well as some word of mouth programs.
- "Black" market: sales to the "bad" guys.
The wide range of pricing has led many to claim that there is no fair market value for vulnerability research. I disagree and stated so in my talk. The reasoning behind my logic is that you get what you are willing to give. The individual markets offer different pricing but also demand different requirements.
On the white market, and specifically at the ZDI for example we don't need nor do we ask for exploit code. If you can point out a bug we'll take it from there. In fact, frequently we receive very limited crash reports through the ZDI that require significant effort to dissect thoroughly. The white market also offers credit for your discovery and the guarantee that the reported issue will be forwarded to the affected vendor and disclosed in a responsible fashion.
On the grey and black markets credit is completely out of the question. You are selling the rights to your information and expected to never talk about it again. The vendor will not be contacted. Crash reports will not be entertained and nor will buggy or basic proof-of-concept exploit code. Exploits must be polished and weaponized. This is a drastic departure from the white market and many researchers are not comfortable operating in it.
Acquired Research StatisticsSince the launch of the Zero Day Initiative in August of 2005 we have signed up over 1,000 researchers and received almost 1,900 vulnerability reports between them. We only accept critical issues affecting top enterprise software and out of the the bugs we have seen we have purchased over 500 (about 30%) at an average of about 10 a month. That's a lot of bugs. The table presented in the following slide outlines purchase statistics for our most prominent vendors:
The first numeric column shows how many vulnerability purchases we have made which affects the captioned vendor. Microsoft accounts for the largest number, 133, that's 33 Microsoft critical issues we are responsible for disclosing on average per year. The next column shows what our accept ratio is for that specific vendor, recall that our overall is roughly 30%. The final column shows how much of our total ZDI budget was spent securing that specific vendor. As Microsoft accounts for most of our purchases it is no surprise that they account for most of our expenditure as well, 30%. Apple comes in at a distance second accounting for 8% of our total budget.
Vendor Patch Time StatisticsFor each of our most prominent vendors we queried and charted their average time to patch per reported year. Let me explain this more clearly with an example. Between all the bugs we reported to Apple in 2006 their averaged time to patch was 166 days. If we reported a bug to Apple in December of 2006 and it wasn't patched until December of 2008 that 2-year patch time was averaged into 2006. For each year the longest and shortest patch times are highlighted in red and green appropriately. Through this measurement Mozilla comes out on top pretty much across the board:
[ full slide ]
We averaged the individual per year columns into the overall column which places Mozilla at the top with the quickest average patch time of 48 days and Symantec trailing the pack at 307 days. One thing to keep in mind however is that these numbers do not include the currently outstanding issues, which brings us to the next and final column.
The upcoming column displays the average number of days that all issues reported to the vendor have been outstanding for. Up to date listings of all our outstanding TippingPoint and ZDI discoveries are available at:
Longest Patch TimesWe have sliced our vendor data subjectively, by patch time and by days outstanding. The final view I offered the audience is a listing of the top 10 most outstanding bugs:
[ full slide ]
Entries with a + (plus) sign indicate that the issue is still outstanding, otherwise it has been patched. Note that although our recent disclosures to Hewlett-Packard improved their average in the previous slides upcoming column, they hold the title for the top two longest patch times... and counting.
Holding the 3rd and 4th place positions is Microsoft with two issues that affect their Office Web Components (OWC) and along with another OWC issue have all been recently patched in MS09-043 (ZDI-09-054, ZDI-09-055, ZDI-09-056). The two and a half year time to patch triggered some media attention but as I mentioned earlier, patching is not a trivial process in some cases. I verbally covered this matter in my "Behind the Hype" slide stating that the cause for delay on Microsoft's behalf was legitimate and not due to any underlying disorganization.
The presented data was our first unveiling of a vendor "report card". Within the next month or so we intend on creating a permanent home on the ZDI website with all these statistics and more. So check back at www.zerodayinitiative.com and let us know if there is anything in specific that you would like to see.