TippingPoint Digital Vaccine Laboratories
DID YOU KNOW... The ZDI has published over 1100 high-risk vulnerabilities since the inception of the program.

Mostrame la Guita!

Recently I presented a talk titled "Mostrame la Guita!" (regional Spanish for "show me the money") at Ekoparty in Buenos Aires, Argentina. The purpose of the talk was to provide transparency into the world of vulnerability markets and for the first time expose the inner workings, statistics and some anecdotal stories behind the TippingPoint Zero Day Initiative (ZDI) to the public. As far as I know, the only other available resource on the matter of markets and pricing is Charlie Miller's 2007 paper "The Legitimate Vulnerability Market". The information presented was garnered through public resources, numerous interviews and my own personal experiences in the marketplace from 2002 to present day.

The talk was well received and there was a lot of interest from various parties in getting their hands on the slides. Along side the release of my presentation I prepared this blog entry as the slides are terse and contain some eye charts that, without having heard the talk, could be easily misinterpreted. The slides were presented through Google docs and are now available at:

Mostrame la Guita! Adventures in Buying Vulnerabilities


Please read on for notes on specific slides of interest. If you have any questions or comments feel free to post them as a comment here or contact me directly via e-mail.

Market Prices

For the purposes of my talk I broke the market down into four categories:
  • Vendor market: bug bounties offered by Mozilla and Ghostscript for example.
  • "White" market: purchase programs where the researcher knows where the information they are selling is going to end up. This includes the TippingPoint ZDI and iDEFENSE VCP who both disclose all purchases to the affected vendor.
  • "Grey" market: programs where the final destination of the information sold is unclear. This includes direct / indirect sales to the government as well as some word of mouth programs.
  • "Black" market: sales to the "bad" guys.
Vendor bug bounties generally fetch less than $1,000 USD per security exposure, for the rest of the markets:


[full slide]

The wide range of pricing has led many to claim that there is no fair market value for vulnerability research. I disagree and stated so in my talk. The reasoning behind my logic is that you get what you are willing to give. The individual markets offer different pricing but also demand different requirements.

On the white market, and specifically at the ZDI for example we don't need nor do we ask for exploit code. If you can point out a bug we'll take it from there. In fact, frequently we receive very limited crash reports through the ZDI that require significant effort to dissect thoroughly. The white market also offers credit for your discovery and the guarantee that the reported issue will be forwarded to the affected vendor and disclosed in a responsible fashion.

On the grey and black markets credit is completely out of the question. You are selling the rights to your information and expected to never talk about it again. The vendor will not be contacted. Crash reports will not be entertained and nor will buggy or basic proof-of-concept exploit code. Exploits must be polished and weaponized. This is a drastic departure from the white market and many researchers are not comfortable operating in it.

Acquired Research Statistics

Since the launch of the Zero Day Initiative in August of 2005 we have signed up over 1,000 researchers and received almost 1,900 vulnerability reports between them. We only accept critical issues affecting top enterprise software and out of the the bugs we have seen we have purchased over 500 (about 30%) at an average of about 10 a month. That's a lot of bugs. The table presented in the following slide outlines purchase statistics for our most prominent vendors:


[full slide]

The first numeric column shows how many vulnerability purchases we have made which affects the captioned vendor. Microsoft accounts for the largest number, 133, that's 33 Microsoft critical issues we are responsible for disclosing on average per year. The next column shows what our accept ratio is for that specific vendor, recall that our overall is roughly 30%. The final column shows how much of our total ZDI budget was spent securing that specific vendor. As Microsoft accounts for most of our purchases it is no surprise that they account for most of our expenditure as well, 30%. Apple comes in at a distance second accounting for 8% of our total budget.

Vendor Patch Time Statistics

For each of our most prominent vendors we queried and charted their average time to patch per reported year. Let me explain this more clearly with an example. Between all the bugs we reported to Apple in 2006 their averaged time to patch was 166 days. If we reported a bug to Apple in December of 2006 and it wasn't patched until December of 2008 that 2-year patch time was averaged into 2006. For each year the longest and shortest patch times are highlighted in red and green appropriately. Through this measurement Mozilla comes out on top pretty much across the board:


[ full slide ]

We averaged the individual per year columns into the overall column which places Mozilla at the top with the quickest average patch time of 48 days and Symantec trailing the pack at 307 days. One thing to keep in mind however is that these numbers do not include the currently outstanding issues, which brings us to the next and final column.

The upcoming column displays the average number of days that all issues reported to the vendor have been outstanding for. Up to date listings of all our outstanding TippingPoint and ZDI discoveries are available at:
This column requires a bit of a caveat as well as the results get skewed with recent bundles of disclosures. For example, we recently disclosed a number of issues to Hewlett-Packard and this brings down their average outstanding patch time.

Longest Patch Times

We have sliced our vendor data subjectively, by patch time and by days outstanding. The final view I offered the audience is a listing of the top 10 most outstanding bugs:


[ full slide ]

Entries with a + (plus) sign indicate that the issue is still outstanding, otherwise it has been patched. Note that although our recent disclosures to Hewlett-Packard improved their average in the previous slides upcoming column, they hold the title for the top two longest patch times... and counting.

Holding the 3rd and 4th place positions is Microsoft with two issues that affect their Office Web Components (OWC) and along with another OWC issue have all been recently patched in MS09-043 (ZDI-09-054, ZDI-09-055, ZDI-09-056). The two and a half year time to patch triggered some media attention but as I mentioned earlier, patching is not a trivial process in some cases. I verbally covered this matter in my "Behind the Hype" slide stating that the cause for delay on Microsoft's behalf was legitimate and not due to any underlying disorganization.

The presented data was our first unveiling of a vendor "report card". Within the next month or so we intend on creating a permanent home on the ZDI website with all these statistics and more. So check back at www.zerodayinitiative.com and let us know if there is anything in specific that you would like to see.

-pedram

Tags:
Published On: 2009-10-29 16:14:54

Comments post a comment

  1. Anonymous commented on 2009-10-30 @ 20:27

    I never new that Digital Armaments DACP was legit. With the misspelling on their webpage and the lack of any discussions about them showing up in google searches I assumed it was a research phising site.

    Goes to show that the grey market is so cloudy its hard to even tell who is actually in the market.

    Thanks for the post!

  2. 0day commented on 2009-11-01 @ 14:45


    I would love to meet someone from gov who could
    set up some 0day with me

  3. Anonymous commented on 2009-11-29 @ 14:22

    Is zdi still in financial trouble like last year? U still haven't announced the csw pwn2own targets.

  4. Pedram Amini commented on 2009-12-09 @ 19:26

    @anonymous The ZDI never has been nor is now in financial trouble. The CansecWest Pwn2Own contest details will be announced within a few weeks or so.


Links To This Post

  1. Brief: Vulnerability sales help secure Microsoft | Hack In The Box
    linked on 2009-10-30 @ 17:59 Show Comment

    The data, part of a presentation given by ZDI’s Pedram Amini last month, shows that about a quarter of the bugs accepted by ZDI since its launch in August of 2005 were vulnerabilities in Microsoft software. The group has only accepted about 30 percent of the 1,900 flaws submitted by researchers for all software.

  2. Brief: Vulnerability sales help secure Microsoft | Random Chaos
    linked on 2009-10-30 @ 18:48 Show Comment

    The data, part of a presentation given by ZDI’s Pedram Amini last month, shows that about a quarter of the bugs accepted by ZDI since its launch in August of 2005 were vulnerabilities in Microsoft software. The group has only accepted about 30 percent of the 1,900 flaws submitted by researchers for all software.

  3. 414C485649 » Ekoparty 2009
    linked on 2009-12-03 @ 05:25 Show Comment

    Mostrame la guita! Adventures in buying vulnerabilities (blog post, slides).   Pedram Amini (profile, twitter, linkedin).


Trackback