TippingPoint Digital Vaccine Laboratories
DID YOU KNOW... DVLabs team members gave 20 presentations throughout 2010. Abstracts and slides are available here.

VMWare VMnc Codec Invalid RFB Message Type Heap Overflow Vulnerability

TPTI-09-01: April 6th, 2009


Affected Vendors

Affected Products

Vulnerability Details

This vulnerability allows remote attackers to execute arbitrary code on vulnerable installations of multiple VMWare products. User interaction is required in that a user must visit a malicious web page or open a malicious video file.

Upon installation VMWare Workstation, Server, Player, and ACE register vmnc.dll as a video codec driver to handle compression and decompression of the fourCC type 'VMnc'. This format is used primarily by Workstation to capture remote framebuffer recordings of sessions within a virtual machine. The resulting video is essentially a recorded session of VNC's RFB protocol. In VMWare's implementation the stream consists solely of FrameBufferUpdate messages (message type 0). However, if the message type of one of these blocks is changed to any value greater than 0x03 a size assumption is made which results in faulty math being applied to a pointer used later in a memcpy. This can be leveraged to execute arbitrary code on the host system under the context of the current user.

Vendor Response

VMWare, Inc. has issued an update to correct this vulnerability. More details can be found at:

Disclosure Timeline

    2009-02-13 - Vulnerability reported to vendor
    2009-04-06 - Coordinated public release of advisory


This vulnerability was discovered by: