VMWare VMnc Codec Invalid RFB Message Type Heap Overflow VulnerabilityTPTI-09-01: April 6th, 2009
Vulnerability DetailsThis vulnerability allows remote attackers to execute arbitrary code on vulnerable installations of multiple VMWare products. User interaction is required in that a user must visit a malicious web page or open a malicious video file.
Upon installation VMWare Workstation, Server, Player, and ACE register vmnc.dll as a video codec driver to handle compression and decompression of the fourCC type 'VMnc'. This format is used primarily by Workstation to capture remote framebuffer recordings of sessions within a virtual machine. The resulting video is essentially a recorded session of VNC's RFB protocol. In VMWare's implementation the stream consists solely of FrameBufferUpdate messages (message type 0). However, if the message type of one of these blocks is changed to any value greater than 0x03 a size assumption is made which results in faulty math being applied to a pointer used later in a memcpy. This can be leveraged to execute arbitrary code on the host system under the context of the current user.
Vendor ResponseVMWare, Inc. has issued an update to correct this vulnerability. More details can be found at:
2009-02-13 - Vulnerability reported to vendor
2009-04-06 - Coordinated public release of advisory