If you've ever tried collaborating with other people while reverse engineering a vulnerability your process probably includes some tedious steps, like transferring:

1. Your IDB
2. Your notes/readme files
3. Virtual machines
4. Proof of concept files
5. IDAPython scripts
6. PCAPs
7. ...

After doing this several hundred times, we came up with a little solution we thought you might all find useful.

We leverage the structures within an executable and IDA's support for interacting with them in order to create a pseudo-filesystem within IDA. The idea is that there is a lot of address space within any given module in IDA. For example, in this screenshot from USP10.dll we can see there are 7 segments defined:




The lowest defined address within a segment is 0x7638000 and the highest is 0x76408000. That means there is free room from 0x0 through 0x7638000 (0x7638000 bytes) and from 0x76408000 to 0xFFFFFFFF (0x89bf7fff bytes) for a total of 0x9122ffff bytes. That's over 2GB, plenty of room to store some data of our own...

So, the idea is to make a new segment (or multiple) to contain arbitrary data of ours. To do this intelligently, we need some way of organizing it so that we can easily rename, resize, move, extract, and insert data. Sounds like we need a filesystem...but, we're lazy (efficient, I mean) and so we can leverage existing code to do this for us (and save space at the same time).

Enter Python's zipfile mode. A zipped file is essentially a filesystem, complete with support for directory structure. Additionally, compression comes for free, thus saving us some disk space. Here we've added our segment:




Now that you all get the idea, let's get into the code. We define an FS class for manipulating the file system:

class FS:
	'''
	File system object
		- default size of ~2MB
	'''

	def __init__(self, segname=".zip", size=0x200000):
		self.segname = segname
		self.size = size
		self.memhandle = StringIO()

		done = False
		# (name, start, end)
		segs = segment.getSegsInfo()
		for s in segs:

			# if our segment is already present, use it
			if segname == s[0]:
				print "[*] fs.py: found an existing segment."

				# start address
				self.addr = s[1]

				# the EOF is the first 4 bytes
				# seek to the actual data
				ea = self.addr + 4
				flags = idc.GetFlags(ea)

				# get the data
				bytes = ""
				while flags != 0:
					bytes += chr(idc.Byte(ea))
					ea += 1
					flags = idc.GetFlags(ea)
				
				self.memhandle.write(bytes)
				self.memhandle.seek(0)

				# save the new size
				self.EOF = len(bytes)
				self.save_eof()
				done = True
				break

		# otherwise, make a new one
		if done == False:
			print "[*] fs.py: making a new segment."
			self.addr = segment.alloc(self.size, segname)
			self.EOF = 0
			self.save_eof()
			

		zipfs = ZipFile(self.memhandle, mode='w')
		zipfs.close()
		self.memhandle.seek(0)

We use a StringIO object so that we never need to touch the disk outside of IDA to create our zip file. Also, you'll notice that each segment we create has a 4 byte size at it's start so that we can easily grab the appropriate amount of data out of it when reading files.

The code that actually creates the segment is here, in our segment.py:

def alloc(size, name):
	'''
	Allocates a segment of the given size.
	'''

	# first lets get the last segment in this binary
	last_seg_end = idaapi.get_last_seg().endEA

	# and the first
	first_seg_start = idaapi.get_first_seg().startEA

	# now see how many bytes we have from there to 0xFFFFFFFF
	bytes_high = 0xFFFFFFFF - last_seg_end

	# now see how many bytes we have 
	# from 0x0 to the first segments start  
	bytes_low = first_seg_start

	# check where we have more room
	if bytes_high > bytes_low:
		print "[*] segment.py: there is room above current segments"
		new_seg_start = last_seg_end + 0x10000
		new_seg_start = new_seg_start & 0xFFFF0000
	else:
		print "[*] segment.py: there is room below current segments"
		new_seg_start = 0 + 0x1000

	idc.SegCreate(new_seg_start, new_seg_start+size, 0, True, 3, 2)
	idc.SegRename(new_seg_start, name)
	
	return new_seg_start

Of course, the above code is useless without the methods of the FS object for storing and reading data:

	def get_current_size(self):
		'''
		Returns the amount of bytes currently stored in the fs segment
		'''
		size = 0
		ea = self.addr

		flags = idc.GetFlags(ea)

		while flags != 0:
			size = size + 1
			ea = ea + 1
			flags = idc.GetFlags(ea)

		return size

	def store(self, k, v):
		'''
		Stores a file (named k) with value (v) in the segment. Directory paths are allowed.
		'''
		zipfs = ZipFile(self.memhandle, mode='a')

		current_size = self.get_current_size()
		len_data = len(v)

		total_size = current_size + len_data
		
		# need to check if our current segment can contain total_size

		our_seg = idc.SegByName(self.segname)

		# this is because IDA doesnt delete segments properly 
		segs = list(idautils.Segments())
		for s in segs:
			name = idc.SegName(s)
			if name == self.segname:
				our_seg = s
				break

		if our_seg == idc.BADADDR:
			raise SyntaxError("[!] Hrm, segment is BADADDR")

		our_seg_size = idc.SegEnd(our_seg) - idc.SegStart(our_seg)

		if total_size > our_seg_size:
			# we need to resize our segment
			# XXX: segment.realloc left as an exercise to the reader
			return False
			
		zipfs.writestr(k, v)
		zipfs.close()
		self.memhandle.seek(0)
		self.commit()

		return True

	def load(self, k):
		'''
		Retrieves the contents of a file (named k) from the segment file system.
		'''

		try:
			zipfs = ZipFile(self.memhandle, mode='r')
		except:
			return False

		try:
			zfile = zipfs.open(k)
			res = zfile.read()
		except KeyError:
			print "[!] File with name '%s' does not exist in the keystore." % k
			return False

		self.memhandle.seek(0)
		return res 

	def save_eof(self):
		ea = self.addr
		idaapi.patch_long(ea, self.EOF)
		return

	def commit(self):
		'''
		Commits any changes made to the in-memory buffer to the segment. This is automatically invoked on any store() or delete() operation.
		'''

		bytes = self.memhandle.read()
		self.memhandle.seek(0)

		ea = self.addr

		# write the EOF
		self.EOF = len(bytes)
		self.save_eof()

		ea = ea+4

		for byte in bytes:
			idaapi.patch_byte(ea, ord(byte))
			ea = ea + 1

Example usage:

Python>reload(fs)

Python>filesystem = fs.FS()
[*] fs.py: didn't find an existing segment, making a new one.
[*] segment.py: there is room above current segments
  7. Creating a new segment  (76410000-76610000) ... ... OK
Python>filesystem.store("my_filename", "A"*200)
True
Python>fh = open(r"C:\Windows\system32\ntdll.dll", 'rb')
Python>ntdll = fh.read()
Python>fh.close()
Python>filesystem.store("ntdll.dll", ntdll)
True
Python>filesystem.list_files()
['my_filename', 'ntdll.dll']
Python>afile = filesystem.load("my_filename")
Python>print afile[0:10]
AAAAAAAAAA
Python>print len(afile)
200

Additionally, we've written code that will launch a new GUI interface within IDA for browsing, adding, and deleting files to the filesystem:








The basic code for this is here: fs.py and segment.py. We store this code within the C:\USERS\user\AppData\Roaming\Hex-Rays\IDA Pro directory so that it is available by default when launching IDA (just 'import fs').

If you think about what else you can store within an IDB you'll hopefully come to some interesting new ways to improve your process as we have. We plan on demonstrating this code and much more at our upcoming training Bug Hunting and Analysis at CanSecWest this March. Feel free to join us ;)

--
Aaron