If you've ever tried collaborating with other people while reverse engineering a vulnerability your process probably includes some tedious steps, like transferring:
- Your IDB
- Your notes/readme files
- Virtual machines
- Proof of concept files
- IDAPython scripts
We leverage the structures within an executable and IDA's support for interacting with them in order to create a pseudo-filesystem within IDA. The idea is that there is a lot of address space within any given module in IDA. For example, in this screenshot from USP10.dll we can see there are 7 segments defined:
The lowest defined address within a segment is 0x7638000 and the highest is 0x76408000. That means there is free room from 0x0 through 0x7638000 (0x7638000 bytes) and from 0x76408000 to 0xFFFFFFFF (0x89bf7fff bytes) for a total of 0x9122ffff bytes. That's over 2GB, plenty of room to store some data of our own...
So, the idea is to make a new segment (or multiple) to contain arbitrary data of ours. To do this intelligently, we need some way of organizing it so that we can easily rename, resize, move, extract, and insert data. Sounds like we need a filesystem...but, we're lazy (efficient, I mean) and so we can leverage existing code to do this for us (and save space at the same time).
Enter Python's zipfile mode. A zipped file is essentially a filesystem, complete with support for directory structure. Additionally, compression comes for free, thus saving us some disk space. Here we've added our segment:
Now that you all get the idea, let's get into the code. We define an FS class for manipulating the file system:
class FS: ''' File system object - default size of ~2MB ''' def __init__(self, segname=".zip", size=0x200000): self.segname = segname self.size = size self.memhandle = StringIO() done = False # (name, start, end) segs = segment.getSegsInfo() for s in segs: # if our segment is already present, use it if segname == s: print "[*] fs.py: found an existing segment." # start address self.addr = s # the EOF is the first 4 bytes # seek to the actual data ea = self.addr + 4 flags = idc.GetFlags(ea) # get the data bytes = "" while flags != 0: bytes += chr(idc.Byte(ea)) ea += 1 flags = idc.GetFlags(ea) self.memhandle.write(bytes) self.memhandle.seek(0) # save the new size self.EOF = len(bytes) self.save_eof() done = True break # otherwise, make a new one if done == False: print "[*] fs.py: making a new segment." self.addr = segment.alloc(self.size, segname) self.EOF = 0 self.save_eof() zipfs = ZipFile(self.memhandle, mode='w') zipfs.close() self.memhandle.seek(0)
We use a StringIO object so that we never need to touch the disk outside of IDA to create our zip file. Also, you'll notice that each segment we create has a 4 byte size at it's start so that we can easily grab the appropriate amount of data out of it when reading files.
The code that actually creates the segment is here, in our segment.py:
def alloc(size, name): ''' Allocates a segment of the given size. ''' # first lets get the last segment in this binary last_seg_end = idaapi.get_last_seg().endEA # and the first first_seg_start = idaapi.get_first_seg().startEA # now see how many bytes we have from there to 0xFFFFFFFF bytes_high = 0xFFFFFFFF - last_seg_end # now see how many bytes we have # from 0x0 to the first segments start bytes_low = first_seg_start # check where we have more room if bytes_high > bytes_low: print "[*] segment.py: there is room above current segments" new_seg_start = last_seg_end + 0x10000 new_seg_start = new_seg_start & 0xFFFF0000 else: print "[*] segment.py: there is room below current segments" new_seg_start = 0 + 0x1000 idc.SegCreate(new_seg_start, new_seg_start+size, 0, True, 3, 2) idc.SegRename(new_seg_start, name) return new_seg_start
Of course, the above code is useless without the methods of the FS object for storing and reading data:
def get_current_size(self): ''' Returns the amount of bytes currently stored in the fs segment ''' size = 0 ea = self.addr flags = idc.GetFlags(ea) while flags != 0: size = size + 1 ea = ea + 1 flags = idc.GetFlags(ea) return size def store(self, k, v): ''' Stores a file (named k) with value (v) in the segment. Directory paths are allowed. ''' zipfs = ZipFile(self.memhandle, mode='a') current_size = self.get_current_size() len_data = len(v) total_size = current_size + len_data # need to check if our current segment can contain total_size our_seg = idc.SegByName(self.segname) # this is because IDA doesnt delete segments properly segs = list(idautils.Segments()) for s in segs: name = idc.SegName(s) if name == self.segname: our_seg = s break if our_seg == idc.BADADDR: raise SyntaxError("[!] Hrm, segment is BADADDR") our_seg_size = idc.SegEnd(our_seg) - idc.SegStart(our_seg) if total_size > our_seg_size: # we need to resize our segment # XXX: segment.realloc left as an exercise to the reader return False zipfs.writestr(k, v) zipfs.close() self.memhandle.seek(0) self.commit() return True def load(self, k): ''' Retrieves the contents of a file (named k) from the segment file system. ''' try: zipfs = ZipFile(self.memhandle, mode='r') except: return False try: zfile = zipfs.open(k) res = zfile.read() except KeyError: print "[!] File with name '%s' does not exist in the keystore." % k return False self.memhandle.seek(0) return res def save_eof(self): ea = self.addr idaapi.patch_long(ea, self.EOF) return def commit(self): ''' Commits any changes made to the in-memory buffer to the segment. This is automatically invoked on any store() or delete() operation. ''' bytes = self.memhandle.read() self.memhandle.seek(0) ea = self.addr # write the EOF self.EOF = len(bytes) self.save_eof() ea = ea+4 for byte in bytes: idaapi.patch_byte(ea, ord(byte)) ea = ea + 1
Python>filesystem = fs.FS() [*] fs.py: didn't find an existing segment, making a new one. [*] segment.py: there is room above current segments 7. Creating a new segment (76410000-76610000) ... ... OK Python>filesystem.store("my_filename", "A"*200) True Python>fh = open(r"C:\Windows\system32\ntdll.dll", 'rb') Python>ntdll = fh.read() Python>fh.close() Python>filesystem.store("ntdll.dll", ntdll) True Python>filesystem.list_files() ['my_filename', 'ntdll.dll'] Python>afile = filesystem.load("my_filename") Python>print afile[0:10] AAAAAAAAAA Python>print len(afile) 200
Additionally, we've written code that will launch a new GUI interface within IDA for browsing, adding, and deleting files to the filesystem:
The basic code for this is here: fs.py and segment.py. We store this code within the C:\USERS\user\AppData\Roaming\Hex-Rays\IDA Pro directory so that it is available by default when launching IDA (just 'import fs').
If you think about what else you can store within an IDB you'll hopefully come to some interesting new ways to improve your process as we have. We plan on demonstrating this code and much more at our upcoming training Bug Hunting and Analysis at CanSecWest this March. Feel free to join us ;)