There was a time, not so long ago, where Apple was the plucky upstart. They weren't the second-largest music retailer in the United States. They didn't hold a virtual monopoly on portable music players, and they didn't capture nearly half of the high-end laptop market.

Apple was instead, a geek's company. They were open and friendly, flexible and more than a little quirky. Sure, the fact that large portions of their code are open source is great, and certainly something of which I approve, but they've definitely started playing their cards a bit closer to their chest, at least on security issues.

At this year's just-passed Black Hat, I was very much looking forward to attending two talks discussing security in Apple's products. One was to be a general discussion of Apple's security practices, and was to be presented by several members of Apple's security staff. That would have been, I'm sure, interesting, but I was far more excited about the discussion of an apparent security flaw in Apple's FileVault technology by Charles Edge.

Both talks were unceremoniously yanked from Black Hat by Apple. I don't know what the details of either talk would have been (had I known, I wouldn't have seen the use of attending, now would I?), but I love FileVault, and I'm sad I wasn't able to see the talk.

For the non-Mac users out there, FileVault is Apple's disk encryption technology. In true Apple fashion, it attempts to be easy to use for the vast majority of users, at the expense of some higher-level features for the real power users. Like Time Machine, Apples simple-but-limited built-in backup solution, FileVault aims to be disk encryption that's so easy to use, people actually use it.

FileVault is easy in part because you don't say what you do and don't want to encrypt - you don't get a choice. It's your home directory, your whole home directory, and nothing but your home directory. This is accomplished by the simple expedient of placing your home directory in a disk image and loopback-mounting that disk image when you login.

This was actually cited as one of FileVault's big File  Faults - most things in your home directory don't need to be encrypted. That's true, of course. The gigabytes upon gigabytes of photos in my home directory don't need to be encrypted. I'm sure if someone steals my laptop, the photos of my fiancee and I in Japan aren't what they're after. So, spending the CPU power to maintain the encryption on those files is seen as pointless.

I'm going to defend Apple on this one, though. Here's why I like that my whole home directory is encrypted: I don't know where everything interesting is. Well, okay, I do, but most people probably don't. Quick: Where does Safari store its cached images? How about Mail, where does it store your mailboxes? And iChat, where are those remembered chats?

See, if FileVault had made me pick and choose what directories to keep encrypted, chances are I would have forgotten to encrypt Safari's cache, or one of my mailboxes. Or maybe I would have remembered to do those (and known where to look), but some application I download reads a sensitive document and caches somewhere I wasn't expecting. Now my security was just completely compromised. Hooray!

So, I actually very much approve of FileVault's all-or-nothing approach, because I don't want to have to track down where all my applications store all of my potentially-sensitive data. Also, I like the elegance of the approach.

So, yes, I use FileVault. I was therefore eager to see what sort of flaws FileVault suffered from. Some of them are well-known: FileVault doesn't encrypt things other than home directories, for example. Well, I'm not in the habit of storing sensitive information outside of my home directory, so that's not really a flaw for the vast majority of users. Besides, Mac's are multi-user now; you need per-user encryption.

Other flaws are more practical: Apple's Time Machine backup software doesn't really work with FileVault. That is is a legitimate criticism, and I really do hope they fix that in the upcoming Snow Leopard edition of Mac OS X.

The biggest flaws, however, deal with how encryption is actually performed. While the disk image is encrypted using the state-of-the-art Advanced Encryption Standard (AES, also known as Rihndael), it is done using Cypher Block Chaining mode, making it potentially easier to guess the unencrypted contents of the disk. Also, keys are stored in a less-than-perfect manner, using a 3DES (Triple DES) system.

Additionally, there exists a Master Key that can be used to decrypt FileVaults for users who have forgotten their passwords. These master keys end up having an effective key size of only 72 bits in some situations. In other words, its not as hard as some people might think to circumvent FileVault encryption.

Not as hard, but still pretty difficult; it would take some specialized hardware a potentially very long time to crack a FileVault volume.

Additionally, there is some concern that encryption keys are stored insecurely in memory, so if someone grabs my laptop while it's asleep or the contents of memory are otherwise available, they could get a big leg up in bypassing the encryption.

Several weaknesses in FileVault have already been addressed by Apple: when a user migrates to FileVault, the old contents of their home directory are "securely' wiped, by overwriting the files with random data numerous times before deleting them. This is not a perfect solution, but it is worlds better than the old method of simply unlinking the files' directory entries and leaving the data around for any forensic investigator to find.

Another flaw was address when Apple provided the option of encrypting the on-disk swap file. This, of course, has reasonably high performance implications, but it does address the issue. (I'm actually pretty pleased to note that, with 4GB of memory, I rarely, if ever, touch swap. Of course, I run my Mac as a Unix workstation primarily, so I generally don't run anything more memory-demaning than vi.)

(That's mostly a joke, of course - I run iTunes and such, but I rarely have more than 2GB of memory in use at any one time; swap is rarely touched.)

In any case, the encryption implementation weaknesses in FileVault are well documented (see Applebaum's and Weinmann's analysis at the NSA here: http://crypto.nsa.org/vilefault/23C3-VileFault.pdf).

Key stealing and cryptanalysis of the algorithms used are likewise well known and widely published.

All this makes me wonder what the FileVault Black Hat talk could have been about. Was it simply a rehash of the known vulnerabilities? Was it something more esoteric, like remanence issues in RAM (also a well-known flaw, and not just with Apple)?

The problem is that I don't know. No one, save for Apple, Mr. Edge, and whomever they chose to tell knows the details.

The big evil corporation guilty of being notoriously tight-lipped about security vulnerabilities used to be Microsoft. However, Microsoft has become considerably more forthright of late: with their new MAPP initiative (http://www.microsoft.com/presspass/press/2008/aug08/05-08BlackHat08PR.mspx) and their rather wry sense of humor about vulnerabilities, Microsoft has actually become one of the most open major vendors when it comes to details about security flaws. This is nothing but a good thing.

Apple has the whizbang factor sewn up, especially with Joe Consumer: Apple is just cool, while Microsoft is seen as stodgy at best. But one of the fanboys' rallying cries, of "better security", may be falling by the wayside. Apple can't really claim the moral high ground on security issues anymore. At best, they're on par with the rest of the industry, and at worst, they're falling behind.

I'd love to see a MAPP-like initiative from Apple. They could even come up with a marketing-friendly snazzy name for it, like MAPPLE or something. I'd love to see Apple embrace the security community. I'd love to see more details in Apple's security advisories, and I'd love to see less restrictive NDAs on development tools and software licenses.

In other words, I'd love to see an Apple that's not just cool for Joe User, but also for Joe Researcher. Apple's got great inroads in the security community: at Black Hat I saw an equal number of Macs and non-Macs. If Apple wants to keep those sort of inroads, a little more goodwill to the security community would be much appreciated.