Installing Mac OS X 10.5.6 under VMware 6.5.2

One of the current “Holy Grails” of virtual machinery is getting a functional installation of Mac OS X into a VMware virtual machine.   There’s a lot of Google talk out there about how to do it, but not everybody seems to be successful.  I certainly wasn’t for quiet some time.  However, I sat down today and decided to give it my best shot again, and this time I succeeded!  Not only did I succeed with iDeneb 1.4 (which installs Mac OS X 10.5.6), I successfully upgraded that release to 10.5.7.  Cool.  🙂

I’m documenting here the steps I followed (many times, just to make sure they worked for me) to achieve a successful Mac OS X 10.5.x install in a VMware Workstation 6.5.2 machine.  There is no sound, but there is networking, which is far more important to me.  If you find a way to get audio working, you might post it here.

Be aware that these steps worked for me, but YMMV.  If you are not successful using them, there’s not much I can do to help you out.  I had to experiment over and over to find these working steps; you will probably have to do the same to overcome your problems.  For the purposes of this tutorial, I’ll be using the file “iDeneb_v1.4_10.5.6.iso”.

Here We Go

1. First, of course, you need to grab the iDeneb 1.4 distribution (Mac OS X 10.5.6).  Google it, or use your favorite torrent search engine.
2. Mount the iDeneb ISO as a drive (using something like UltraISO or Virtual CloneDrive).  Note the drive letter of the DVD.
3. Create a new virtual machine in VMware.  The following sequence of screen grabs should show you all the settings you need to create for the new environment:

	Tell VMware that you will install the operating system later:
	For operating system type, Select “Other”, and then “FreeBSD 64-bit”:
	I named it “10.5” because I will upgrade from the 10.5.6 on the iDeneb 1.4 image to 10.5.7. You can call it whatever you like, of course:
	Choose the number of processors to let the OS use:
	Allocate an amount of memory to use:
	Unless you know what you’re doing, select “Bridged networking”:
	Drive-type settings:
	Set the space available on the disc. The installation will consume 6-7GB, and the update will consume even more. If you need to do more on the drive, increase that size here:
	Don’t power on the machine yet:

4. Edit the machine settings, and set the CD/DVD drive to use the one that you noted earlier where the iDeneb 1.4 image is mounted:

5. Power on the machine. You’ll want to be sure you click in the window immediately so it captures the mouse/keyboard focus. When prompted to press “F8” for startup options, press it. Then, at the “boot:” prompt, you can enter options you want for starting the installer. I use the following (‘-v’ means show diagnostic messages; I recommend entering at least that):

6. Now, if you’re lucky, at this point you should boot into the iDeneb installer. When you get to the installation prompt, you’ll want to go to the “Utilities” menu, and launch “Disk Utility”. From this interface, select the VM disc, and then select the “Partition” tab:

7. Create a single-partition layout, leaving the volume type as “Mac OS Extended (Journaled)”. This volume type is important: I was unable to get the VM to boot with any other type:

8. Select the “Options” button, and change the disc to use a “Master Boot Record” partition. The other types failed to work for me:

9. “Apply” your partition settings, and exit the “Disk Utility”.
10. Proceed with the installation, selecting your VM disc as the target partition (it will be the only one, if you followed these instructions).
11. When the installation completes, you’ll be prompted to restart. Do so, but when the machine gets back to the boot prompt, “Stop” it immediately (don’t worry, this won’t harm anything) so that you are back to the VMware desktop and the VM is powered off.
12. Change the “CD/DVD” settings of the VM from the specific iDeneb drive back to “Auto”. DO NOT UNMOUNT THE DVD. You’ll want to leave the iDeneb DVD mounted in the operating system, because it seems to be the only way to allow the VM to actually boot correctly.
13. Shut down VMware.
14. Open the .vmx file of the new iDeneb machine (e.g., “Mac OS X 10.5.vmx”), and find the line that starts with “guestOS”. It should contain the entry “freebsd-64”. Change this to “darwin-64”, and save the file.
15. Re-start VMware, and power on the Mac OS X VM. Let the boot screen time out, and if the gods are smiling down upon you, you will boot into Mac OS X 10.5.6.

Post-installation Booting

Of course, things did not go this smoothly for me.  It took me a couple of boot attempts to get the operating system to come up (boot-hang-kill-repeat).  When it did, I did all the set up stuff, and it took me into the OS X desktop. Once I get everything set up that is boot dependent (like updates and screen resolution), I plan to simply suspend the VM so that I can unmount the iDeneb DVD, and then simply power on the VM whenever I need it.

Screen Resolution

There are two ways to control the screen resolution.  You can enter it at the boot prompt (see the previous image), but this is cumbersome, and can typically lead to a failed boot.  Or, you can change the com.apple.Boot.plist file to make the resolution permament (my preferred approach).  Both approaches require a boot up sequence in order to be successful.  You can read about implementing both approaches here.

Good Luck

If you’ve been looking for a successful way to create a portable Macintosh, I hope this approach works for you. Being a software engineer, it’s really fantastic to be able to carry Linux, Mac OS X, and Vista along with me on a single machine when I’m away from home.