I've been holding onto this little secret for a long time; I've finally broken down. A search on google seems to imply that no one else is aware of this, or at least they aren't willing to spill the beans. So, here goes ... Apple's X11 and the XDarwin server both support running X11 full screen or rootless. Many people (including me) have been frustrated by the fact that when you run X11 full screen, you effectively lose OS X, but when you run Gnome or KDE in rootless mode, everything looks funny and you lose the X11 desktop. There is actually an in-between option.
Virtually every release of XFree86 includes a nest X server, called Xnest. This is simply an X server that is simultaneously an X client. In most instances, this is completely useless. Why would you want to run an X server inside of an X server on Linux? Debugging. Why would you do it in OS X? You do it so you can run KDE in a window.
Read the rest of the hint for the how-to...
For every window manager that you want to be able to run, you need a separate .xinitrc file. For KDE, I have a file in my home directory called .xinitrc.kde that only contains the following line:
exec /sw/bin/startkde
Note that I do not actually have a file in my home directory called .xinitrc, as I am using Apple's quartz-wm to control these windows. The tricky part is to get the new x-session open. The xinit script makes this quite simple:
xinit ~/.xinitrc.kde -- /usr/X11R6/bin/Xnest :3 -geometry 800x600
The ~/.xinitrc.kde represents the script we have written that launches kde. Anything that immediately follows this gets passed to the script.Mac OS X Hints
http://hints.macworld.com/article.php?story=20040131222238684