This article put up with the permission of my friend Girish Venkatachalam. He has written quite a few articles including some for the Linux Journal. Below are some links to his articles: http://wiki.openbsd.org http://linuxjournal.com/9004 (Cover story of Aug 2006) http://linuxjournal.com/8289 (first international article) http://www.openbsd-wiki.org/index.php?title=Multiboot http://www.ciol.com/content/search/showarticle.asp?arid=23466&way=search http://www.mindtree.com/kc/view_art.php?wid=8 http://www.mindtree.com/kc/view_art.php?wid=9 http://mouthshut.com/user/girish1729/1.html Sandboxing and Virtualization demystified I Hi, It is me again. I am going to take you through a tour of the most thoroughly fascinating world of virtualization. It is an oft abused term largely due to "name space pollution". A wide variety of techniques have come to be referred to as virtualization. So much so, that the original meaning and technical definition lie buried deep inside and can be quite hard to recognize. This is because corporate marketing has so thoroughly promoted all sorts of related but distinctly different technologies in the name of virtualization. Once you finish reading this series, you shall know why. Why is it that all of a sudden virtualization has become so crucial to business? Let us get to that in a while. To begin with, let us have some fun. I am going to talk about some really hardcore technical concepts and math, so let us relax a bit. "Ekam sadvipra bahutha vadanthi." So goes a saying in Sanskrit. "Ekam sat" means one "One Truth" or "One God". "Vipra bhutha vadanthi" translates to "Learned men call it by various names. Anyway the literal translation is "One God called by various names." Yesterday I went to fill air in my bicycle. And while I was about it, I just took a look at the three stickers on the wall. One was Mecca, the other Jesus, and the third a Hindu God. As you can see, there is immense profundity in the above saying. That whatever path we may adopt, we all meet at the end. In a related note, according to Hindu philosophy, there are four types of Yoga; Karma Yoga, Jnana Yoga, Raja Yoga and Bhakti Yoga. Depending on the temperament and psychology of the aspirant, one is free to choose whichever path suits him or her best. Obviously that begs the question. Then why are we, the hacker community, so religious and zealous about Linux? Isn't the world so much better by flowers of different colors, fragrance and beauty? Now you see the point. Right. There is no one size fits all solution. If Linux is good at many things, there are so many equally good or sometimes better suited alternatives for doing other things. Ultimately, we are interested in getting our job done in the best and easiest manner possible. This series is split into three parts. I am deliberately leaving out the details and focusing instead on the big picture, how things fit together, explaining the missing pieces of the jigsaw puzzle of this unity in diversity. Once you understand the philosophy and rationale behind the approaches you, can easily fill in the details by reading the relevant documentation. In particular, I will be dealing with the following techniques in this issue: a) Multibooting Gentoo, Linux, NetBSD, OpenBSD and FreeBSD from a single hard disk b) Qemu In the next issue we will be dealing with Xen. Needless to mention, I am sure you shall have as much fun as I had in learning the innards of Xen myself. Then we will conclude with various other sandboxing techniques and emulation technologies like running Skype on NetBSD and FreeBSD through Linux emulation, linux-vserver project, FreeBSD jails, etc. To me open source software is a synonym for excellence. Whenever a free alternative kills a proprietary tool, I feel happy. The recent gparted project that obviates the need to buy Partition Magic is a case in point. Similarly, Xen will soon eradicate VMWare. Not that I am against proprietary software. Just that open source software is better. I shall also be dealing with ways and means of running proprietary software using open source emulation tools. Let us get down to business now. Finally! Multi Booting all popular free Operating Systems :- Many of you will be familiar with dual boot configurations in which you boot Linux and Windows from the same hard disk. I have also multibooted Windows, Linux and FreeBSD from the same hard disk. What about Linux, NetBSD, OpenBSD and FreeBSD from the same disk? Sounds cool, eh? It isn't, because multiboot configurations are usually a big pain in the neck. Most of the time you think multiboot is the solution. You should think again. Nobody wants to reboot to be able to run an application. However, it definitely has its ecological niche. For instance, I am developing an open source multiseat computer project in which one CPU can be shared between multiple users with multiple terminals. I want to release this software under all operating systems I mentioned above. Hence my need for multibooting. It serves as a great test setup. And of course by attaining the theoretical upper limit of installing 4 different OSes on 4 primary partitions, you should be able to mix and match various combinations to suit your needs. This part, unlike the other parts in the series, will talk about details since there is nothing to speak of in so far as the big picture goes. But that said, certain key concepts have to be borne in mind. And, make sure you know what you are doing. I cannot stress this enough! It is very hard to get it right the first time no matter how much you read and learn. Experience is the best teacher as always. However, I shall try my level best to chronicle my experience and explain the concepts. Before we begin, a very important warning. Nearly every piece of documentation on this topic has to be taken with a great deal of skepticism. It may or may not apply in your case. This is not only because things are changing so fast, but also because several things are undocumented and unclear. Whether some undocumented artifact hits you or not is a guess only God can make. All of us have heard of the MBR (Master Boot Record). What is it? It is composed of two parts, IPL ( Initial Program Load ) code and Partition table data ending with 0xaa55. It is: 446 bytes + 64 bytes + 2 bytes = 512 bytes. In that sequence. That was simple math, wasn't it? All of the math in this article is simple. Just because it is math doesn't mean it has to be complicated. Now, the code in the MBR is used to bootstrap an i386 box. This is not always true, but let us assume it is so in our case. The machine BIOS inspects the MBR partition table, figures out which of the primary partitions is active (set bootable), and invokes the PBR (Partition Boot Record). Let us take a peek at the grub.conf that I used since that is the most critical piece in the scheme of things. default 0 timeout 10 splashimage=(hd0,0)/grub/splash.xpm.gz title Gentoo Linux root (hd0,0) kernel /boot/kernel root=/dev/sda1 title NetBSD root (hd0,1) chainloader +1 makeactive title OpenBSD root (hd0,2) chainloader +1 makeactive title FreeBSD root (hd0,3) chainloader +1 makeactive Don't forget the splash image as it can enliven things up for you. The rest of the section is quite self explanatory. Each of the primary partitions are booted using the chain loading technique. Remember that I am explaining the simplest possibly way to do things. You can complicate by doing lot of non standard things, but if you keep things simple it will save everyone a great deal of grief. If instead you follow the advice of creating a small boot partition within the top 8 GB of the hard disk and all that... Oh my God, are you a masochist? Do you derive pleasure out of inflicting pain on yourself? All modern BIOSes support LBA translation. Of course, I am assuming that you try all virtualization stuff I speak about with modern hardware. It wouldn't make much sense with old hardware, as we are talking about enterprise class stuff. Running multiple simultaneous operating systems is not important for your old PIII box, is it? Anyway, getting back on track, grub.conf is one of the final steps in the setup process. Before we get down to the task, a little bit of theory and high school math. A hard disk is organized as a bunch of thin platters stacked one above other that sit on a spindle. Each platter has two heads, one on each side. Data is stored on concentric circles or rings starting from the periphery towards the middle. Each circle is further sub divided into sectors. The smallest unit of storage on disk is a sector. Circles on the disk are called tracks, and multiple tracks are aggregated into one cylinder. Now this should give you a reasonably clear picture of the Cylinder/Head/Sector notation used to represent disk storage. It is important to bear in mind that the geometry reported by BIOS is different from the geometry reported by the OS which is different from... Well, suffice it to say that all modern BIOS/OS software are totally agnostic to the exact physical layout. Some more details to impress you. A sector holds exactly 512 bytes. And, the total capacity of a disk usually reported by fdisk as total sectors can be multiplied by 512 to obtain the disk capacity. This will not be accurate but quite close. If you think of the entire disk as data stored from sector 0 to sector N, it will help you visualize things perfectly which will, in turn, help you avoid make mistakes that bite you really badly. Take my case. I installed Gentoo on the first 40 GB of the 164 GB disk. Then I installed NetBSD on another 40 GB partition, then OpenBSD on another 40 GB with FreeBSD on the last remaining 40 GB. So, hypothetically speaking, let us say 164 GB is 320312500 sectors. I just divided it by 512. Now the first partition will be from sector 1 to sector 78125000. The next will be from 78125001 to 156250000 and so on. Stupid me! I oversimplified. The first track should be left alone. You should always begin from sector 63. Since the MBR is stored on the first track. And also, each partition should begin at a cylinder boundary. So the above calculation obviously is wrong. But the idea is the same and fdisk usually takes care of these details for you. But you the user can always goof up. The way I did, I started with creating a primary partition of partition type Linux. Its identifier is hex 0x83. I formatted that with XFS file system and installed Gentoo Linux on it. I made sure that grub is installed on the MBR and filled up grub.conf. It is of course imperative that you verify that each of the OS boots individually first. After that I booted using the NetBSD CD and created the NetBSD partition of type hex 0xa9 right below the Gentoo fdisk partition. I had to mark the NetBSD partition as active or it would fail to install. Inside the NetBSD partition I created BSD disklabel partitions that are visible only to software that can read BSD labels. I created /, /usr, /home, etc. Remember that all the BSD disklabel partitions should be well within the NetBSD fdisk partition. In other words, it should lie between the sectors that delineate the fdisk partition. If you make sure of this fact life becomes really easy. Install and reboot. Too bad, you cannot boot Linux now. Never mind, we will get to that later. First make sure NetBSD boots properly. Good, now let us proceed. By the way, there are certain nuances in the way fdisk displays partition table in different OSes. And there will be minor inaccuracies. Don't get petrified. All will be well. God is there. Once you get to OpenBSD install, I preferred to create OpenBSD fdisk partitions using raw cylinders. You have to be very careful here. OpenBSD is meant for the smartest of the smart, hence it assumes by default that you know what you are doing. So please don't misuse your power. Make sure you understand the above before creating OpenBSD fdisk partition and corresponding disklabels. Make sure you mark the OpenBSD fdisk partition as active and reboot. You should be able to boot OpenBSD. Now is a good time to boot with the Gentoo liveCD and rerun grub. Also check for the three OSes booting with the above grub.conf. Booting? Great. FreeBSD now. FreeBSD install is the simplest as it will not in general confuse you in any way. It usually helps you out with things automatically. You can also try installing the FreeBSD boot manager in place of grub. For me, however, FreeBSD boot manager could not boot Gentoo. It could boot all BSDs perfectly. Perhaps I went wrong? Anyway, I prefer the nice graphic of grub. Moreover, grub seems to be very popular these days. Here is my fdisk output from Gentoo. #fdisk -lu /dev/sda ( Remember it is SATA disk, for you it might be /dev/hda) Disk /dev/sda: 164.6 GB, 164695473664 bytes 255 heads, 63 sectors/track, 20023 cylinders, total 321670847 sectors Units = sectors of 1 * 512 = 512 bytes Device Boot Start End Blocks Id System /dev/sda1 63 78124094 39062016 83 Linux /dev/sda2 78124095 160055594 40965750 a9 NetBSD /dev/sda3 160055595 242308394 41126400 a6 OpenBSD /dev/sda4 * 242308395 321669494 39680550 a5 FreeBSD #fdisk -l /dev/sda Disk /dev/sda: 164.6 GB, 164695473664 bytes 255 heads, 63 sectors/track, 20023 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 1 4863 39062016 83 Linux /dev/sda2 4864 9963 40965750 a9 NetBSD /dev/sda3 9964 15083 41126400 a6 OpenBSD /dev/sda4 * 15084 20023 39680550 a5 FreeBSD Now you want to overwrite the FreeBSD boot manager with grub? No problem. Just boot with your Gentoo liveCD, mount your Linux partition, chroot to it, rerun grub install and you are done. That is it. Now, I know what is on your mind. What has this got to do with the title, "Sandboxing and Virtualization"? Well, we normally use sandboxing and virtualization techniques to run foreign OSes or simulate foreign platforms like Sparc hardware. What better place to start on this topic other than the all too familiar multiboot? You should be able to share /home partition and also swap space by appropriate configuration. But, as I said before, multiboot is useful only under specific situations, hence you are the best judge what you need and what you don't. All the BSDs have read/write support for ext2fs/ext3fs. Anyway, let us move forward. Life is going to get really exciting now. Qemu :- Before we start with Qemu, you should excuse me as I will have to spend a moment about the various terms being bandied about viz, emulation, simulation, virtualization (that will be later), instruction set simulator, partitioning, etc. For the complete scoop, you have to wait until the series finishes. But for now I will clarify whatever is required to understand the rationale behind Qemu, the original goal of the creator Fabrice Bellard, and why it is so cool. For details and other formal descriptions, of course, there is no better source than wikipedia. This applies to all topics I cover. Wikipedia, however, leaves a lot of things to be desired, and also has some random errors here and there. Not to mention that certain open source projects are under represented and are more Windows and Linux focused than they should be. Consequently, it is not as unbiased as one would prefer. I, as a hacker/critic, am not associated with any of these open source projects. I have tried my level best to give as unbiased and objective a view as I can. However, there is always human error, a certain unavoidable degree of affinity, and prejudice. Wherever possible, I am leaving you to decide what best to use to solve your particular problem in hand. Sometimes you don't have any immediate problem to solve. In that case also it is worth knowing what tools are available so you can find a new way to use a particular tool like Qemu far beyond the original goal conceived by its creator. This really is the acid test of any well designed product. It should be like a piece of beautiful poetry from which different people get different meanings all within the theme envisaged by the poet. Qemu was created by the same chap who created the immensely popular easy to use multimedia tool, ffmpeg. Its amazing power is hidden by its pure simplicity and ease of use. It is so intuitive to extract audio from video or convert between formats and bit rates that even normal people can use it without much ado. Mencoder, on the other hand, is extremely powerful and sometimes works better, but it is very hard to use. Qemu is most definitely the easiest way for you to get initiated into the black magic called emulation/virtualization. You get instant gratification and its sheer power and elegance is immediately apparent, though not suitable for everyday use like VMWare or Xen. This is simply because the solution space of Qemu and Xen are completely different. Most benchmarks and comparisons miss this point entirely. Qemu can be compared only with Qemu and nothing else. It had a lofty goal and the author's genius is apparent from the very fact that he achieved the ideal in a manner that is also practical. Let me explain. First some concepts to go with. For those of you embedded engineers and hardware folks, terms like instruction set simulator, ICE (in circuit emulator), and VLSI simulators are nothing new or foreign. They make life simpler for us by "emulating" or "simulating" the actual running machine or hardware. Too bad emulation and simulation have come to have more or less confused meanings. Anyway, the way I understand it, emulation is just a sophisticated simulation. Just like an in circuit emulator emulates the particular processor target in embedded systems, Qemu emulates the entire PC architecture for you. The list is available on the Qemu man page. For the OS kernel, the processor is not the only thing that matters. The bus interconnects the motherboard, the BIOS (most importantly), and peripherals, disk and network card. Of course, Qemu does much more, but the concept is this, that Qemu realizes the ideal of making nearly impossible things possible and also practical. You can run programs written for the Sparc architecture very simply by emulating it under Qemu. This is the reason it appears so paradoxical why Qemu emulates x86 under x86. The author goes to great lengths to ensure acceptable performance, even though the goal is actually testing and debugging of different platforms, architectures, and OSes within the Qemu emulator. To achieve this, a great deal of assembly language tricks, compiler optimizations and dynamic rewriting of code is done. The traditional approach of interpreting assembly instructions is done away with, since the performance would not be acceptable. Consequently, Qemu is extremely powerful and versatile, and it has self- contained support for nearly everything including TFTP, DHCP, Samba, etc. So far I have only been using Qemu for testing liveCD images, but that is only the tip of the iceberg. Go explore! As you can see, Qemu has more to do with emulation rather than sandboxing or virtualization. Since the May 2006 issue focused on virtualization and covers details about everything I talk about, please pick that up and read if you are curious. Also, as always, refer to the resources for more info. Resources :- http://www.floobydust.com/virtualization/lawton_1999.txt (Running multiple operating systems concurrently on an IA32 PC using virtualization techniques) http://www.Gentoo.org (Gentoo Linux ) http://www.freebsd.org (FreeBSD) http://www.netbsd.org (NetBSD) http://www.openbsd.org (OpenBSD) http://en.wikipedia.org/wiki/QEMU (Qemu wikipedia article) http://fabrice.bellard.free.fr/qemu/ (Qemu home page) http://www.linuxjournal.com/article/8808 (May issue article on qemu)