Chapter 4: Storage with Xen
Throughout this book, so far, we’ve talked
about Xen mostly as an integrated whole, a complete virtualization solution, to use marketing’s word. The reality is a bit more complex than that. Xen itself is only one component of a platform that aims to free users from having to work with real hardware. The Xen hypervisor virtualizes a processor (along with several other basic components, as outlined in Chapter 2), but it relies on several underlying technologies to provide seamless abstractions of the resources a computer needs. This distinction is clearest in the realm of storage, where Xen has to work closely with a virtualized storage layer to provide the capabilities we expect of a virtual machine.
By that we mean that Xen, combined with appropriate storage mechanisms, provides near total hardware independence. The user can run the Xen machine anywhere, move the instance about almost at will, add storage freely, save the filesystem state cleanly, and remove it easily after it’s done.
Sounds good? Let’s get started.
- 1 Storage: The Basics
- 2 Varying Types of Storage
- 3 Basic Setup: Files
- 4 Enlarge Your Disk
- 5 Copy-on-Write and Snapshots
- 6 LVM and Snapshots
- 7 Storage and Migration
- 8 Closing Suggestions
- 9 Footnotes
- 10 Navigation
Storage: The Basics
The first thing to know about storage—before we dive into configuration on the dom0 side—is how to communicate its existence to the domain. DomUs find their storage by examining the domU config file for a disk= line. Usually it’ll look something like this:
disk = [ 'phy:/dev/cleopatra/menas,sda,w', 'phy:/dev/cleopatra/menas_swap,sdb,w' ]
This line defines two devices, which appear to the domU as sda and sdb. Both are physical,1 as indicated by the phy: prefix—other storage backends have their own prefixes, such as file: and tap: for file-backed devices. You can mix and match backing device types as you like—we used to provide a pair of phy: volumes and a file-backed read-only “rescue” image.
We call this a line, but it’s really more of a stanza—you can put the strings on separate lines, indent them with tabs, and put spaces after the commas if you think that makes it more readable. In this case, we’re using LVM, with a volume group named cleopatra and a pair of logical volumes called menas and menas_swap.
NOTE: By convention, we’ll tend to use the same name for a domain, its devices, and its config
file. Thus, here, the logical volumes menas and menas_swap belong to the domain menas, which has the config file /etc/xen/menas and network interfaces with
similar names. This helps to keep everything organized.
You can examine the storage attached to a domain by using the xm block-list command—for example:
# xm block-list menas Vdev BE handle state evt-ch ring-ref BE-path 2049 0 0 4 6 8 /local/domain/0/backend/vbd/1/2049 2050 0 0 4 7 9 /local/domain/0/backend/vbd/1/2050
Now, armed with this knowledge, we can move on to creating backing storage in the dom0.
Varying Types of Storage
It should come as little surprise, this being the world of open source, that Xen supports many different storage options, each with its own strengths, weaknesses, and design philosophy. These options broadly fall into the categories of file based and device based.
Xen can use a file as a block device. This has the advantage of being simple, easy to move, mountable from the host OS with minimal effort, and easy to manage. It also used to be very slow, but this problem has mostly vanished with the advent of the blktap driver. The file-based block devices differ in the means by which Xen accesses them (basic loopback versus blktap) and the internal format (AIO, QCOW, etc.).
Xen can also perform I/O to a physical device. This has the obvious drawback of being difficult to scale beyond your ability to add physical devices to the machine. The physical device, however, can be anything the kernel has a driver for, including hardware RAID, fibre channel, MD, network block devices, or LVM. Because Xen accesses these devices via DMA (direct memory access) between the device driver and the Xen instance, mapping I/O directly into the guest OS’s memory region, a domU can access physical devices at near-native speeds.
No matter what, though, all storage backends look the same from within the Xen virtual domain. The hypervisor exports a Xen VBD (virtual block device) to the domU, which in turn presents the device to the guest OS with an administrator-defined mapping to traditional Unix device nodes. Usually this will be a device of the form hdx or sdx, although many distros now use xvdx for xen virtual disk. (The hd and sd devices generally work, as well.)
We recommend blktap (a specialized form of file backend) and LVM for storage backends. These both work, offer good manageability, can be resized and moved freely, and support some mechanism for the sort of things we expect of filesystems now that we Live In The Future. blktap is easy to set up and good for testing, while LVM is scalable and good for production.
None of this is particularly Xen-specific. LVM is actually used (outside of Xen) by default for the root device on many distros, notably Red Hat, because of the management advantages that come with an abstracted storage layer. blktap is simply a Xen-specific mechanism for using a file as a block device, just like the traditional block loop driver. It’s superior to the loop mechanism because it allows for vastly improved performance and more versatile filesystem formats, such as QCOW, but it’s not fundamentally different from the administrator’s perspective.
Let’s get to it.
Basic Setup: Files
For people who don’t want the hassle and overhead of LVM, Xen supports fast and efficient file-backed block devices using the blktap driver and library.
blktap (blk being the worn-down stub of “block” after being typed hundreds of times) includes a kernel driver and a userspace daemon. The kernel driver directly maps the blocks contained by the backing file, avoiding much of the indirection involved in mounting a file via loopback. It works with many file formats used for virtual block devices, including the basic “raw” image format obtainable by dding a block device.
You can create a file using the dd command:
# dd if=/dev/zero of=/opt/xen/anthony.img bs=1M count=1024
NOTE: Your version of dd might require slightly different syntax—for example, it might require you to specify the block size in bytes.
Now dd will chug away for a bit, copying zeroes to a file. Eventually it’ll finish:
1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 15.1442 seconds, 70.9 MB/s
Thus armed with a filesystem image, you can attach it using the tap driver, make a filesystem on it, and mount it as usual with the mount command.
# xm block-attach 0 tap:aio:/opt/xen/anthony.img /dev/xvda1 w 0 # mkfs /dev/xvda1 # mount /dev/xvda1 /mnt/
First, we use the xm(8) command to attach the block device to domain 0. In this case the xm command is followed by the block-attach subcommand, with the arguments <domain id to attach the device to> <backend device> <frontend device> <mode> and optionally [backend domain id]. To decompose our example, we are attaching anthony.img read/write using the tap:aio driver to /dev/xvda1 in domain 0 using domain 0 to mediate access (because we tend to avoid using non-dom0 driver domains). When the file is attached as /dev/xvda1, we can create a filesystem on it and mount it as with any block device.
# cp -a /opt/xen/images/centos-4.4/* /mnt/
Add a disk= line to the domU config (in our example, /etc/xen/anthony) to reflect the filesystem:
disk = ['tap:aio:/opt/xen/anthony.img']
Now you should be able to start the domain with its new root device:
# xm create -c anthony
Watch the console and bask in its soothing glow.
MOUNTING PARTITIONS WITHIN A FILE-BACKED VBD
There’s nothing that keeps you from partitioning a virtual block device as if it were a hard drive. However, if something goes wrong and you need to mount the subpartitions from within dom0, it can be harder to recover. The standard mount -o loop filename /mnt won’t work, and neither will something like mount /dev/xvda1 /mnt (even if the device is attached as /dev/xvda, Xen will not automatically scan for a partition table and create appropriate devices).
kpartx will solve this problem. It reads the partition table of a block device and adds mappings for the device mapper, which then provides device file-style interfaces to the partitions. After that, you can mount them as usual.
Let’s say you’ve got an image with a partition table that describes two partitions:# xm block-attach 0 tap:aio:/path/to/anthony.img /dev/xvda w 0 # kpartx -av /dev/xvda
kpartx will then find the two partitions and create /dev/mapper/xvda1 and /dev/mapper/xvda2. Now you should be able to mount and use the newly created device nodes as usual.
LVM: Device-Independent Physical Devices
Flat files are well and good, but they’re not as robust as simply providing each domain with its own physical volume (or volumes). The best way to use Xen’s physical device support is, in our opinion, LVM.
LVM, short for logical volume management, is Linux’s answer to VxFS’s storage pools or Windows Dynamic Disks. It is what the marketing people call enterprise grade. In keeping with the software mantra that “all problems can be solved by adding another layer of abstraction,” LVM aims to abstract away the idea of “disks” to improve manageability.
Instead, LVM (as one might guess from the name) operates on logical volumes. This higher-level view allows the administrator much more flexibility—storage can be moved around and reallocated with near impunity. Even better, from Xen’s perspective, there’s no difference between an LVM logical volume and a traditional partition.
Sure, setting up LVM is a bit more work up front, but it’ll save you some headaches down the road when you have eight domUs on that box and you are trying to erase the partition for the third one. Using LVM and naming the logical volume to correspond to the domU name makes it quite a bit harder to embarrass yourself by erasing the wrong partition.2
Up to this point, we’ve talked exclusively about the “raw” file format—but it’s not the only option. One possible replacement is the QCOW format used by the QEMU project. It’s got a lot to recommend it—a fast, robust format that supports sparse allocation, encryption, compression, and copy-on-write. We like it, but support isn’t quite mature yet, so we’re not recommending it as your primary storage option.
Nonetheless, it might be fun to try. To start working with QCOW, it’ll be convenient to have QEMU. (While Xen includes some of the QEMU tools, the full package includes more functionality.) Download it from http://www.nongnu.org/qemu/download.html. As usual, we recommend the source install, especially because the QEMU folks eschew standard package management for their binary distribution.
Install QEMU via the standard process:# tar zxvf <qemu source package> # cd <qemu source directory> # ./configure # make # su # make install
QEMU includes the qemu-img utility, which is used to create and manipulate the various sorts of image files that QEMU supports, including QCOW, vmdk, raw, and others.# qemu-img create -f qcow enobarbus.qcow 1024M
This command creates an image in QCOW format (-f qcow) with a size of 1,024MB. Of course, you’ll want to replace the filename and size with appropriate values for your application. You can also convert a raw image to a QCOW image with the img2qcow utility, which is included as part of the Xen distribution:# img2qcow enobarbus.qcow enobarbus.img
You can use the QCOW image directly as a domain’s root disk with the tap driver. Configure the guest domain to use the QCOW image as its root filesystem. In the domain’s config file under /etc/xen, add a disk= line similar to:disk = [ 'tap:qcow:/opt/xen/enobarbus/enobarbus.qcow,sda1,w' ]
You can extend this line with another disk, thus:disk = [ 'tap:qcow:/opt/xen/enobarbus/enobarbus.qcow,sda1,w' , 'tap:qcow:/opt/xen/enobarbus/enobarbus_disk2.qcow,sdb1,w']
Basic Setup: LVM
The high-level unit that LVM operates on is the volume group, or VG. Each group maps physical extents (disk regions of configurable size) to logical extents. The physical extents are hosted on what LVM refers to as physical volumes, or PVs. Each VG can contain one or more of these, and the PVs themselves can be any sort of block device supported by the kernel. The logical extents, reasonably enough, are on logical volumes, abbreviated LVs. These are the devices that LVM actually presents to the system as usable block devices.
As we’re fond of saying, there really is no substitute for experience. Here’s a five-minute illustrated tutorial in setting up logical volumes (see Figure 4-1).
Figure 4-1: This diagram shows a single VG with two PVs. From this VG, we’ve carved out three logical volumes, lv1, lv2, and lv3. lv1 and lv3 are being used by domUs, one of which treats the entire volume as a single partition and one of which breaks the LV into subpartitions for / and /var.
Begin with some hard drives. In this example, we’ll use two SATA disks.
NOTE: Given that Xen is basically a server technology, it would probably be most sensible to use RAID-backed redundant storage, rather than actual hard drives. They could also be partitions
on drives, network block devices, UFS-formatted optical media . . . whatever sort of block device you care to mention. We’re going to give instructions using a partition on two hard drives, however. These instructions will also hold if you’re just
using one drive.
WARNING: Note that we are going to repartition and format these drives, which will destroy all data on them.
First, we partition the drives and set the type to Linux LVM. Although this isn’t strictly necessary—you can use the entire drive as a PV, if desired—it’s generally considered good Unix hygiene. Besides, you’ll need to partition if you want to use only a portion of the disk for LVM, which is a fairly common scenario. (For example, if you want to boot from one of the physical disks that you’re using with LVM, you will need a separate /boot partition.)
So, in this example, we have two disks, sda and sdb. We want the first 4GB of each drive to be used as LVM physical volumes, so we’ll partition them with fdisk and set the type to 8e (Linux LVM).
If any partitions on the disk are in use, you will need to reboot to get the kernel to reread the partition table. (We think this is ridiculous, by the way. Isn’t this supposed to be the future?)
Next, make sure that you’ve got LVM and that it’s LVM2, because LVM1 is deprecated.3
# vgscan --version LVM version: 2.02.23 (2007-03-08) Library version: 1.02.18 (2007-02-13) Driver version: 4.5.0
You might need to load the driver. If vgscan complains that the driver is missing, run:
# modprobe dm_mod
In this case, dm stands for device mapper, which is a low-level volume manager that functions as the backend for LVM. Having established that all three of these components are working, create physical volumes as illustrated in Figure 4-2.
# pvcreate /dev/sda1 # pvcreate /dev/sdb1
Figure 4-2: This diagram shows a single block device after pvcreate has been run on it. It’s mostly empty, except for a small identifier on the front.
Bring these components together into a volume group by running vgcreate. Here we’ll create a volume group named cleopatra on the devices sda1 and sdb1:
# vgcreate cleopatra /dev/sda1 /dev/sdb1
Finally, make volumes from the volume group using lvcreate, as shown in Figure 4-3. Think of it as a more powerful and versatile form of partitioning.
# lvcreate -L <length> -m1 --corelog -n menas cleopatra
Here we’ve created a mirrored logical volume that keeps its logs in core (rather than on a separate physical device). Note that this step takes a group name rather than a device node. Also, the mirror is purely for illustrative purposes—it’s not required if you’re using some sort of redundant device, such as hardware RAID or MD. Finally, it’s an administrative convenience to give LVs human-readable names using the -n option. It’s not required but quite recommended.
Figure 4-3: lvcreate creates a logical volume, /dev/vg/lvol, by chopping some space out of the LV, which is transparently mapped to possibly discontinuous physical extents on PVs.
Create a filesystem using your favorite filesystem-creation tool:
# mkfs /dev/cleopatra/menas
At this point, the LV is ready to mount and access, just as if it were a normal disk.
# mount /dev/cleopatra/menas /mnt/hd
To make the new device a suitable root for a Xen domain, copy a filesystem into it. We used one from http://stacklet.com/—we just mounted their root filesystem and copied it over to our new volume.
# mount -o loop gentoo.img /mnt/tmp/ # cp -a /mnt/tmp/* /mnt/hd
Finally, to use it with Xen, we can specify the logical volume to the guest domain just as we would any physical device. (Note that here we’re back to the same example we started the chapter with.)
disk = ['phy:/dev/cleopatra/menas,sda1,w']
At this point, start the machine. Cross your fingers, wave a dead chicken, perform the accustomed ritual. In this case our deity is propitiated by an xm create. Standards have come down in the past few millennia.
# xm create menas
Enlarge Your Disk
Both file-backed images and LVM disks can be expanded transparently from the dom0. We’re going to assume that disk space is so plentiful that you will never need to shrink an image.
Be sure to stop the domain before attempting to resize its underlying filesystem. For one thing, all of the user-space resize tools that we know of won’t attempt to resize a mounted filesystem. For another, the Xen hypervisor won’t pass along changes to the underlying block device’s size without restarting the domain. Most important, even if you were able to resize the backing store with the domain running, data corruption would almost certainly result.
The principle behind augmenting file-backed images is simple: We append more bits to the file, then expand the filesystem.
First, make sure that nothing is using the file. Stop any domUs that have it mounted. Detach it from the dom0. Failure to do this will likely result in filesystem corruption.
Next, use dd to add some bits to the end. In this case we’re directing 1GB from our /dev/zero bit hose to anthony.img. (Note that not specifying an output file causes dd to write to stdout.)
# dd if=/dev/zero bs=1M count=1024 >> /opt/xen/anthony.img
Use resize2fs to extend the filesystem (or the equivalent tool for your choice of filesystem).
# e2fsck -f /opt/xen/anthony.img # resize2fs /opt/xen/anthony.img
resize2fs will default to making the filesystem the size of the underlying device if there’s no partition table.
If the image contains partitions, you’ll need to rearrange those before resizing the filesystem. Use fdisk to delete the partition that you wish to resize and recreate it, making sure that the starting cylinder remains the same.
It’s just as easy, or perhaps even easier, to use LVM to expand storage. LVM was designed from the beginning to increase the flexibility of storage devices, so it includes an easy mechanism to extend a volume (as well as shrink and move).
If there’s free space in the volume group, simply issue the command:
# lvextend -L +1G /dev/cleopatra/charmian
If the volume group is full, you’ll need to expand it. Just add a disk to the machine and extend the vg:
# vgextend /dev/cleopatra /dev/sdc1
Finally, just as in the previous example, handle the filesystem-level expansion—we’ll present this one using ReiserFS.
# resize_reiserfs -s +1G /dev/cleopatra/charmian
Copy-on-Write and Snapshots
One of the other niceties that a real storage option gives you is copy-on-write, which means that, rather than the domU overwriting a file when it’s changed, the backend instead transparently writes a copy elsewhere.4 As a corollary, the original filesystem remains as a snapshot, with all modifications directed to the copy-on-write clone.
This snapshot provides the ability to save a filesystem’s state, taking a snapshot of it at a given time or at set intervals. There are two useful things about snapshots: for one, they allow for easy recovery from user error.5 For another, they give you a checkpoint that’s known to be consistent—it’s something that you can conveniently back up and move elsewhere. This eliminates the need to take servers offline for backups, such as we had to do in the dark ages.
CoW likewise has a bunch of uses. Of these, the most fundamental implication for Xen is that it can dramatically reduce the on-disk overhead of each virtual machine—rather than using a simple file as a block device or a logical volume, many machines can share a single base filesystem image, only requiring disk space to write their changes to that filesystem.
CoW also comes with its own disadvantages. First, there’s a speed penalty. The CoW infrastructure slows disk access down quite a bit compared with writing directly to the device, for both reading and writing.
If you’re using sparse allocation for CoW volumes, the speed penalty becomes greater due to the overhead of allocating and remapping blocks. This leads to fragmentation, which carries its own set of performance penalties. CoW can also lead to the administrative problem of oversubscription; by making it possible to oversubscribe disk space, it makes life much harder if you accidentally run out. You can avoid all of this by simply allocating space in advance.
There’s also a trade-off in terms of administrative complexity, as with most interesting features. Ultimately, you, the Xen administrator, have to decide how much complexity is worth having.
We’ll discuss device mapper snapshots, as used by LVM because they’re the implementation that we’re most familiar with. For shared storage, we’ll focus on NFS and go into more detail on shared storage systems in Chapter 9. We also outline a CoW solution with UnionFS in Chapter 7. Finally, you might want to try QCOW block devices—although we haven’t had much luck with them, your mileage may vary.
LVM and Snapshots
LVM snapshots are designed more to back up and checkpoint a filesystem than as a means of long-term storage. It’s important to keep LVM snapshots relatively fresh—or, in other words, make sure to drop them when your backup is done.6
Snapshot volumes can also be used as read-write backing store for domains, especially in situations where you just want to generate a quick domU for testing, based on some preexisting disk image. The LVM documentation notes that you can create a basic image, snapshot it multiple times, and modify each snapshot slightly for another domain. In this case, LVM snapshots would act like a block-level UnionFS. However, note that when a snapshot fills up, it’s immediately dropped by the kernel. This may lead to data loss. The basic procedure for adding an LVM snapshot is simple: Make sure that you have some unused space in your volume group, and create a snapshot volume for it.
THE XEN LIVECD REVISITED: COPY-ON-WRITE IN ACTION
The Xen LiveCD actually is a pretty nifty release. One of its neatest features is the
ability to automatically create copy-on-write block devices when a Xen domain starts, based on read-only images on the CD.
The implementation uses the device mapper to set up block devices and snapshots based on flat files, and is surprisingly simple.
First, the basic storage is defined with a line like this in the domain config file:disk=['cow:/mnt/cdrom/rootfs.img 30,sda1,w']
Note the use of the cow: prefix, which we haven’t mentioned yet. This is actually a custom prefix rather than part of the normal Xen package.
We can add custom prefixes like cow: because /etc/xen/scripts/create_block_device falls through to a script with a name of the form block-[type] if it finds an unknown device type—in this case, cow. The block-cow script expects one argument, either create or destroy, which the domain builder provides when it calls the script. block-cow then calls either the create_cow or destroy_cow script, as appropriate.
The real setup takes place in a script, /usr/sbin/create_cow. This script essentially uses the device mapper to create a copy-on-write device based on an LVM snapshot,* which it presents to the domain. We won’t reproduce it here, but it’s a good example of how standard Linux features can form the basis for complex, abstracted functions. In other words, a good hack.
* More properly, a device mapper snapshot, which LVM snapshots are based on. LVM snapshots are device mapper snapshots, but device mapper snapshots can be based on any pair of block devices, LVM or not. The LVM tools provide a convenient frontend to the arcane commands used by dmsetup.
Storage and Migration
These two storage techniques—flat files and LVM—lend themselves well to easy and automated cold migration, in which the administrator halts the domain, copies the domain’s config file and backing storage to another physical machine, and restarts the domain.
Copying over a file-based backend is as simple as copying any file over the network. Just drop it onto the new box in its corresponding place in the filesystem, and start the machine.
Copying an LVM is a bit more involved, but it is still straightforward: Make the target device, mount it, and move the files in whatever fashion you care to.
Check Chapter 9 for more details on this sort of migration.
These two storage methods only apply to locally accessible storage. Live migration, in which a domain is moved from one machine to another without being halted, requires one other piece of this puzzle: The filesystem must be accessible over the network to multiple machines. This is an area of active development, with several competing solutions. Here we’ll discuss NFSbased storage. We will address other solutions, including ATA over Ethernet and iSCSI, in Chapter 9.
NFS is older than we are, and it is used by organizations of all sizes. It’s easy to set up and relatively easy to administer. Most operating systems can interact with it. For these reasons, it’s probably the easiest, cheapest, and fastest way to set up a live migration-capable Xen domain.
The idea is to marshal Xen’s networking metaphor: The domains are connected (in the default setup) to a virtual network switch. Because the dom0 is also attached to this switch, it can act as an NFS server for the domUs.
In this case we’re exporting a directory tree—neither a physical device nor a file. NFS server setup is quite simple, and it’s cross platform, so you can use any NFS device you like. (We prefer FreeBSD-based NFS servers, but NetApp and several other companies produce fine NFS appliances. As we might have mentioned, we’ve had poor luck using Linux as an NFS server.) Simply export your OS image. In our example, on the FreeBSD NFS server at 192.0.2.7, we have a full Slackware image at /usr/xen/images/slack. Our /etc/exports looks a bit like this:
/usr/xen/images/slack -maproot=0 192.0.2.222
We leave further server-side setup to your doubtless extensive experience with NFS. One easy refinement would be to make / read-only and shared, then export read-write VM-specific /var and /home partitions—but in the simplest case, just export a full image.
NOTE: Although NFS does imply a performance hit, it’s important to recall that Xen’s network
buffers and disk buffers are provided by the same paravirtualized device infrastructure, and so the actual network hardware is not involved. There is increased overhead in transversing the networking stack, but performance is usually better than gigabit
Ethernet, so it is not as bad as you might think.
Now configure the client (CONFIG_IP_PNP=y). First, you’ll need to make some changes to the domU’s kernel to enable root on NFS:
networking-> networking options-> ip: kernel level autoconfiguration
If you want to do everything via DHCP (although you should probably still specify a MAC address in your domain config file), add DHCP support under that tree: CONFIG_IP_PNP_DHCP: or CONFIG_IP_PNP_BOOTP if you’re old school. If you are okay specifying the IP in your domU config file, skip that step.
Now you need to enable support for root on NFS. Make sure NFS support is Y and not M; that is, CONFIG_NFS_FS=Y. Next, enable root over NFS: CONFIG_ROOT_NFS=Y. In menuconfig, you can find that option under:
File systems -> Network File Systems -> NFS file system support -> Root over NFS
Note that menuconfig won’t give you the option of selecting root over NFS until you select kernel-level IP autoconfiguration.
Build the kernel as normal and install it somewhere where Xen can load it. Most likely this isn’t what you want for a dom0 kernel, so make sure to avoid overwriting the boot kernel.
Now configure the domain that you’re going to boot over NFS. Edit the domain’s config file:
# Root device for nfs. root = "/dev/nfs" # The nfs server. nfs_server = '18.104.22.168' # Root directory on the nfs server. nfs_root = '/usr/xen/images/slack' netmask="255.255.255.0" gateway="22.214.171.124" ip="126.96.36.199"
Note that we’re just adding extra Linux kernel configuration to the domain config—values not used by Xen will be passed to the kernel command line. You can also explicitly put this configuration in the “extra” parameter. If you want to set the IP address via DHCP, you can replace the last three lines above with:
You can then use DHCP to specify the NFS server and NFS root as well in the usual manner. Boot the domain and you’re done. Because the storage is accessible to any machine on the network, Xen’s live migration should work as well.
This might seem like a bewildering, or even excessive, variety of storage options, but all of these have their places—be it in a hosting environment, or on the desktop, or in a storage pool for utility computing. The recommendations we’ve made in this chapter are a start, but in the end the best advice we can offer is to try all of these and see what works best. Find the right trade-off between ease of administration and scalability. Finally, you can combine and extend many of these options. For example, the Xen LiveCD uses flat images with LVM snapshots. Depending on your application, the best solution might be simple filesystem images or a combination of software RAID and LVM. Keep experimenting and see what fits best. These are all examples of the flexibility of Xen’s standards-based architecture, which relies on user-extensible scripts to define available storage using easily understood semantics. In Chapter 5, we’ll look at how these same principles apply to Xen’s network setup.
1As you may gather, a physical device is one that can be accessed via the block device semantics,
rather than necessarily a discrete piece of hardware. The prefix instructs Xen to treat the device
as a basic block device, rather than providing the extra translation required for a file-backed
2This example is not purely academic.
3This is unlikely to be a problem unless you are using Slackware.
4This is traditionally abbreviated CoW, partly because it’s shorter, but mostly because “cow” is an inherently funny word. Just ask Wikipedia.
5It’s not as hard you might suppose to rm your home directory.
6Even if you add no data to the snapshot itself, it can run out of space (and corrupt itself) just keeping up with changes in the main LV.