Appendix B: The Structure of the Xen Config File

The domain config file is the conventional way to define a Xen domain (and the method that we’ve used throughout this book). It works by specifying Python variables in a config file, conventionally kept in /etc/xen/<domainname>. When the domain is created, xend executes this file and uses it to set variables that will eventually control the output of the domain builder.

Note also that you can override values in the config file from the xm command line. For example, to create the domain coriolanus with a different name:

xm create coriolanus name=menenius

The config file—and it would be difficult to overstate this point—is executed as a standard Python script. Thus, you can embed arbitrary Python in the config file, making it easy to autogenerate configurations based on external constraints. You can see a simple example of this in the example HVM config shipped with Xen, /etc/xen/xmexample.hvm. In this case, the library path is selected based on the processor type (i386 or x86_64).

The xmexample2 file takes this technique even further, using a single config file to handle many domains, which are differentiated by a passed-in vmid variable.

Python in the config file isn’t limited to domain configuration, either. If you’re using Xen for hosting, for example, we might suggest tying the domain configuration to the billing and support-ticketing systems, using some Python glue to keep them in sync. By embedding this logic in the config files, or in a separate module included by the config files, you can build a complex infrastructure around the Xen domains.

First, let’s start with the basic elements of a domain configuration. Here’s a basic config file, specifying the VM name, kernel image, three network cards, a block device, and a kernel parameter:

name = coriolanus
kernel = "/boot/linux-2.6-xen"
vif = ['','','']
disk = ['phy:/dev/corioles/coriolanus-root,sda,rw']
root = "/dev/sda ro"

Here we’re setting some variables (name, kernel, disk, and so on) to strings or lists. You can easily identify the lists because they’re enclosed in square brackets.

String quoting follows the standard Python conventions: a single quote for noninterpreted strings, double quotes for strings with variable substitution, and three single quotes to begin and end a multiline string.

Whitespace has significance just as in standard Python—newlines are significant and spacing doesn’t matter, except when used as indentation.

NOTE: Although these syntax rules are usually true, some external tools that parse the config file may have stricter rules. pypxeboot is an example.

Here’s another, more complex example, with an NFS root. In addition, we’ll specify a couple of parameters for the vif:

name = coriolanus
kernel = "/boot/linux-2.6-xen"
initrd = "/boot/initrd-xen-domU"
memory = 256
vif =
['mac=08:de:ad:be:ef:00,bridge=xenbr0','mac=08:de:ad:be:ef:01,bridge=xenbr1']
netmask = '255.255.255.0'
gateway = '192.168.2.1'
ip = '192.168.2.47'
broadcast = '192.168.2.255'
root = "/dev/nfs"
nfs_server ='192.168.2.42'
nfs_root = '/export/domains/coriolanus'

NOTE: Your kernel must have NFS support and your kernel or initrd needs to include xennet for this to work.

Finally, HVM domains take some other options. Here’s a config file that we might use to install an HVM FreeBSD domU.

import os, re
arch = os.uname()[4]
if re.search('64', arch):
   arch_libdir = 'lib64'
else:
   arch_libdir = 'lib'
kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
memory = 1024
name = "coriolanus"
vcpus=1
pae=1
acpi=0
vif = [ 'type=ioemu, bridge=xenbr0' ]
disk = [
       'phy:/dev/corioles/coriolanus_root,hda,w',
       'file:/root/8.0-CURRENT-200809-i386-disc1.iso,hdc:cdrom,r'
]
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
boot="cd"
vnc=1
vnclisten="192.168.1.102"
serial='pty'

Here we’ve added options to specify the QEMU-based backing device model and to control certain aspects of its behavior. Now we pass in a boot option that tells it to boot from CD and options for a virtual framebuffer and serial device.

List of Directives

Here we’ve tried to list every directive we know about, whether we use it or not, with notes indicating where we cover it in the main text of this book. We have, however, left out stuff that’s marked deprecated as of Xen version 3.3.

There are some commands that work with the Xen.org version of Xen but not with the version of Xen included with Red Hat Enterprise Linux/ CentOS 5.x. We’ve marked these with an asterisk (*).

Any Boolean parameters expect values of true or false; 0, 1, yes, and no will also work.

bootargs=string

This is a list of arguments to pass to the boot loader. For example, to tell PyGRUB to load a particular kernel image, you can specify bootargs='kernel=vmlinuz-2.6.24'.

bootloader=string

The bootloader line specifies a program that will be run within dom0 to load and initialize the domain kernel. For example, you can specify bootloader=pygrub to get a domain that, on startup, presents a GRUBlike boot menu. We discuss PyGRUB and pypxeboot in Chapter 7 and Chapter 3.

builder=string

This defaults to “Linux”, which is the paravirtualized Linux (and other Unix-like OSs) domain builder. Ordinarily you will either leave this option blank or specify HVM. Other domain builders are generally regarded as historical curiosities.

cpu_capp=int *

This specifies a maximum share of the CPU time for the domain, expressed in hundredths of a CPU.

cpu=int

This option specifies the physical CPU that the domain should run VCPU0 on.

cpu_weight=int *

This specifies the domain’s weight for the credit scheduler, just like the xm sched-credit -w command. For example, cpu_weight = 1024 will give the domain twice as much weight as the default. We talk more about CPU weight in Chapter 7.

cpus=string

The cpus option specifies a list of CPUs that the domain may use. The syntax of the list is fairly expressive. For example, cpus = "0-3,5,^1" specifies 0, 2, 3, and 5 while excluding CPU 1.

dhcp=bool

This directive is only needed if the kernel is getting its IP at boot, usually because you’re using an NFS root device. Ordinary DHCP is handled from within the domain by standard userspace daemons, and so the DHCP directive is not required.

disk=list

The disk line specifies one (or more) virtual disk devices. Almost all domains will need at least one, although it’s not a requirement as far as Xen’s concerned. Each definition is a stanza in the list, each of which has at least three terms: backend device, frontend device, and mode. We go into considerably more detail on the meaning of these terms and the various types of storage in Chapter 4.

extra=string

The extra option specifies a string that is appended, unchanged, to the domU kernel options. For example, to boot the domU in single user mode:

extra = "s"

Many of the other options listed here actually append to the kernel command-line options.

hpet

This option enables a virtual high-precision event timer.

kernel=string

This option specifies the kernel image that Xen will load and boot. It is required if no bootloader line is specified. Its value should be the absolute path to the kernel, from the dom0’s perspective, unless you’ve also specified a bootloader. If you’re using a bootloader and specify a kernel, the domain creation script will pass the kernel value to the bootloader for further action. For example, PyGRUB will try to load the specified file from the boot media.

maxmem=int

This specifies the amount of memory given to the domU. From the guest’s perspective, this is the amount of memory plugged in when it boots.

memory=int

This is the target memory allocation for the domain. If maxmem isn’t specified, the memory= line will also set the domain’s maximum memory. Because we don’t oversubscribe memory, we use this directive rather than max-mem. We go into a little more detail on memory oversubscription in Chapter 14.

name=string

This is a unique name for the domain. Make it whatever you like, but we recommend keeping it under 15 characters, because Red Hat’s (and possibly other distros’) xendomains script has trouble with longer names. This is one of the few non-optional directives. Every domain needs a name.

nfs_server=IP

These two arguments are used by the kernel when booting via NFS. We describe setting up an NFS root in Chapter 4.

nics=int

This option is deprecated, but you may see it referenced in other documentation. It specifies the number of virtual NICs allocated to the domain. In practice, we always just rely on the number of vif stanzas to implicitly declare the NICs.

on_shutdown

These three commands control how the domain will react to various halt states—on_shutdown for graceful shutdowns, on_reboot for graceful reboot, and on_crash for when the domain crashes. Allowed values are:

• destroy: Clean up after the domain as usual.
• restart: Restart the domain.
• preserve: Keep the domain as-is until you destroy it manually.
• rename-restart: Preserve the domain, while re-creating another instance with a different name.

on_xend_stop=ignore|shutdown|suspend

Similarly, these two items control how the domain will react to xend exiting. Because xend sometimes needs to be restarted, and we prefer to minimize disruption of the domUs, we leave these at the default: ignore.

pci=BUS:DEV.FUNC

This adds a PCI device to the domain using the given parameters, which can be found with lspci in the dom0. We give an example of PCI forwarding in Chapter 14.

ramdisk=string

The ramdisk option functions like the initrd line in GRUB; it specifies an initial ramdisk, which usually contains drivers and scripts used to access hardware required to mount the root filesystem.

Many distros won’t require an initrd when installed as domUs, because the domU only needs drivers for extremely simple virtual devices. However, because the distro expects to have an initrd, it’s often easier to create one. We go into more detail on that subject in Chapter 14.

root=string

This specifies the root device for the domain. We usually specify the root device on the extra line.

rtc_offset

The rtc_offset allows you to specify an offset from the machine’s realtime clock for the guest domain.

sdl=bool

Xen supports an SDL console as well as the VNC console, although not both at the same time. Set this option to true to enable a framebuffer console over SDL. Again, we prefer the vfb syntax.

shadow_memory=int

This is the domain shadow memory in MB. PV domains will default to none. Xen uses shadow memory to keep copies of domain-specific page tables. We go into more detail on the role of page table shadows in Chapter 12.

uuid=string

The XenStore requires a UUID to, as the name suggests, uniquely identify a domain. If you don’t specify one, it’ll be generated for you. The odds of collision are low enough that we don’t bother, but you may find it useful if, for example, you want to encode additional information into your UUID.

vcpu_avail=int

These are active VCPUs. If you’re using CPU hotplugging, this number may differ from the total number of VCPUs, just as max-mem and memory may differ.

vcpus=int

This specifies the number of virtual CPUs to report to the domain. For performance reasons, we strongly recommend that this be equal to or fewer than the number of physical CPU cores that the domain has available.

vfb=list

vfb = [type='vnc' vncunused=1]

In this case, we specify a VNC virtual framebuffer, which uses the first unused port in the VNC range. (The default behavior is to use the base VNC port plus domain ID as the listen port for each domain’s virtual framebuffer.)

Valid options for the vfb line are: vnclisten, vncunused, vncdisplay, display, videoram, xauthority, type, vncpasswd, opengl, and keymap. We discuss more details about virtual framebuffers in Chapter 14 and a bit in Chapter 12. See the vnc= and sdl= options for an alternative syntax.

videoram=int

The videoram option specifies the maximum amount of memory that a PV domain may use for its frame buffer.

vif=list

The vif directive tells Xen about the domain’s virtual network devices. Each vif specification can include many options, including bridge, ip, and mac. For more information on these, see Chapter 5.

Allowable options in the vif line are backend, bridge, ip, mac, script, type, vifname, rate, model, accel, policy, and label.

vnc=bool

Set vnc to 1 to enable the VNC console. You’ll also want to set some of the other VNC-related options, such as vncunused. We prefer the vfb syntax, which allows you to set options related to the vfb in a single place, with a similar syntax to the vif and disk lines.

vncconsole=bool

If vncconsole is set to yes, xend automatically spawns a VNC viewer and connects to the domain console when the domain starts up.

vncdisplay=int

This specifies a VNC display to use. By default, VNC will attach to the display number that corresponds to the domain ID.

vnclisten=IP

This specifies an IP address on which to listen for incoming VNC connections. It overrides the value of the same name in xend-config.sxp.

vncpasswd="Swordfish"¹

These options set the password for the VNC console to the given value. Note that this is independent of any authentication that the domU does.

vscsi=PDEV,VDEV,DOM *

This adds a SCSI device to the domain. The paravirtualized SCSI devices are a mechanism for passing a physical SCSI generic device through to a domain. It’s not meant to replace the Xen block driver. Rather, you can use pvSCSI, the SCSI pass-through mechanism, to access devices like tape drives or scanners that are hooked up to the machine’s physical SCSI bus.

vtpm=['instance=INSTANCE,backend=DOM,type=TYPE']

The vtpm option, just like the vif or disk options, describes a virtual device—in this case, a TPM. The TPM instance name is a simple identifier; something like 1 will do just fine. The backend is the domain with access to the physical TPM. Usually 0 is a good value. Finally, type specifies the type of the TPM emulation. This can be either pvm or hvm, for paravirtualized and HVM domains, respectively.

HVM Directives

acpi=bool

The acpi option determines whether or not the domain will use ACPI, the Advanced Configuration and Power Interface. Turning it off may improve stability, and will enable some versions of the Windows installer to complete successfully.

apic=bool

The APIC, or Advanced Programmable Input Controller,2 is a modern implementation of the venerable PIC. This is on by default. You may want to turn it off if your operating system has trouble with the simulated APIC.

builder=string

With HVM domains, you’ll use the HVM domain builder. With most paravirtualized domains, you’ll want the default Linux domain builder. The domain builder is a bit more low level than the parts that we usually work with. For the most part, we are content to let it do its thing.

device_model=string

The device_model directive specifies the full path of the executable being used to emulate devices for HVM domains (and for PV domains if the framebuffer is being used). In most situations, the default qemu-dm should work fine.

feature=string

This is a pipe-separated list of features to enable in the guest kernel. The list of available features, fresh from the source, is as follows:

[XENFEAT_writable_page_tables] = "writable_page_tables",
[XENFEAT_writable_descriptor_tables] = "writable_descriptor_tables",
[XENFEAT_auto_translated_physmap] = "auto_translated_physmap",
[XENFEAT_supervisor_mode_kernel] = "supervisor_mode_kernel",
[XENFEAT_pae_pgdir_above_4gb] = "pae_pgdir_above_4gb"

We have always had good luck using the defaults for this option.

hap=bool

This directive tells the domain whether or not to take advantage of Hardware-Assisted Paging on recent machines. Implementations include AMD’s nested paging and Intel’s extended paging. If the hardware supports this feature, Xen can substantially improve HVM performance by taking advantage of it.

loader=string

This is the path to HVM firmware. We’ve always been completely satisfied with the default.

pae=bool

This enables or disables PAE on an HVM domain. Note that this won’t enable a non-PAE kernel to run on a PAE or 64-bit box. This option is on by default.

Device Model Options

There are some directives that specify options for the device model. As far as we know, these are specific to the QEMU-based model, but, because no others exist, it seems safe to consider them part of Xen’s configuration.

access_control_policy=POLICY,label=LABEL

The access_control_policy directive defines the security policy and label to associate with the domain.

tpmif=bool

These three variables are all Booleans. If they are enabled, the builder will make the domain a backend for the specified device type.

To use a non-dom0 backend, specify the backend parameter in the definition for your device of choice.

boot=string

Set boot to one of a, b, c, or d to boot from the first floppy, second floppy, hard drive, or CD drive, respectively.

fdb=string

This option specifies the disk image or device file used to emulate the first or second floppy drive—fda and fdb, respectively.

guest_os_type=string

This is the type of the guest OS. It’s a free-form identifier, limited to eight characters.

irq=IRQ

These two options instruct Xen to forward a range of (real) ioports and an IRQ to the domU. The main use for this option that we’ve seen is for serial ports, so that the domU has access to a physical serial port on the server.

keymap=string

The keymap option specifies a keymap file by name. Xen (or rather, the device model) keeps its keymaps under /usr/share/xen/qemu/keymaps. On our machines, the default is en-us.

localtime=bool

This is a simple Boolean option indicating whether the hardware clock is set to local time or GMT.

monitor=string

If monitor is set to yes, the device model will attach the QEMU monitor, which you can use to give commands to the device model. Use CTRL-ALT-2 to break out to the monitor. From there, you can issue commands—try help.

nographic=bool

This indicates whether the device model should use graphics.

serial

serial='file:/filename'

serial='/dev/pts/n'

serial='pty'

serial='stdio'

The serial option specifies a file (or file-like object, such as a named pipe) to use as an emulated serial port. Other options are to have Xen pick a pty, or use STDIN and STDOUT for its serial port; none is also a valid option.

soundhw=bool

This indicates whether to emulate an audio device.

stdvga=bool

If stdvga is set to yes, the device model will use standard VGA emulation. If it’s set to no or omitted, it’ll use emulated Cirrus Logic graphics instead. Ordinarily, the default is just fine.

usb=bool

This is a Boolean value that indicates whether to emulate USB.

usbdevice=HOST:id:id

This item indicates the name of the USB device to add.

Footnotes

¹Terry Pratchett, in Night Watch, has this to say on the subject of passwords: “Every password was ‘swordfish’! Whenever anyone tried to think of a word that no one would ever guess, they always chose ‘swordfish.’ It was just one of those strange quirks of the human mind.”