Migrating without shared storage

Migration Principles

As we all know, migration requires some form of network storage. Unfortunately, this introduces a layer of storage complexity that may not be suited to all Xen deployments.

What we want is some mechanism by which a domain can, at migration time, have its storage copied over to the target host in such a way as to allow seamless migration.

Migration Steps

1. Host A sends a request to Host B.
2. Host B reserves memory and checks for the availability of storage.
3. Iterative pre-copy -- Domain runs on Host A.
4. Stop and copy.
5. Domain running on Host B.

What we need is a hook that Xend calls after the domain's stopped, which transfers the storage between Host A and Host B.

External-device-migrate

In theory, the external-device-migrate script (in the /etc/xen/scripts directory) is the answer. This script was originally meant for TPM migration, but can be extended to handle any sort of virtual device -- Xend calls it after each migration step with the device type, a handle, and the migration step. You can add calls to further scripts as needed.

External-device-migrate doesn't seem to work with RHEL/CentOS. I would be very interested in getting it functional.

DRBD

The distributed replicated block device (DRBD) people include a block-drbd script that allows you to use drbd: block devices directly as domU backing store.

A blog post [1] claims that DRBD enables Xen live migration, probably through a similar mechanism. I believe, based on perusal of the block-drbd script, that they mean it "works for two-node setups," but I could be wrong. (Since that would imply no benefit over specifying a DRBD phy: device.) Workflow should run as follows:

1. Domain running on Host A with generic DRBD backing store
2. Initiate migration. Host B checks storage requirement.
3. Stop and copy
4. Host B creates backing store, syncs it with Host A
5. Domain resumes on Host B with local store.

Alternatively the domain could begin running on Host B immediately and rely on the DRBD backend to transfer data to B's mirror as needed.