chris t: July 2008 Archives

New diagram for migration.  I'm actually pretty happy with this one, although I'm sure it'll require some tweaking.  (Perhaps all-lowercase isn't the most professional choice.  Ah well.)

Decided to cut out the "checkpoint" section from migration, on the grounds that we couldn't get it to work.  Maybe we'll get it to work soon?

In the meantime, here it is!

Checkpointing a Domain with xm save

As we mentioned previously, xm save halts a domain in the process of writing its internal state to a save file, and rebuilds it entirely when the save file is restored. With recent versions of Xen, however, the save command supports an option to keep the machine running after it's been saved - a feature called "checkpointing."

Checkpoint a domain thus:

# xm save -c <domain> <checkpoint file>

After the checkpoint's been saved, the domain will continue running.

Although the principle of checkpointing seems simple, it's tricky to actually get it to work in practice. The problem is that the running domain will continue to modify its storage, which causes the saved state to be out of sync with the backing store.

To avoid this problem, you can use a script that hooks into Xen's external device migration framework to checkpoint the domain storage during the save process, as described in the textbox.

[textbox title: Device Migration Hooks]

Xen includes the option of calling an external device migration script at each step of a save or migrate. Although the feature was originally added for the sake of TPM migration, it's since been extended to migrating arbitrary devices.

Enable the external-device-migrate script by setting the external-migration-tool directive in /etc/xen/xend-config.sxp. Most likely you'll want to set (external-migration-tool 'external-device-migrate') and let that script source appropriate scripts for external devices.

[end textbox]

You know that feeling, when you just go through frustration and come out the other side?  Like, the giant structural problems have actually gotten so large that they're part of the landscape, and therefore no longer worth worrying about?

Decided to clear out the discussion of timing attacks from the hosting chapter. I mean, sure, it's an important point. On the other hand, we don't have anything useful to say about them -- you're going to leak some information, in the information-theoretic sense. You can use the "Chinese Wall" policy, but that implies that you can partition your domains into groups with mutual trust -- not an especially likely scenario in the VPS model.

The important thing is really just to remember that information will leak. There's been a lot of work on access controls, but the current Xen policy for dealing with covert channels is still the "Chinese Wall" -- labels to define certain domains that will never be run simultaneously on the same machine. We outline certain controls here, but they're never going to be perfect.

Damnit, I still think we should talk about the security modules at least a little. But I really don't know anything about them.

Today I installed netperf.  I'm not sure about putting it in the book directly -- the package feels like there's some bit rot.  I only mind that a little, but in the book context, everything's got to continue to work for a while at least after it gets published.

I don't know.  It's worked up until now; rationally, that would suggest it'll keep working in the future.  Reason often leads us to conclusions which are untrue, I guess.

Anyway, here's an install procedure:

First, download netperf from http://netperf.org/netperf/DownloadNetperf.html .  We picked up version 2.4.4 .

 # wget ftp://ftp.netperf.org/netperf/netperf-2.4.4.tar.bz2

Untar it and enter the netperf directory.

# tar xjvf netperf-2.4.4.tar.bz2
# cd netperf-2.4.4

Configure, build, and install netperf.  (Note that these directions are a bit at variance with the documentation -- the documentation claims that /opt/netperf is the hard-coded install prefix, whereas it seems to install in /usr/local for me.)

# ./configure
# make
# su

# make install

In the standard configuration, netserver would run under inetd; however, inetd is obsolete.  Many distros don't even include it by default.  Besides, you probably don't want to leave the benchmark server running all the time.

Instead of configuring inetd, therefore, run netserver in standalone mode:

# /usr/local/bin/netserver
Starting netserver at port 12865
Starting netserver at hostname 0.0.0.0 port 12865 and family AF_UNSPEC

(On my system, for some reason, there was no /etc/hosts file.  I populated /etc/hosts with a localhost entry to make the next step work.  I'm hoping your system is better-configured.)

# netperf
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to localhost (127.0.0.1) port 0 AF_INET
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

 87380  16384  16384    10.01    10516.33

Okay, looks good.  Now I'll test from the dom0 to this domU:

# netperf -H 216.218.223.74,ipv4
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 216.218.223.74 (216.218.223.74) port 0 AF_INET
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

 87380  16384  16384    10.00     638.59

Cool.  Not as fast, obviously, but that's kind of to be expected.  Now from another dom0 to this machine:

# netperf -H  216.218.223.74
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 216.218.223.74 (216.218.223.74) port 0 AF_INET
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

 87380  16384  16384    10.12      93.66

Ouch.  Well, so how much of that is Xen, and how much is the network we're going through?

#  netperf -H  216.218.223.66
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 216.218.223.66 (216.218.223.66) port 0 AF_INET
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

 87380  16384  16384    10.25      87.72

Huh.  Could be worse, I guess.  Of course, other tests, with varying, for example, packet sizes, might tell us more.  But that's a start for our performance measure.

I've been spend a lot of time in the copyediting galley, listening to that tireless drum and manning a sweep, trudging back and forth across the bare and splintered deck.