Xen 4.x Testing and Unstable packager for Ubuntu

Hello guys. Finally an update (2011-07-10) that was looong due.
Now also builds 4.1.1 and 4.0.2.

This is an attempt to make the simplest possible Ubuntu packager for releases or top (rc0) in xen-4.1-testing.hg(default), xen-4.0-testing.hg and xen-unstable.hg repositories.
Run by:

mkdir xen4x; cd xen4x
wget -q -O - https://virtualusr.files.wordpress.com/2011/07/\
mk-xen40x-1107091941.doc | catdoc > .sh; sh .sh

# to build xen-hypervisor run # debian/bin/packageXen4x.sh
# to build xen-kernel     run # debian/bin/kpkgXenKernel.sh

More usage examples can be shown by:
head debian/bin/packageXen4x.sh
head debian/bin/kpkgXenKernel.sh

As I was asked to port minimal set of patches to Xen unstable repository, I got forced to rewrite all scripts to be able to work with both Testing and Unstable repositories. Inspired by current Debian 4.0 source, I also completely rewrote debian/rules to gain control over each dh_* command ran for each resulting .deb file.

While analyzing python patch (not on debian), I found out that it can be dropped if I use the newer recommended dh_pysupport to pack python modules. Another patch eliminated was debian-stubdom.Makefile.diff by calling make in debian/rules as “env -u LDFLAGS make install-stubdom”. This rewrite results in all Xen-4.x builds being simpler for -avoiding- these two patches.

Xen unstable 4.1.x has stronger cross-platform support, so only one patch is currently needed to package the source (bug in /etc/init.d/xen-watchdog stop).

This proves that we are getting to the end of patch simplification for raw xensource debian packages. I was thinking about possibile debian/control simplification by joining xen-libs and python-xen packages to xen-utils package, but I am afraid that might trigger dependency problems I would not have the time to maintain. Maybe sometime in the future taking Debian Xen 4.0 source as a base for patch minimization, but I am afraid this might not make much sense either. Especially if Debian becomes very stable, and/or taken over by Ubuntu, this whole blog looses it’s meaning.

As Xen unstable repository had no release tags as I started, rc0 is used to indicate the current top of the repository (obviously varies with the time of taking the snapshot). All builds made by this tool default to debug=n, except 4.1.0-rc1 to debug=y due to upstream bug. For more notes on this rewrite and new (4.1.1) recommended network setup see debian/template/NOTES .

Tested building 4.1.0-rc1 and most releases 4.0.[012]-rc* on Ubuntu 9.10 and 10.04, 2011-01-20 – 2011-07-05 running Xen-4.0.2-rc1 Domain-0 on Ubuntu 9.10 due to local graphic card driver issue.
Since 2011-07-05 running 4.1.1 no patches whatsoever only packaged into deb-s.

Comments are welcome.

Enjoy Ubuntu Xen4.x servers

Contrat Creative Commons

(Prev.ver. https://virtualusr.files.wordpress.com/2011/01/mk-xen40x-1101222338.doc )

Posted in Xen | Tagged , | 11 Comments

Xen 4.0.*.rc* packager for Ubuntu

NOTE: Depreciated in favor of the next post:
https://virtualusr.wordpress.com/2011/01/20/xen-4-x-testing-unstable-ubuntu/

This is an attempt to make simplest possible Ubuntu packager for releases in xen-4.0-testing.hg repository. This is to be able to enjoy latest xen-testing releases while using debian package upgrades for both Xen-hypervisor and xen-kernels.

During the new year holidays a I made a new version of my Xen packager for Ubuntu. No big changes in resulting packages, mostly cleanup and consolidation. Major changes from the previous version:

  • Patches documented and adapted with conditional expansion to be hopefully included upstream. One patch backported from xen-devel and one old replaced by Debian Xen4.0 source version.
  • Debian source format raised to ‘3.0 (quilt)’
  • Everything needed is consolidated in one debian/* tree and can be run from there. Everything gets also included in resulting source package.
  • Kernel packager makes kernel source package first, then builds it in /usr/src/ so resulting links from /lib/modules point correctly to the source.
  • All inclusive Xen-source package is downloaded, but the following configuration in tools/Makefile (target ioemu-dir-find: ioemu-dir/xen-setup) is skipped, resulting in cleaner source.orig.

TODO:   Mark Pryor mailed me a patch http://paste.ubuntu.com/544701/ to enable builds on the coming Ubuntu Natty. No time this time. If you need it, it should not be to hard to place it into debian/patches/ and add it to debian/patches/series.

Tested building all releases 4.0.[01].rc* on Ubuntu 9.10 and 10.04, running Xen-4.0.1 Domain-0 on Ubuntu 9.10 due to local graphic card driver issue.

Run by

mkdir xen4x; cd xen4x
wget -q -O - \
https://virtualusr.files.wordpress.com/2011/01/mk-xen40x-1101032202.doc | \
catdoc > .sh; sh .sh

#build xen-hypervisor # debian/bin/packageXen40x.sh
#build xen-kernel     # debian/bin/kpkgXenKernel.sh

More usage examples can be shown by:
head debian/bin/packageXen40x.sh
head debian/bin/kpkgXenKernel.sh

Comments are welcome.

Enjoy Ubuntu Xen4.0.1 server

Contrat Creative Commons

Posted in Xen | Tagged , | 2 Comments

Flying Linux Containers on Xen 4.0.* on PV Ubuntu 10.04 Lucid

Do you have 3 minutes and 17 seconds of your time to spend, well, and some download time?

I was dreaming for some time now about a possibility to run virtualized machines inside one PV xen instance in a cloud like Amazon or similar… That would be great, but several references on the net state that ex. amazons cloud can not – will not run dom0 kernels. Lately I found a reference stating that it became possible to run own kernels like ones having container capabilities…, well not dom0 but… It would certainly be exciting to see Linux Containers “lightest” virtualized birds on “heaviest” Xen, don’t you think so? (“heaviest” == coolest 😉 )
Unfortunately making that work on my test Xen PV instance proved not to be so easy…

  • PV Ubuntu 10.04 Lucid with stock kernel linux-image-2.6.32-24-generic-pae (didn’t know that will also work as PV) running on
  • Ubuntu 9.10 Karmic xen-hypervisor-4.0 4.0.1rc5-0ubuntu1 and dom0 kernel linux-image-2.6.32.17-xen v. 100803222122.
  • Latter tested on a real system without Xen and confirmed to work just as well.

As you can see at the moment you can find far more complaints about (im)maturity and usability of Linux Containers (LXC) than working examples. Actually none of them really worked for me. Closest came lxc-ubuntu script (Copyright (C) 2010 Nigel McNie – Modified for Ubuntu 2010 phbaer), but even that didn’t not work for me for interactive ssh sessions and console, only scp. I took more than two weeks of my time to get into intricacies of LXC and it wasn’t an easy task, but I think the result is worth it. As I couldn’t find working lxc ubuntu template anywhere I patched the one from latest ubuntu lxc testing sources to make it work and a bit more Ubuntu like (most notable first user in admin group and no root logins).

I added one script to create networking environment on hosting PV instance that could be used remotely in the cloud while not messing up with the working default gateway interface as we might loose connectivity.

# export LXCBASE=.../  # default demo base root dir
LXCBASE=${LXCBASE:-/lxc}; sudo mkdir -p $LXCBASE
wget -q -O - https://virtualusr.files.wordpress.com/2010/09/ubuntu1004lxcflight-100908.doc | \
catdoc > .r.sh; sh ./.r.sh; rm .r.sh; sudo tar xzvf Ubuntu1004LXCflight.tgz -C $LXCBASE

The following does quite some weight lifting so make sure it doesn’t brake something in your environment before you run it. You can copy paste it line by line to test or run the whole script ./runscript

#!/bin/sh
# runscript
# (C) 2010 Virtual User <virtualusr@gmail.com>, GPLv2

# export LXCBASE=.../
LXCBASE=${LXCBASE:-/lxc}  # default demo base dir

#cd to lxc demo directory
cd $LXCBASE

### Cleanup of previous demo attempt / environment
# purge previous myUbuntu demo container
lxc-stop -n myUbuntu; lxc-destroy -n myUbuntu
rm -rf myUbuntu
# Purge previous cache of lxc-ubuntu script
#rm -r /var/cache/lxc/ubuntu/
# Remove previous bridge br0
ifconfig br0 down; brctl delbr br0
# make backup of current dhcp3 & bind9
tar czvf lxc-demo-backup-$(date +%y%m%d%H%M).tgz /etc/bind/ /etc/dhcp3/ /etc/default/dhcp3-server
# Purge lxc demo related software
apt-get -y purge lxc dhcp3-server bind9

# Demo START
# Make localized config file
head -n15 ./my-lxc-environ.sh | \
        sed "s/\bserver1\b/$(hostname)/" > $(hostname)-lxc-environ.sh
# Create lxc environment
apt-get -y install lxc dhcp3-server bind9 expect tcllib
./my-lxc-environ.sh # Also installs lxc dhcp3-server bind9
# Local test config for dhcpd
cat << EOF | tee -a /etc/dhcp3/dhcpd.conf 

subnet 192.168.199.0 netmask 255.255.255.0 {
  range 192.168.199.20 192.168.199.30;
  option domain-name-servers 192.168.199.1;
  option domain-name "lan";
  option routers 192.168.199.1;
  option broadcast-address 192.168.199.255;
  default-lease-time 600;
  max-lease-time 7200;
}

host myUbuntu{
  hardware ethernet 86:d7:66:8a:04:90;
  fixed-address 192.168.199.20;
}
EOF
sed -i 's/INTERFACES=""/INTERFACES="br0"/' /etc/default/dhcp3-server
/etc/init.d/dhcp3-server restart

# Create an image of a new container with working Ubuntu Lucid
firstuser=me firstpass=mypassword ./lxc-ubuntu -p /$LXCBASE/myUbuntu

# Add MAC hwaddr to config to keep the same IP on container restarts
sed -i "/^lxc.network.link =/ a\\
lxc.network.hwaddr = 86:d7:66:8a:04:90" myUbuntu/config

# Copy X install script into home directory of the admin user in the new container.
cp -va install.x.sh myUbuntu/rootfs/home/me/
# Set vncpasswod of the admin user
expect_run() { cmd64=$(echo -n $1 | base64); pas64=$(echo -n $2 | base64)
expect -c 'package require base64
set cmd [base64::decode "'$cmd64'"]
set pas [base64::decode "'$pas64'"]
if 1 [linsert $cmd 0 spawn ]
while true {
  expect -re "Password:$" { send "$pas\r"} \
  expect -re "Verify:$" { send "$pas\r"} \
  expect -re "s password: $" { send "$pas\r"} \
  expect -re "password for *: $" { send "$pas\r"} \
  eof { break}
}'; }
mkdir -p myUbuntu/rootfs/home/me/.vnc
expect_run "vncpasswd myUbuntu/rootfs/home/me/.vnc/passwd" mypassword
chown -Rv 1000:1000 myUbuntu/rootfs/home/me/.vnc

# Expand var/cache/apt/archives saved by
# tar cf aptcache.tar myUbuntu/rootfs/var/cache/apt/archives/*.deb
[ -r ./aptcache.tar ] && tar xf ./aptcache.tar

# Create & start LXC container
lxc-create -n myUbuntu -f /$LXCBASE/myUbuntu/config
lxc-start -n myUbuntu -d; sleep 15

# ssh into it, install very basic X workstation & start it
expect_run "ssh -t me@192.168.199.20 ./install.x.sh" mypassword
expect_run "ssh me@192.168.199.20 /usr/bin/vncserver -localhost" mypassword

# Connecting instructions
EXTERNAL_INTERFACE=$(ip route | sed -n 's/  *metric.*//;s/default via [0-9.]* dev //p')
EXTERNAL_IP=$(ip addr show dev $EXTERNAL_INTERFACE | sed -n 's|/.*||;s/ *inet //p')
cat << EOF
### Contact internal container by NAT through external interface $EXTERNAL_INTERFACE
### from anywhere by
sudo apt-get -y install  xvnc4viewer openssh-client
ssh -N -f -p 2220 me@$EXTERNAL_IP -L 5911:127.0.0.1:5901
SSHPID=\$(ps ax | grep -v grep | \\
  grep "ssh -N -f -p 2220 me@$EXTERNAL_IP -L 5911:127.0.0.1:5901" | sed 's/^ *//;s/ .*//')
vncviewer localhost:11  # port 5911 tunneled above

### Remeber to kill ssh tunelling client running in background when
### you are finished using the vncviewer by:
kill \$SSHPID
EOF

I wonder how long does it take? (all packages already downloaded to cache)

time ./runscript
# . . .
### Contact internal container by NAT through external interface eth0
### from anywhere by
sudo apt-get -y install  xvnc4viewer openssh-client
ssh -N -f -p 2220 me@209.85.227.104 -L 5911:127.0.0.1:5901
SSHPID=$(ps ax | grep -v grep | \
  grep "ssh -N -f -p 2220 me@209.85.227.104 -L 5911:127.0.0.1:5901" | sed 's/^ *//;s/ .*//')
vncviewer localhost:11  # port 5911 tunneled above:

### Remeber to kill ssh tunelling client running in background when
### you are finished using the vncviewer by:
kill $SSHPID

real    3m16.794s
user    0m4.816s
sys     0m8.473s

Let me know if anyone makes it actually fly in the Amazons cloud. Any suggestions how to test this on some free cloud infrastructure?

3m16.794s … Was this one perfect flight or what?Enjoy your flying

Contrat Creative Commons
Posted in LXC, Xen | Leave a comment

Xen 4.0.*.rc* source & kernels packaged for Ubuntu

The whole procedure can be run by simple:

wget -q -O - https://virtualusr.files.wordpress.com/2010/09/mk-xen40x-1009281438.doc | \
catdoc > .r.sh; sh ./.r.sh; rm .r.sh

Ubuntu-package latest “xen 4.0.*”:

./packageXen40x.sh RELEASE-4.0.0  # or
./packageXen40x.sh 4.0.1-rc5      # or
./packageXen40x.sh                # latest tagged release, today 4.0.1

Ubuntu-package latest “xensource” kernels by: (prefered way)

#   ./kpkgXenKernel.sh stable-2.6.32.x 100706173022  #or# b97a312997
#   ./kpkgXenKernel.sh stable-2.6.32.x 100430201127  #or# 4dd582f35f
#   ./kpkgXenKernel.sh stable-2.6.31.x 100402212843  #or# f3d5fe8050
#   ./kpkgXenKernel.sh stable-2.6.33.x # picks up the last commit

or older vanilla kernel way to Ubuntu-package latest “xensource” kernels by:

# ./remakeXenKernel.sh stable-2.6.32.x 100706173022  #or# b97a312997

* The above are commit examples known to more or less work
* List of all available (last1000) commit-id-s sorted by UTC time stamp of the CommitDate: to xen/stable-2.6.32.x can be seen here /usr/src/build/xen01/stable-2.6.32.x-*.git.log . Feel free to test any of them.

It feels like this is maturing into a quite useful tool. Changes since the last post:

  • Corrected bug in sorting releases code; final release gets sorted as bigger than all release candidates ( 4.0.1 > 4.0.1-rc9). Previous code (not tested enough) tried to apply patches of 4.0.0 to 4.0.1, now correctly applies patches from 4.0.1-rc1
  • One diff.gz builds all Xen 4.0.* releases. Patches from xen-4.0_4.0.1rc1 proves to be applicable to newer releases. Script has been modified to automatically apply latest patches in attempt to build all later releases. Minor change since last update: final release gets sorted as bigger than all release candidates ( 4.0.1 > 4.0.1-rc9).
  • Xen .orig sources are including everything needed to build without network connectivity.
  • Linux kernel can be easily manually reconfigured, build resumes automatically when configuration is finished.
  • Provided config file is picked up to build the kernel. (Example: in-config-linux-2.6-xen-stable-2.6.32.x-100517202304_amd64)
  • Updated 09_xen for grub2 (in grub2setup.tgz)
  • Just a note… it seems that standard ubuntu kernels linux-ec2 (linux-image-2.6.32-308-ec2) also work perfectly as the domU kernels

ToDo:

  • Links in linux-image*-xen*.deb -s still point to wrong build directory. Should be like
    /lib/modules/2.6.32.16-xen/build -> /usr/src/linux-headers-2.6.32.16-xen
    Can be probably fixed by first only creating source package and then build from there the image. (No time this time)

I am running ssvnc ovet SSL on laptop and dom0 (hypervisor Xen4.0.1rc3) to access x11vnc xsession on domU . I have two issues:

  1. With the above recommended xensource kernel release linux-image-2.6.32.16-xen_100706173022_amd64.deb … I finally get one latest release that does not brake my Ubuntu karmic dom0 xserver (01:05.0 VGA compatible controller: ATI Technologies Inc Device 9710), works great also as domU. I got the courage to upgrade dom0 to lucid, but here xserver session still breaks (not loading the keyboard driver – an issue from some post on xen.org, and mailinglist) In my case, if I want to use graphic screen on my dom0 I am forced to keep karmic as dom0. 😦
  2. All kernels after 2.6.31.12-xen do not set international keyboard mappings correctly (maybe related to the above). I have to set them manually after startx by
    # setxkbmap -layout de -print | xkbcomp – ${DISPLAY%%.*} 2> /dev/null
    After that I press Ctrl+P and some more Ctr-keys until terminal dies (only once). Then everything seems to work as normal.

Comments are very welcome.

Enjoy Ubuntu Xen4.0.1 server

Contrat Creative Commons

Posted in Xen | Tagged , | 20 Comments

Xen 4.0.1-rc2 … source & kernels packaged for Ubuntu

The whole procedure can be run by simple:

wget https://virtualusr.files.wordpress.com/2010/06/mk-xen40x11.doc
catdoc mk-xen40x11.doc > mk-xen40x.sh
sh mk-xen40x.sh

Ubuntu-package latest “xen 4.0″ by:

./packageXen40x.sh RELEASE-4.0.0    # or
./packageXen40x.sh 4.0.1-rc1    # or
./packageXen40x.sh # latest tagged release, today 4.0.1-rc3

Ubuntu-package latest “xensource” kernels by:

# ./remakeXenKernel.sh stable-2.6.31.x 100402212843
#or# ./remakeXenKernel.sh stable-2.6.31.x f3d5fe8050
# ./remakeXenKernel.sh stable-2.6.32.x 100430201127
#or# ./remakeXenKernel.sh stable-2.6.32.x 4dd582f35f
# ./remakeXenKernel.sh stable-2.6.32.x 100517202304
#or# ./remakeXenKernel.sh stable-2.6.32.x ba739f9abd
# ./remakeXenKernel.sh stable-2.6.33.x # picks up the last commit

The same can be done even more Ubuntu-Debian-way with
# ./kpkgXenKernel.sh stable-2.6.32.x 100517202304

* The above are commit examples known to more or less work
* List of all available (last1000) commit-id-s sorted by UTC time stamp of the CommitDate: to xen/stable-2.6.32.x can be seen here /usr/src/build/xen01/stable-2.6.32.x-*.git.log .  Feel free to test any of them and let us know if they are any good.

I have noticed that most of commits are broken. Does anybody know how to track better working commits on kernel git repository?

It feels like this is maturing into a quite useful tool. Changes since the last post:

  • Latest xen-4.0_4.0.1rc1-0ubuntu1.diff.gz proves to be applicable to newer release. Script has been modified to automatically apply latest diff.gz in attempt to build later release candidates. Let’s hope it will work for some time… It would probably build RELEASE-4.0.0 also (only grub2 patch would be missing).
  • Some examples for my grub2 config, and latest ssvnc and x11vnc packages are also included.

I am running ssvnc ovet SSL on laptop and dom0 to access x11vnc xsession on  domU. I have two bigger issues:

1. gnome-session manager or openbox-session break international keyboard mappings on x11vnc session. Openbox without session manager works.

2. All kernels after 2.6.13.12-xen do not set international keyboard mappings correctly. I have to set them manually after startx by

# setxkbmap -layout de -print | xkbcomp – ${DISPLAY%%.*} 2> /dev/null

After that I press Ctrl+P and some more Ctr-keys until terminal dies (only once). After that everything seems to work as normal.

Comments on my config examples are very welcome.

Enjoy Ubuntu Xen4.0 server

* Updated 2010-06-19  Load the xen-evtchn module only if needed, if device /dev/xen/evtchn is missing.

* Updated 2010-06-21  Major cleanup. One diff.gz for all versions. Creates all inclusive orig.source.gz skipping downloads on repeated builds.

* Updated 2010-06-30

– Added the very important missing xen-4.0-0ubuntu1.diff.gz for all versions. This was a big mistake making this post unusable since the 21st of Jun, SORRY.

– One line was missing in packageXen40x.sh to update xen repository every time it is run without version argument. Previous version could not automatically pick up 4.0.1-rc3, this can and does the build without error.

– Bruce Edge reminded me to retry standard ubuntus “make-kpkg kernel-image” which was broken when I started. Now it seems to be working again. I added a script for really Ubuntu-way of packaging kpkgXenKernel.sh. It’s practically the same as the older remakeXenKernel.sh except the “make-kpkg (Ubuntu) vs deb-pkg (Linux Kernel source)”.
If one of them fails to build you can try the other.

Posted in Xen | 21 Comments

Xen 4.0.1-rc1 source & kernels packaged for Ubuntu

The whole procedure can be run by simple:

wget https://virtualusr.files.wordpress.com/2010/06/mk-xen40x6.doc
catdoc mk-xen40x6.doc > mk-xen40x.sh
sh mk-xen40x.sh

Ubuntu-package latest “xen 4.0” by:

./packageXen40x.sh RELEASE-4.0.0    # or
./packageXen40x.sh 4.0.1-rc1    # or
./packageXen40x.sh # latest tagged release, today same as above

Ubuntu-package latest “xensource” kernels by:

./remakeXenKernel.sh stable-2.6.31.x~1
./remakeXenKernel.sh stable-2.6.32.x
./remakeXenKernel.sh stable-2.6.33.x

Any corrections and suggestions are very welcome.

WARNING: No warranty whatsoever, this happens to work for me, while it may or may not work for you.

Enjoy Ubuntu Xen4.0 server

* Updated 2010-05-28: Corrected package dependency error preventing install of the latest 4.0.1rc1-0ubuntu1. I am running this at the moment.

* Updated 2010-06-03: Murphies Law, same error again, corrected for good I hope. I will reinstall my system with the published version to test for the 3rd time.

Posted in Xen | Tagged , | 10 Comments

Xen-4.0 source & kernels packaged for Ubuntu

I’ve spent a couple of months trying to pack some recent Xen releases to ubuntu-debian packages.

The whole procedure can be run by simple:

wget https://virtualusr.files.wordpress.com/2010/05/mk-xen400.doc
catdoc mk-xen400.doc > mk-xen400.sh
sh mk-xen400.sh

mk-Xen400 – Ubuntu debian packager for Xen-4.0 and kernels

Ubuntu-package latest “xen 4.0.0” by:

./packageXen400.sh

Ubuntu-package latest “xensource” kernels by:

./remakeXenKernel.sh stable-2.6.31.x~1
./remakeXenKernel.sh stable-2.6.32.x
./remakeXenKernel.sh stable-2.6.33.x

Tested on Ubuntu 9.10 and 10.4 with 64bit AMD 2 core CPU (AMD Athlon(tm) II X2 250). Intel-64bit and i686-32bit not tested, might need some tweak in remakeXenKernel.sh where kernels .config is autogenerated. Tested running 32 bit OpenSUSE 11.2 and 64bit Ubuntu 10.4 as a domU-s in 64bit Ubuntu server 9.10 dom0.

Enjoy Ubuntu Xen4.0 server

.

Reference:

https://bugs.launchpad.net/ubuntu/+source/xen-3.3/+bug/378240
mikmak  wrote on 2009-08-13: xen-3.4_3.4.1-1ubuntu1.diff.gz  (25.9 KiB, text/x-diff)

.

Posted in Xen | Tagged | Leave a comment