Xen 4.x Testing and Unstable packager for Ubuntu

Hello guys. Finally an update (2011-07-10) that was looong due.
Now also builds 4.1.1 and 4.0.2.

This is an attempt to make the simplest possible Ubuntu packager for releases or top (rc0) in xen-4.1-testing.hg(default), xen-4.0-testing.hg and xen-unstable.hg repositories.
Run by:

mkdir xen4x; cd xen4x
wget -q -O - https://virtualusr.files.wordpress.com/2011/07/\
mk-xen40x-1107091941.doc | catdoc > .sh; sh .sh

# to build xen-hypervisor run # debian/bin/packageXen4x.sh
# to build xen-kernel     run # debian/bin/kpkgXenKernel.sh

More usage examples can be shown by:
head debian/bin/packageXen4x.sh
head debian/bin/kpkgXenKernel.sh

As I was asked to port minimal set of patches to Xen unstable repository, I got forced to rewrite all scripts to be able to work with both Testing and Unstable repositories. Inspired by current Debian 4.0 source, I also completely rewrote debian/rules to gain control over each dh_* command ran for each resulting .deb file.

While analyzing python patch (not on debian), I found out that it can be dropped if I use the newer recommended dh_pysupport to pack python modules. Another patch eliminated was debian-stubdom.Makefile.diff by calling make in debian/rules as “env -u LDFLAGS make install-stubdom”. This rewrite results in all Xen-4.x builds being simpler for -avoiding- these two patches.

Xen unstable 4.1.x has stronger cross-platform support, so only one patch is currently needed to package the source (bug in /etc/init.d/xen-watchdog stop).

This proves that we are getting to the end of patch simplification for raw xensource debian packages. I was thinking about possibile debian/control simplification by joining xen-libs and python-xen packages to xen-utils package, but I am afraid that might trigger dependency problems I would not have the time to maintain. Maybe sometime in the future taking Debian Xen 4.0 source as a base for patch minimization, but I am afraid this might not make much sense either. Especially if Debian becomes very stable, and/or taken over by Ubuntu, this whole blog looses it’s meaning.

As Xen unstable repository had no release tags as I started, rc0 is used to indicate the current top of the repository (obviously varies with the time of taking the snapshot). All builds made by this tool default to debug=n, except 4.1.0-rc1 to debug=y due to upstream bug. For more notes on this rewrite and new (4.1.1) recommended network setup see debian/template/NOTES .

Tested building 4.1.0-rc1 and most releases 4.0.[012]-rc* on Ubuntu 9.10 and 10.04, 2011-01-20 – 2011-07-05 running Xen-4.0.2-rc1 Domain-0 on Ubuntu 9.10 due to local graphic card driver issue.
Since 2011-07-05 running 4.1.1 no patches whatsoever only packaged into deb-s.

Comments are welcome.

Enjoy Ubuntu Xen4.x servers

Contrat Creative Commons

(Prev.ver. https://virtualusr.files.wordpress.com/2011/01/mk-xen40x-1101222338.doc )

Advertisements
This entry was posted in Xen and tagged , . Bookmark the permalink.

11 Responses to Xen 4.x Testing and Unstable packager for Ubuntu

  1. Anonymous says:

    On my amd64 Debian Squeeze system I was able to complile Xen 4.1.-rc2-pre (make xen, make tools, make stubdom) on Jan.18th 2011, changeset: 22779:d1631540bcc4 from the mericurical repository at: http://xenbits.xen.org/staging/xen-unstable.hg

    — a/Config.mk Tue Jan 18 17:23:24 2011 +0000
    +++ b/Config.mk Fri Jan 21 10:34:16 2011 +0000
    @@ -1,7 +1,8 @@
    # -*- mode: Makefile; -*-

    # A debug build of Xen and tools?
    -debug ?= y
    +#debug ?= y
    +debug ?= n

    XEN_COMPILE_ARCH ?= $(shell uname -m | sed -e s/i.86/x86_32/ \
    -e s/i86pc/x86_32/ -e s/amd64/x86_64/)
    @@ -56,7 +57,8 @@
    FLEX ?= flex

    PYTHON ?= python
    -PYTHON_PREFIX_ARG ?= –prefix=”$(PREFIX)”
    +PYTHON_PREFIX_ARG ?=
    +#PYTHON_PREFIX_ARG ?= –prefix=”$(PREFIX)”
    # The above requires that PREFIX contains *no spaces*. This variable is here
    # to permit the user to set PYTHON_PREFIX_ARG to ” to workaround this bug:
    # https://bugs.launchpad.net/ubuntu/+bug/362570

  2. virtualusr says:

    Hi!
    Ok debug=n seems to work now. Thanks. I guess …

    DEVEL=true debian/bin/packageXen4x.sh 4.1.0-rc0
    # rc0 selects top which is now 4.1.-rc2-pre
    CDNM=$(sed -n 's/DISTRIB_CODENAME=//p' /etc/lsb-release)
    cd /usr/src/build-$CDNM/xen-4.1-4.1.0rc0
    debug=n dpkg-buildpackage -rfakeroot -b

    would get the same 4.1.-rc2-pre to be packaged into debian packages called 4.1.0rc0 (rc0=top).

    However, I can not select untagged pre-releases, changesets like yours : 22779:d1631540bcc4, only the current top; well, not yet.

  3. Bruce Edge says:

    Nice job on picking up 4.1. I’ve been following your progress on xen-dev.

    Why don’t you setup a debian/ubuntu repository and post make binaries available? There are no sites that I can find that are providing pre-built .debs for either distro. Debian might be easier as they have 4.0 in the squeeze release already.

    -Bruce

  4. virtualusr says:

    While I was polishing patches for upstream during past few days, I happened to break debian-sysconfig-default.diff
    If you get anything answered when you execute the line
    grep /etc/sysconfig /etc/init.d/xend* 2>/dev/null && \
        echo ERROR PLEASE REBUILD THE PACKAGES

    … please rebuild the packages by using mk-xen40x-1101222338.doc or latter.

  5. virtualusr says:

    I have no affiliation with neither debian or ubuntu, just their user. This package is my personal solution to running xen. I want to avoid manual cluttering with the sources and installing unpackaged binaries.

    My xen-utils package is allinclusive as plain xensource, nothing is (should be) missing. For example: you already get xen-qemu-dm-4.0 included in my package.

    It is meant to function identically as if one would build, install and run xensources on debian based machine. No patches that are not absolutely necessary (except on 4.0 patched /etc/init.d/xend and some extra documentation build Makefile).
    One should be able to ask mail list xen-users for support.

    Only thing missing might be unfulfilled dependencies for other packages depending on debian xen packages, but this should many users be capable of tweaking themselves in the debian/control (my debian/tempate/control-4.1 which gets auto-edited-downgraded for version 4.0 also)

    You can post your corrections to dependencies here and I’ll eventually include them.

  6. thiagocmc says:

    Hi brother!

    Here is Thiago, of the Xen-Users email list! I’ve made the Xen Live CD v2.0 based on Debian! How are you?!

    I need to ask you something! Let’s help the community and give to them the BEST Xen packaged out there?! We can do it together! What do you think?!

    The Ubuntu 11.10 Aplha2 was released: https://wiki.ubuntu.com/OneiricOcelot/TechnicalOverview/Alpha2

    And they need to much tests with new Xen, with Linux 3.0! If the Ubuntu 11.10 receives our help, I’m pretty sure that the next Ubuntu will provide the best Xen experience of the open source community…

    I agree with did you said, the package doesn’t need a lot of patchs and the HVM domains should come from the main package… Closest as possible with the Xen compiled and installed from sources.

    If you’d like to do this, let me know! I can provide a server to our test environment…

    Meu email is: thiagocmartinsc@gmail.com

    Cheers!
    Thiago

  7. virtualusr says:

    Hi Thiago. I was teased earlier by Bruce to provide debian/ubuntu repository with packages. If you can provide access to public IP http server I could slowly build and upload 4.1.1 for all debian and ubuntu versions on my newly established sbuild factory. Actually it could be done quite fast… All source and binary packages build specially on (and for) each debian/ubuntu release. Regards

  8. Bruce says:

    HI all.
    As long as I’m being singled out by name I might as well comment 🙂
    I think it’s a fantastic idea to put prebuilt binaries out there for the latest & greatest. While not a huge hindrance for the devs, it’ll stop a lot of rookies from trying it.
    Also, not sure if you saw this or not, but I got a patch in to the xen build system that lets you set a few env vars to control where the /etc/init.d stuff and a few other things go so that you can generate debian compatible dir structures without a lot of patching and/or post build moving stuff around. It worked for me, but I was doing a hack and putting everything in one binary package. I guess it’s a toss up as to whether it’s worth reworking the upstream debian patch to not rearrange everything or just keep doing it the way it is. It should be pretty obvious, but let me know if you want some instructions for the debian build layout options.
    Keep up the good work.

    -Bruce

    • virtualusr says:

      I guess it would be interesting for everibody if you still had your instrustions, and/or logs og your latest attempts. Please share them if you can.

  9. virtualusr says:

    Hi Bruce! I noticed it JUST WORKS (R) as of 4.1.1. Thank for that. No more patches after 4.1.1. :). You can mail me your experiences and I’ll check them out.

  10. Matthew Drobnak says:

    Hey, just wanted to let you know I found your scripts very useful when testing on Ubuntu 11.04. The Xen packaging one worked fine. The Kernel one had trouble because of the repos disappearing at kernel.org.
    The only line in your script I changed was the git clone line to:
    git clone https://github.com/jsgf/linux-xen.git linux-2.6-xen

    and the branch is now called “next-2.6.32”

    So far it seems to be building. It was frustrating to have what seemed like a working system to realize that Ubuntu didn’t build the backend drivers in for networking. Or at least it seems that way. This my first experience setting up Xen on my own, and I had the “hotplug scripts not working” error.

    Anyway, thanks again!

    -Matt

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s