Xen 4.0.*.rc* source & kernels packaged for Ubuntu

The whole procedure can be run by simple:

wget -q -O - https://virtualusr.files.wordpress.com/2010/09/mk-xen40x-1009281438.doc | \
catdoc > .r.sh; sh ./.r.sh; rm .r.sh

Ubuntu-package latest “xen 4.0.*”:

./packageXen40x.sh RELEASE-4.0.0  # or
./packageXen40x.sh 4.0.1-rc5      # or
./packageXen40x.sh                # latest tagged release, today 4.0.1

Ubuntu-package latest “xensource” kernels by: (prefered way)

#   ./kpkgXenKernel.sh stable-2.6.32.x 100706173022  #or# b97a312997
#   ./kpkgXenKernel.sh stable-2.6.32.x 100430201127  #or# 4dd582f35f
#   ./kpkgXenKernel.sh stable-2.6.31.x 100402212843  #or# f3d5fe8050
#   ./kpkgXenKernel.sh stable-2.6.33.x # picks up the last commit

or older vanilla kernel way to Ubuntu-package latest “xensource” kernels by:

# ./remakeXenKernel.sh stable-2.6.32.x 100706173022  #or# b97a312997

* The above are commit examples known to more or less work
* List of all available (last1000) commit-id-s sorted by UTC time stamp of the CommitDate: to xen/stable-2.6.32.x can be seen here /usr/src/build/xen01/stable-2.6.32.x-*.git.log . Feel free to test any of them.

It feels like this is maturing into a quite useful tool. Changes since the last post:

  • Corrected bug in sorting releases code; final release gets sorted as bigger than all release candidates ( 4.0.1 > 4.0.1-rc9). Previous code (not tested enough) tried to apply patches of 4.0.0 to 4.0.1, now correctly applies patches from 4.0.1-rc1
  • One diff.gz builds all Xen 4.0.* releases. Patches from xen-4.0_4.0.1rc1 proves to be applicable to newer releases. Script has been modified to automatically apply latest patches in attempt to build all later releases. Minor change since last update: final release gets sorted as bigger than all release candidates ( 4.0.1 > 4.0.1-rc9).
  • Xen .orig sources are including everything needed to build without network connectivity.
  • Linux kernel can be easily manually reconfigured, build resumes automatically when configuration is finished.
  • Provided config file is picked up to build the kernel. (Example: in-config-linux-2.6-xen-stable-2.6.32.x-100517202304_amd64)
  • Updated 09_xen for grub2 (in grub2setup.tgz)
  • Just a note… it seems that standard ubuntu kernels linux-ec2 (linux-image-2.6.32-308-ec2) also work perfectly as the domU kernels

ToDo:

  • Links in linux-image*-xen*.deb -s still point to wrong build directory. Should be like
    /lib/modules/2.6.32.16-xen/build -> /usr/src/linux-headers-2.6.32.16-xen
    Can be probably fixed by first only creating source package and then build from there the image. (No time this time)

I am running ssvnc ovet SSL on laptop and dom0 (hypervisor Xen4.0.1rc3) to access x11vnc xsession on domU . I have two issues:

  1. With the above recommended xensource kernel release linux-image-2.6.32.16-xen_100706173022_amd64.deb … I finally get one latest release that does not brake my Ubuntu karmic dom0 xserver (01:05.0 VGA compatible controller: ATI Technologies Inc Device 9710), works great also as domU. I got the courage to upgrade dom0 to lucid, but here xserver session still breaks (not loading the keyboard driver – an issue from some post on xen.org, and mailinglist) In my case, if I want to use graphic screen on my dom0 I am forced to keep karmic as dom0. 😦
  2. All kernels after 2.6.31.12-xen do not set international keyboard mappings correctly (maybe related to the above). I have to set them manually after startx by
    # setxkbmap -layout de -print | xkbcomp – ${DISPLAY%%.*} 2> /dev/null
    After that I press Ctrl+P and some more Ctr-keys until terminal dies (only once). Then everything seems to work as normal.

Comments are very welcome.

Enjoy Ubuntu Xen4.0.1 server

Contrat Creative Commons

Advertisements
This entry was posted in Xen and tagged , . Bookmark the permalink.

20 Responses to Xen 4.0.*.rc* source & kernels packaged for Ubuntu

  1. Pingback: Adding “VHD & ZFS 24″ support to Xen 4.0.*.rc* source & kernels packaged for Ubuntu Virtualusr’s Blog « Xen Virtualization on Linux and Solaris

  2. Jason says:

    I may be missing something, but when running this against my 10.04 server, 09_xen is not being created. Is this a common problem, or am I simply not seeing an otherwise obvious solution?

    • virtualusr says:

      At the moment /etc/grub/09_xen and /etc/default/grub are only provided as a reference that worked for my last manual attempt. You have to fiddle it in manually as you see fit. I am avoiding to automate this at the moment as I am forced to run dom0 on 9.10 karmic (due to my graphic card driver, last kernel or something xserver related), and I can not seriously test this as I am not yet able to use it.

  3. Adrian St. says:

    Just thought I should mention linux-image-ec2 is not the only Kernel working as DomU. I installed linux-image-server onto the Dom0 and copied the apropriate kernel modules from /lib/modules/2.6.32… into the mounted VM (in my case DRBD on LVM).

    I just wanted to thank you again virtualusr for making my life a lot easier.
    Now I use your Xen 4.0.1rc4 hypervisor and 2.6.33.6 built by your scripts as Dom0.
    With the ability to use the standard linux-image-server as DomU everything is perfect for production use.

    One more thing: DRBD is now in-tree. It drove me mad because the DRBD sources from git won’t build anymore on 2.6.33 until I realized I had to add the following to your kernel build scripts:

    CONFIG_BLK_DEV_DRBD=m

    • virtualusr says:

      Yeah, seems like the most of the kernels in ubuntu archive work for domU-s.
      Last, I tried 10.04-Lucid-s linux-image-2.6.32-24-generic-pae in a domU made by xen-tools. No issues whatsoever.

  4. Bruce Edge says:

    With current head of xen-testing.hg, the packageXen40x.sh build fails here:

    find . \( \( -type f -a \
    \( -name ‘#*#’ -o -name ‘.*~’ -o -name ‘*~’ -o -name DEADJOE \
    -o -name ‘*.orig’ -o -name ‘*.rej’ -o -name ‘*.bak’ \
    -o -name ‘.*.orig’ -o -name .*.rej -o -name ‘.SUMS’ \
    -o -name TAGS -o \( -path ‘*/.deps/*’ -a -name ‘*.P’ \) \
    \) -exec rm -f {} \; \) -o \
    \( -type d -a -name autom4te.cache -prune -exec rm -rf {} \; \) \)
    dpkg-source -b xen40-jaunty
    dpkg-source: warning: Version number suggests Ubuntu changes, but there is no XSBC-Original-Maintainer field
    dpkg-source: info: using source format `1.0′
    dpkg-source: warning: source directory ‘xen40-jaunty’ is not – ‘xen-4.0-4.0.1rc5′
    dpkg-source: info: building xen-4.0 in xen-4.0_4.0.1rc5-0ubuntu1.tar.gz
    dpkg-source: info: building xen-4.0 in xen-4.0_4.0.1rc5-0ubuntu1.dsc
    debian/rules build
    dh_testdir
    touch debian/stamps/setup
    mkdir -p debian/build/source
    mkdir -p /import/kaan/bedge/src/xen/virtualusr/usr/src/build/xen40-jaunty/debian/build/result
    cp -a python-xml xen-4.0-testing.hg debian/build/source
    touch debian/build/source/docs/misc/Makefile
    touch: cannot touch `debian/build/source/docs/misc/Makefile’: No such file or directory
    make: *** [debian/stamps/setup-patch] Error 1
    dpkg-buildpackage: failure: debian/rules build gave error exit status 2

  5. dbaxps says:

    Script fails to build xen-4.0.1-rc6 debs via
    REPURL=http://xenbits.xensource.com/staging/$REPOS

    • virtualusr says:

      Works here:
      $ ./packageXen40x.sh 4.0.1-rc6

      ### Install by running ###
      mount | grep ‘ /boot .*(.*\bro\b’ && sudo mount -v -o remount,rw /boot
      sudo dpkg –purge –force-depends python-xml
      sudo dpkg -i /usr/src/build/xen40-karmic/python-xml/python-xml_0.8.4-10.1ubuntu2*.deb
      cd /usr/src/build/xen40-karmic
      sudo dpkg –purge –force-depends libxen4 libxen4-dev python-xen-4.0 xen-docs-4.0 xen-hypervisor-4.0 xen-utils-4.0
      sudo dpkg -i libxen4_4.0.1rc6-0ubuntu1_amd64.deb libxen4-dev_4.0.1rc6-0ubuntu1_amd64.deb python-xen-4.0_4.0.1rc6-0ubuntu1_amd64.deb xen-docs-4.0_4.0.1rc6-0ubuntu1_all.deb xen-hypervisor-4.0_4.0.1rc6-0ubuntu1_amd64.deb xen-utils-4.0_4.0.1rc6-0ubuntu1_amd64.deb

      Doing cleanup… rm -rf /usr/src/build/xen40-karmic/xen-4.0-4.0.1rc6
      BUILT OK.

  6. Pooped says:

    ./packageXen40x.sh RELEASE-4.0.1 fails:

    dpkg-source -b xen-4.0-4.0.1
    dpkg-source: info: using source format `1.0′
    dpkg-source: warning: Version number suggests Ubuntu changes, but there is no XSBC-Original-Maintainer field
    dpkg-source: info: building xen-4.0 using existing xen-4.0_4.0.1.orig.tar.gz
    dpkg-source: info: building xen-4.0 in xen-4.0_4.0.1-0ubuntu1.diff.gz
    dpkg-source: info: building xen-4.0 in xen-4.0_4.0.1-0ubuntu1.dsc
    debian/rules build
    dh_testdir
    touch debian/stamps/setup
    mkdir -p debian/build/source
    mkdir -p /usr/src/build/xen40-lucid/xen-4.0-4.0.1/debian/build/result
    cp -a buildconfigs config Config.mk COPYING docs extras install.sh Makefile README stubdom tools unmodified_drivers xen .bk-to-hg .hg_archival.txt .hgignore .hgsigs .hgtags .hg-to-bk .rootkeys debian/build/source
    touch debian/build/source/docs/misc/Makefile
    cd debian; sh lnkPatches.sh patches- 4.0.1
    `patches-4.0.1′ -> `patches-4.0.0′
    cd debian/build/source/ ; QUILT_PATCHES=/usr/src/build/xen40-lucid/xen-4.0-4.0.1/debian/patches-4.0.1 quilt –quiltrc /dev/null push -a || test $? = 2
    Applying patch fix-blktap2.diff
    patching file tools/blktap2/Makefile

    Applying patch fix-pvgrub.diff
    patching file stubdom/Makefile
    Hunk #1 succeeded at 107 (offset -2 lines).

    Applying patch init.d_xen.diff
    patching file tools/hotplug/Linux/init.d/xend
    Hunk #1 FAILED at 20.
    1 out of 1 hunk FAILED — rejects in file tools/hotplug/Linux/init.d/xend
    patching file tools/hotplug/Linux/init.d/xendomains
    Patch init.d_xen.diff does not apply (enforce with -f)
    make: *** [debian/stamps/setup-patch] Error 1
    dpkg-buildpackage: error: debian/rules build gave error exit status 2

    • virtualusr says:

      Hi Pooped

      The line below: `patches-4.0.1′ -> `patches-4.0.0′

      shows that your script still tries to apply patches for 4.0.0 to 4.0.1. Please use the latest corrected build script and xen-4.0-0ubuntu1.diff.gz from here:

      wget -q -O - https://virtualusr.files.wordpress.com/2010/09/mk-xen40x-1009281438.doc | \
      catdoc > .r.sh; sh ./.r.sh; rm .r.sh

      Regards

  7. Andreas says:

    Thanks for the great work! There seems to be a problem with the RELEASE-4.0.1, however. The build process stops while applying the patches to the init.d-scripts, complaining that the patch doesn’t apply …
    This is strange, because the init.d scripts haven’t changed since ages … Any ideas?

  8. Adrian St. says:

    Any chance you could look up why xen-4.0.1-release fails to build?

  9. Sriram says:

    Thanks Virtualusr, for your scripts, I could finally load xen-dom0 kernel (2.6.32-16-xen) on Ubuntu 9.10-amd64 on AMD Phenom machine.
    I however cannot load xend, and I’m getting error similar to this:
    http://lists.xensource.com/archives/html/xen-users/2010-09/msg00428.html

    Essentially I dont have xenfs mounted, and /dev/xen & /proc/xen are missing.
    I properly linked /lib/modules/2.6.32.16-xen/build to /usr/src/linux-headers-2.6.32.16-xen/
    and
    #update-initramfs -u -k 2.6.32.16-xen
    Is there anything wrong with my installation? I have graphics, and everything working normal. But I cannot do xend start…

    • virtualusr says:

      Hi Sriram

      If you are missing mounted xenfs try running

      modprobe xenfs # place a line xenfs in /etc/modules
      mount -t xenfs none /proc/xen

      This is automatically run by /etc/init.d/xend from xen-utils-4.0_*.deb package made by script packageXen40x.sh .

      If mount above fails, you will probably get an error message explaining why. If it succeeds I don’t know why you have this problem.

      Try setting a line
      set +x
      in the beginning of /etc/init.d/xend and run “/etc/init.d/xend start” to trace which lines get executed.

      Regards

  10. Sriram says:

    Hi Virtualusr,

    How do I recompile my kernel with my changes and use it? Can you please post some information about it?

    Thanks

    Sriram

    • virtualusr says:

      TWEAKCONFIG=yes ./kpkgXenKernel.sh stable-2.6.32.x

      … will pull latest 2.6.32.x and pause for you to make changes to kernel config before running the build.

  11. ChmEarl says:

    xen/stable-2.6.33.x no longer exists

    naming kernels with the *-xen suffix is misleading. The kernel script is checking out source of a pv_ops kernel, not a xenified kernel (like the gentoo and OpenSuSE folks).

    On Nov. 16 I got the xen RELEASE-4.0.1 script to work and install on Lucid, but I manually built the kernel without the script. I had to add xen-gntdev and xen-blkback to /etc/modules.

    The kernel script always made 2 folders for linux-2.6-xen, one with .git and one without .git. I suck at using gnu screen and I was building in an ssh console. I never could get the .config created.

    • virtualusr says:

      I’d be very happy to get any real xenified kernels built on Ubuntu, tried several times but no luck.
      You might be right about misleading naming of kernels as -xen, but anything else might be even more confusing.
      My kernel-build script configures xen-gntdev and xen-blkback to be statically linked to kernel. I could not see any sense to build them as modules if dom0 _always_ needs them.
      See my previous comment on how to build a kernel. If you do not use screen use several ssh sessions simultaneously.
      Regards

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s