Gutsy Tribe 5 (KVM GUEST) needs -no-kvm to install

This is a problem because intel hw does not provide big real mode, and expects software to handle it. Gfxboot causes the installer to stop at the gfx boot screen. According to #kvm, suse has a patch in gfxboot that detects kvm install and does not attempt to use big real mode, and allows things to get past this issue. The problem with -no-kvm installs is that it is _very_ slow.

Note, this affects xen guest installation as well.

Gutsy currently has gfxboot 3.3.28-ubuntu2. A work around was added for Xen guests running on VT for gfxboot 3.2.35-1. The current issue we're seeing with gfxboot under KVM is very likely a bug in KVM.

Though it's not clear to me that that patch is still there. Hard to tell - the code was refactored a lot since then.

FYI, SLES10 did boot correctly, as did GRML 1.0. None of the *ubuntu ISO did :-\

OpenSUSE 10.3 beta, failed in the same way as Gutsy, with a crash dump

$ sudo /usr/local/kvm-40/bin/qemu-system-x86_64 -localtime -m 1024 -cdrom gutsy-alternate-testing5-i386.iso -boot d
Password:
exception 13 (33)
rax 0000000000000469 rbx 0000000000800001 rcx 0000000000004300 rdx 0000000000000000
rsi 000000000005961d rdi 000000000005961c rsp 00000000fffaa9cc rbp 000000000000200c
r8 0000000000000000 r9 0000000000000000 r10 0000000000000000 r11 0000000000000000
r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
rip 000000000000b04b rflags 00033006
cs 4143 (00041430/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
ds 4004 (00040040/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
es 4004 (00040040/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
ss 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
fs 3002 (00030020/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
gs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
tr 0000 (40850000/00002088 p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
gdt 40920/47
idt 0/ffff
cr0 60000010 cr2 0 cr3 0 cr4 0 cr8 0 efer 0
code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 --> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Aborted

Here's a response I received from Steffen Winterfelt (the author of gfxboot). I think this confirms that it's a KVM issue. I'm going to open an issue on the KVM SourceForge tracker.

> Could you point to either where this check is happening, the proper place to
> ask, or the revision control system for gfxboot so I can find the patchset
> myself?

The short answer: the check you mention no longer exists.

It is enssential to distinguish between pre-SL-10.2 gfxboot and post-SL-10.2.

Up to SL 10.1/SLES10 gfxboot used to pass 4GB selectors to real mode to
access memory. That didn't work in Xen, so I added a check that verified the
selector.

Partly due to this I rewrote the code entirely for 10.2 to run in PM all the
time. So the check was no longer required.

I'm seeing the same issue on Gutsy 64-bit with kvm-intel:

kernel: [ 1093.429150] emulation failed but !mmio_needed? rip a45 f4 eb fd 89

And the bug-trace in the terminal.

To being with I had thought it was the Gutsy ISO I was trying to boot, so I removed that.

However, I'm seeing it whenever /dev/kvm is used - even an empty qcow2 image with no bootable file-system, just a plain 'BIOS' session.

/usr/bin/kvm -monitor stdio -boot c -m 128 -hda '/home/all/VirtualMachines/default-test.qcow2' -net nic,vlan=0 -net user,vlan=0 -localtime &
QEMU 0.9.0 monitor - type 'help' for more information
(qemu) exception 13 (0)
rax 000000000000f001 rbx 000000000000d713 rcx 0000000000000001 rdx 0000000000000000
rsi 00000000ffff004c rdi 000000000008f7f4 rsp 000000000000ffb8 rbp 000000000000ffcc
r8 0000000000000000 r9 0000000000000000 r10 0000000000000000 r11 0000000000000000
r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
rip 0000000000000a45 rflags 00033002

Hi TJ,

I have no problem booting installed guests (Gutsy, XP, Vista = 100%) when running KVM (kvm-44), from what I hear you saying, you can't boot an installed Gutsy guest... if so, that's odd, however I am running 32-Bit host and guests.

Hopefully this gfxboot issue will be resolved, as personally I think it is very important, especially when it comes to Ubuntu Server installations! (anyone agree with me on this?). We use Ubuntu server widely (6.06LTS mainly), and I am keen to push KVM usage in our company, the idea of waiting hours does not appeal to me to install, but worst case schenario will be extracting a qcow2 image of a working base installation guest.

For my testing here at home, I made some modifications to my guest structure.

I have got in the host a 500gb SATA-II (/dev/sdb). I only have one shell script to boot the guest:

/usr/local/bin/kvm contains

#!/bin/bash
/usr/local/kvm-44/bin/qemu-system-x86_64 -soundhw all -localtime -m 1024 -hda /dev/sdb

I also have in my /etc/rc.local

echo 1024 > /proc/sys/dev/rtc/max-user-freq &>/dev/null
chmod o+rw /dev/sdb &>/dev/null

I find having a dedicated drive much easier to manage my guests, and improves preformance. All the partitions can be mounted in my host, etc, etc.

Sorry the above is off topic, but maybe a useful bit of info for some.

Regards

My issue might be another symptom, but the guest is an empty image - no installed OS - that I'd expect to run BIOS and then report "No System Disk" and stop, waiting. I've had that before when forgetting to aim the CD at an installer ISO image.

It worked okay about a week ago when I installed Windows XP so I'm guessing one of the recent kernel updates is responsible.

TJ, I'd recommend building the kvm-44, for me it's preforming very well, and stable, but as I said thats on pre-installed guests, I have not attempted a guest installation with it.

FYI: Beta release still has same issue....

I seriously hope that server release either does not have gfxboot, or fixes this issue....

Attached is a patch that fixes the problem with gfxboot. Below is a link to the KVM bugzilla entry that has a more in-depth explaination of the problem[1]. There's another bug in KVM that makes the install screen black[2] but that should get fixed real soon. I have a really bad hack in the bug entry that lets the graphics show up. Using a remastered Gutsy install CD with my gfxboot patch and a patched KVM with my VGA update hack, I can boot a Gutsy install CD on my VT laptop. I'm submitting this patch to upstream gfxboot too. Hopefully this patch can be applied to the Gutsy gfxboot package before Gutsy is released.

[1] https://sourceforge.net/tracker/?func=detail&atid=893831&aid=1798813&group_id=180599
[2] https://sourceforge.net/tracker/index.php?func=detail&aid=1804997&group_id=180599&atid=893831

Hi Anthony,

Thanks for investigating this, and producing a patch, I'm so happy to hear you have it working. You say you have a remastered Gutsy that boots. The required changes (binary ones), is it possible you can share, and a brief description on the remaster process please? I'd like to test this out.

Kind regards

Unfortunately there was a small bug in my previous patch. I should have used a 16 bit mov instead of a 32 bit mov or else data would potentially leak from ebx into esp. Turns out that in practice, the high bits of ebx were zero so everything still worked.

To test out the patch, I took a gutsy-desktop CD (tribe 4 I believe), mounted it via loopback (mount -oloop,ro gutsy-desktop-i386.iso /mnt). I then copied (via cp -ra /mnt iso) to another working directory.

Applying this patch to the gfxboot source package (apt-get source gfxboot), then doing a make install. Then rebuild the gfxboot-theme-ubuntu package from source (apt-get source gfxboot-theme-ubuntu), and copy the bootlogo into the staged iso (cp gfxboot-theme-ubuntu-0.4.3/install/bootlogo iso/isolinux/bootlogo).

Finally, I built a new ISO with mkisofs (mkisofs -R -b isolinux/isolinux.bin -c isolinux/boot.cat -J -no-emul-boot -boot-load-size 4 -boot-info-table -o gutsy-remastered.iso iso).

I should add that for now, it's necessary to use -std-vga on the qemu command line until we fix the graphics updating issue in KVM.

I can confirm this works. Here is the binary bootlogo file I used if anyone wants to skip the build process and jump straight to the copy

Great work Anthony!!

Ben, can you get this in before final?

Ok, some interesting info, and a fix / work around.

To install a guest (Tested this with Ubuntu Gutsy Beta), remaster the ISO with the bootlogo file removed. At the boot line, use vga mode, eg:

install vga=791

Your guest installation will install fast, and perfectly.

Anthony has written a small C application to remaster an ISO with the bootlogo file removed, I expect he will release the code shortly.

The second option , that I have not fully tested, is to tell KVM/QEMU to boot the kernel and initrd directly off the CD/ISO, this therefore by-passes the gfxboot completly and goes straight to installation. I will attempt to finish testing this out, and with other guest distros if possible, this evening.

The gfxboot maintainer has posted a patch that does the same thing in a different way. I've tested the patch and it works as expected.

From Steffen Winterfeldt:

Anyway, ss is already saved, so no need for an extra register. Here is
my version (tested and works on my machine):

--- bincode.asm (revision 650)
+++ bincode.asm (working copy)
@@ -15546,7 +15546,11 @@
   mov ax,pm_seg.prog_d16
   mov ds,ax

- mov eax,ss
+ ; needed for KVM:
+ ; ss:rpl must equal cs:rpl in PM for VT. We can't rely on ss
+ ; maintaining its value after the transition.
+
+ movzx eax,word [rm_seg.ss]
   and esp,0ffffh
   shl eax,4
   add esp,eax

Now that gutsy's been released, can this fix get pulled in so we can at least make sure that gutsy+1 doesn't have this problem?

Ok, don't flame me for repeating myself here, but flagging this as Medium is wrong in my view. Wrong for the following reasons:

1) It's easy and quick to patch, and (to me and others) SHOULD have been done prior to the release of 7.10

2) I work as a systems administrator, and use as much as possible Ubuntu Server (generally LTS, but on occasions latest release), and we are about to any day now put into productions a suite of virtual servers using KVM. The installation of guest RHEL5/CentOS5 works in KVM (53), but Ubuntu Server... no, I have to remaster the ISO to remove the gfxboot, ok no major issue, but it should not be like that, it SHOULD boot.

3) As I assume most KVM end-users will be geeks and admins, it's annoying to people to find that booting your guest Ubuntu is a no-go

I'm hoping that this get cleared up ASAP, it's been reported, it's been patched, it's been acknowledged , ... erm, PLEASE lets get it sorted.

I'm also toying with the idea of writing a GUI (Mono GTK#) and / or console app to remaster any given Ubuntu CD/DVD iso to a KVM friendly bootable ISO...

Sorry for ranting :P

Soren believed he'd fixed this a while back, well before Hardy Alpha 1:

gfxboot (3.3.28-0ubuntu3) hardy; urgency=low

  * 07_kvm_real_mode.dpatch:
    - Apply patch from Steffen Winterfeldt to work around bug in kvm that
      makes Ubuntu uninstallable in kvm on Intel hardware.

 -- Soren Hansen <email address hidden> Mon, 19 Nov 2007 16:06:04 +0100

El Diablo, could you please confirm that you actually reproduced this bug on Hardy Alpha 1, and aren't just assuming its presence from the fact that the bug is still open?

Colin,

Hardy Alpha 1 does not work with KVM. It appears that the CD was not mastered with the proper version of gfxboot.

To confirm this, I first downloaded the source package for gfxboot-3.3.28-0ubuntu3 and applied the patches in debian/patches by hand. I remastered the CD with this version of gfxboot and the CD worked fine in KVM. I then tried the amd64 binary package and built a new bootlogo using these binaries. I remastered the CD and it worked fine in KVM.

Is the CD mastering a manual process or an automatic one? If it's automatic, what I should file a new bug against?

Oh, that. It needed a new gfxboot-theme-ubuntu built against the new gfxboot, that's all. Soren did this on 30 Nov, but it was slightly too late to make it into the alpha-1 images. It'll be in alpha-2.

I downloaded an alternate daily build and it works just fine under KVM.

Thanks!

hallelujah praise the lord

jeje

Sorry, ok great news! The big thing to me is that because 8.04 will I believe be LTS, it would be terrible not to have this patched. Ok, I'm a happy little devil now.

Anthony, many thanks for your research and work yeah.... see you on irc