apt/dpkg in qemu-system-arm hangs if a big task is installed

Bug #532733 reported by Oliver Grawert
84
This bug affects 11 people
Affects Status Importance Assigned to Milestone
QEMU
Invalid
Medium
Unassigned
qemu-linaro (Ubuntu)
Expired
High
Unassigned
Nominated for Maverick by Ricardo Salveti
Lucid
Expired
High
Unassigned

Bug Description

Binary package hint: qemu-kvm

running rootstock and installing ubuntu-netbook^ makes the VM hang in "unpacking iso-codes" this is reproducable every time in rootstock as well as in a standard qemu-system-arm vm that contains a minimal ubuntu with running apt-get install ubuntu-netbook

Revision history for this message
Oliver Grawert (ogra) wrote :

apparently this hang only occurs with raw qemu images, converting the disk image to qcow2 does not produce the hang.

Revision history for this message
Oliver Grawert (ogra) wrote :

seems i spoke to soon, even with qcow2 this hang occurs, i was also experimenting with different blocksizes on the filesystem and as well with different filesystems (ext2/3/4) none of it changed a thing in behavior, the hang also always occurs at the same place.

Revision history for this message
Chuck Short (zulcss) wrote :

Oliver,

Which version are we talking about here?

chuck

Changed in qemu-kvm (Ubuntu):
importance: Undecided → Medium
status: New → Incomplete
Revision history for this message
Oliver Grawert (ogra) wrote :

sadly all versions of qemu system-arm expose this in lucid ...

over the weekend i tried all qemu.-system-arm binaries we have in the archive going backwards in history up to the first karmic version ... i also tried the versatile kernel we used to use in karmic (which worked fine during karmic for i.e. installing the ubuntu-desktop task)

i'm solwly starting to suspect that something in the lucid userspace causes these disk I/O hangs, i will:

a) do some tests building a karmic under lucid userspace now
b) start a script in a qemu arm VM under lucid that will do endless loop reading/writing to see if it hangs at some point

Revision history for this message
Oliver Grawert (ogra) wrote :

note also that the VM itself doesnt hang, its just the disk IO ... if i stop the hanging process teh VM is usable as before.
if you want to reproduce install rootstock and run:

sudo rootstock -f test -s ubuntu-netbook^
(note that wont give you any access to the VM, try the thing below for this)

or to have more debugging abilities use rootstock to create an armel qemu minimal install (with username test and password test):

sudo rootstock -f test -s ubuntu-minimal -l test -p test --keepimage

then run qemu-system-arm as described at https://wiki.ubuntu.com/ARM/RootfsFromScratch and install the ubuntu-netbook task with:

sudo apt-get install ubuntu-netbook^

Revision history for this message
Oliver Grawert (ogra) wrote :

i just finished a complete build of a karmic rootfs under lucid, that finished fine so it is definately a problem of interaction of the lucid userspace with the lucid VM or versatile kernel (the problem does not occur on real hardware)

Revision history for this message
Oliver Grawert (ogra) wrote :
Revision history for this message
Oliver Grawert (ogra) wrote :

running apt under strace shows it hangs silently in read()
running top in a different console during teh hang first shows apt running moderately at 45% CPU usage, dpkg at about 10% ...
i see some dpkg-deb processes with <defunct> tag pass by, then dpkg vanishes completely from top output (there are still processes in ps) and apt goes up to 90-95%

Revision history for this message
Alexander Sack (asac) wrote :

this needs to be RC ... it blocks rootstock spec.

Changed in qemu-kvm (Ubuntu Lucid):
milestone: none → ubuntu-10.04-beta-2
status: Incomplete → Confirmed
Alexander Sack (asac)
Changed in qemu-kvm (Ubuntu Lucid):
importance: Medium → High
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Oliver-

Just to confirm....

Is 256M memory, and 3GB disk, and no swap, enough resources to actually do "sudo apt-get install ubuntu-netbook" ? I just want to make sure we're not hitting a memory/disk limit inside of the guest before I chase down the qemu IO emulation code path....

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

I think I've reproduced it here. Stuck unpacking firefox-gnome-support. Screenshot shows uptime of 46 minutes (how far into the install of ubuntu-netbook this occurs), load at 1.74 which is high but not crash-the-machine high, memory usage is at 72%, and disk is at 52%, so there's still free memory and disk.

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

That last run was executed with scsi emulated disk, like this:
 $ qemu-system-arm -M versatilepb -cpu cortex-a8 -kernel ./vmlinuz -hda ubuntu-arm.img -m 256 -append "root=/dev/sda mem=256M devtmpfs.mount=0 rw single" -net nic -net user

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Hmm, well this screen shot is a little puzzling...

From within the ARM VM, I was able to dd all of the 3GB /dev/sda to /dev/null in about 2.5 minutes (20.1MB/s).

That's a lot of read I/O. I need to find some way of reproducing this problem outside of apt-get before sending it upstream...

Revision history for this message
Dustin Kirkland  (kirkland) wrote :
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

One more... this one used dd to read 1G of data from disk, and write that to file, successfully. Took 247s, at 4.3MB/s (doing both reads and writes).

I'm having trouble finding a way to reproduce this outside of apt.... Ideas?

Changed in qemu-kvm (Ubuntu Lucid):
status: Confirmed → Incomplete
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

One more question, Oliver ... Would be it possible to enable the
virtio networking and disk drivers in the Lucid arm kernel in the
guest?

I have a very strong feeling that upstream QEMU is going to first
recommend that we test the virtio disk driver model (instead of the
scsi emulation one).

As a benefit, you should get a *noticeable* performance benefit from
virtio disk (and networking, too!).

Revision history for this message
Loïc Minier (lool) wrote :

Dustin: virtio isn't available on ARM. I researched virtio on ARM a couple of weeks ago, I did some Kconfig includes hackery and managed to build the modules, but I get an oops when the first virtio device is registered on the bus. Happy if you can help with getting this working, this would indeed help performance a lot!

Revision history for this message
Dustin Kirkland  (kirkland) wrote : Re: [Bug 532733] Re: apt/dpkg in qemu-system-arm hangs if a big task is installed

Bummer, okay, thanks Loic.

Revision history for this message
Oliver Grawert (ogra) wrote :

just to answer the above question, 3G disk and 256M RAM should be enough for netbook

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Oliver-

I was able to apt-get install ubuntu-netbook earlier today.

It took a very, very long time (since I don't have a ports mirror, and
since this is pure emulation), but it ran to completion.

Several times during the dist-upgrade, though, the SDL window seemed
to go unresponsive for a while. I tapped enter a few times, and then
the screen refreshed.

I'm not sure where that puts us...

We really need to find a way to reproduce this outside of apt/dpkg.

Steve Langasek (vorlon)
Changed in qemu-kvm (Ubuntu Lucid):
milestone: ubuntu-10.04-beta-2 → none
Thierry Carrez (ttx)
Changed in qemu-kvm (Ubuntu Lucid):
assignee: nobody → Dustin Kirkland (kirkland)
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

I see that Thierry has re-assigned this to me.

Note that this bug is marked Incomplete.

I need a way to reproduce this besides of apt-get in the guest, such that I can report this upstream.

Note that I was able to dd from the entire guest disk to /dev/null without a problem, which should have maximized the read throughput on the guest.

Revision history for this message
Oliver Grawert (ogra) wrote :

dustin, the actual issue shows up under rootstock where we dont use SDL at all, using plain qemu is just to easier reproduce the error, rootstock reads/writes through a fifo via emulated serial console so there is neither SDL nor any ui stuff involved.

it shows up for everyone and the hang apprars to be at the same package for nearly everyone no matter if you use rootstock with its pipe or the plain SDL based VM, the curious thing as i mentioned above is that it vanishes completely as soon as you install apt and dpkg -dbgsym packages.

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Opening an upstream task.

Anthony, any ideas?

Changed in qemu:
importance: Undecided → Medium
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Hi Ogra-

Can you try the aio=native,cache=off backing disk image options to qemu?

  -drive file=foo.img,aio=native,cache=off

where foo.img is your backing disk image?

This should ensure that disk writes are synchronous, and *might* help the issue. Can you try that and report back?

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Alternatively, aio=threads is your other option. Perhaps try that too?

Also, perhaps play with the cache, which can be cache=none, cache=writethrough, or cache=writeback.

(Sorry, in my last post, I wrote cache=off).

See:
 * http://manpages.ubuntu.com/manpages/lucid/en/man1/qemu.1.html

Revision history for this message
Oliver Grawert (ogra) wrote :

-drive file=${IMAGENAME},aio=native,cache=none definately doesnt change behavior, will try other combinations too.

Revision history for this message
Oliver Grawert (ogra) wrote :

-drive file=${IMAGENAME},aio=threads,cache=writethrough doesnt change anything either

Revision history for this message
Loïc Minier (lool) wrote :

Writethrough is the default already; writeback might make a difference.

Revision history for this message
Oliver Grawert (ogra) wrote :

same for -drive file=${IMAGENAME},aio=native,cache=writethrough, testing writeback now

Revision history for this message
Oliver Grawert (ogra) wrote :

-drive file=${IMAGENAME},aio=native,cache=writeback appears to work faster overall but still hangs sadly ... trying the same with aio=threads now but my hopes arent high

Revision history for this message
Oliver Grawert (ogra) wrote :

as expected, same hang using -drive file=${IMAGENAME},aio=threads,cache=writeback

Revision history for this message
Robert Nelson (robertcnelson) wrote :

For comparison sake, I gave qemu access to a 2nd hardrive in my system "-hda /dev/sda1" same result as with using disk images..

Selecting previously deselected package libgnomekbd-common.
Unpacking libgnomekbd-common (from .../libgnomekbd-common_2.30.0-0ubuntu2_all.deb) ...
Selecting previously deselected package iso-codes.
Unpacking iso-codes (from .../iso-codes_3.12.1-1_all.deb) ...

Patch attached for kicks

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Thanks for testing ogra... Bummer.

Alexander Sack (asac)
Changed in qemu-kvm (Ubuntu Lucid):
milestone: none → lucid-updates
Revision history for this message
Carey Underwood (cwillu) wrote :

With the same procedure, I now get a segfault rather than a hang; this would seem to support the userspace theory. Can anybody else confirm the behaviour change?

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Segfault in the host or guest?

Revision history for this message
Oliver Grawert (ogra) wrote :

hmm, i dont see segfaults here, still just the plain hang

Revision history for this message
Carey Underwood (cwillu) wrote :

In the host:

7720 Segmentation fault qemu-system-arm ${QEMUOPTS} ${ROOTDEV} ${SWAPDEV} -append "${APPEND}" > $QEMUFIFO 2>&1

[ 1844.178068] qemu-system-arm[7720]: segfault at ffffffffd07dca2c ip ffffffffd07dca2c sp 00007fffabecc210 error 14

Revision history for this message
Zygmunt Krynicki (zyga) wrote :

Hi

I can reproduce this each time by running netboot installer using the versatile kernel:

#!/bin/sh
qemu-img create -f qcow2 sda.qcow2 16G
gdb --args qemu-system-arm -M versatilepb -m 256 -cpu cortex-a8 -kernel vmlinuz -initrd initrd.gz -hda sda.qcow2 -append "mem=256M"

Here is the backtrace:

(gdb) bt
#0 0xffffffffcdc546ec in ?? ()
#1 0x00000000000000eb in ?? ()
#2 0x0000000000567064 in tlb_set_page (env=0x4, address=13491680, access_type=3584, mmu_idx=0, is_softmmu=13476464)
    at /build/buildd/qemu-kvm-0.12.3+noroms/exec-all.h:98
#3 cpu_arm_handle_mmu_fault (env=0x4, address=13491680, access_type=3584, mmu_idx=0, is_softmmu=13476464)
    at /build/buildd/qemu-kvm-0.12.3+noroms/target-arm/helper.c:1178
#4 0x0000000000562151 in tlb_fill (addr=3930382336, is_write=<value optimized out>, mmu_idx=<value optimized out>, retaddr=0x0)
    at /build/buildd/qemu-kvm-0.12.3+noroms/target-arm/op_helper.c:98
#5 0x0000000000514aa9 in __ldb_cmmu (addr=240640, mmu_idx=1) at /build/buildd/qemu-kvm-0.12.3+noroms/softmmu_template.h:131
#6 0x0000000000515a78 in cpu_arm_exec (env1=<value optimized out>) at /build/buildd/qemu-kvm-0.12.3+noroms/cpu-exec.c:630
#7 0x000000000040dfd3 in qemu_cpu_exec (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>)
    at /build/buildd/qemu-kvm-0.12.3+noroms/vl.c:4073
#8 tcg_cpu_exec (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>) at /build/buildd/qemu-kvm-0.12.3+noroms/vl.c:4102
#9 main_loop (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>) at /build/buildd/qemu-kvm-0.12.3+noroms/vl.c:4226
#10 main (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>) at /build/buildd/qemu-kvm-0.12.3+noroms/vl.c:6238

Revision history for this message
Oliver Grawert (ogra) wrote :

smells like an alignment issue, and in fact i'm just doing a rootstock rootfs build with an added "cat 3 /proc/cpu/alignement" that got past the well, known iso-codes hang (lets see if its a red herring or if it finishes though)

Revision history for this message
Oliver Grawert (ogra) wrote :

err, cat should indeed be echo above :)

Revision history for this message
Zygmunt Krynicki (zyga) wrote :

Ogra proposed change (echo 3 > /proc/cpu/alignement in the guest vm) did not work for me. Ogra is running on i386, I'm on amd64.

Revision history for this message
Zygmunt Krynicki (zyga) wrote :

I built qemu-kvm with DEB_BUILD_OPTIONS=noopt that replaces -O2 with -O0 and (after _long_ session) ran into this crasher:

Program received signal SIGSEGV, Segmentation fault.
0xffffffffcde9719c in ?? ()
(gdb) bt
#0 0xffffffffcde9719c in ?? ()
#1 0x00007fffffffe090 in ?? ()
#2 0x00000000005986a4 in tb_find_slow (pc=Cannot access memory at address 0xffffffffffffffbe
) at /home/zyga/Canonical/ubuntu-qa/qemu-try-2/qemu-kvm-0.12.3+noroms/cpu-exec.c:172
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Revision history for this message
Oliver Grawert (ogra) wrote :

it definately worked for three ubuntu-netbook builds now, i'm just running an ubuntu-desktop one, if that works as well, i'll prepare an SRU package for rootstock to verify it works on other i386 systems too.

Revision history for this message
Zygmunt Krynicki (zyga) wrote :
Download full text (3.9 KiB)

Soren pointed out that is more useful than plain backtrace:

From the qemu-kvm -O0:

(gdb) thread apply all bt full

Thread 1 (Thread 0x7ffff7fc7760 (LWP 31739)):
#0 0xffffffffcde9719c in ?? ()
No symbol table info available.
#1 0x00007fffffffe090 in ?? ()
No symbol table info available.
#2 0x00000000005986a4 in tb_find_slow (pc=Cannot access memory at address 0xffffffffffffffbe
) at /home/zyga/Canonical/ubuntu-qa/qemu-try-2/qemu-kvm-0.12.3+noroms/cpu-exec.c:172
        tb = Cannot access memory at address 0xffffffffffffffd2

From the qemu-kvm -O2 (stock lucid):
(gdb) thread apply all bt full

Thread 1 (Thread 0x7ffff7fc7760 (LWP 30205)):
#0 0xffffffffcdc546ec in ?? ()
No symbol table info available.
#1 0x00000000000000eb in ?? ()
No symbol table info available.
#2 0x0000000000567064 in tlb_set_page (env=0x4, address=13491680, access_type=3584, mmu_idx=0, is_softmmu=13476464)
    at /build/buildd/qemu-kvm-0.12.3+noroms/exec-all.h:98
No locals.
#3 cpu_arm_handle_mmu_fault (env=0x4, address=13491680, access_type=3584, mmu_idx=0, is_softmmu=13476464)
    at /build/buildd/qemu-kvm-0.12.3+noroms/target-arm/helper.c:1178
        phys_addr = 102251520
        prot = 1
        ret = <value optimized out>
        is_user = <value optimized out>
#4 0x0000000000562151 in tlb_fill (addr=3930382336, is_write=<value optimized out>, mmu_idx=<value optimized out>, retaddr=0x0)
    at /build/buildd/qemu-kvm-0.12.3+noroms/target-arm/op_helper.c:98
        tb = <value optimized out>
        saved_env = 0x1
        ret = <value optimized out>
#5 0x0000000000514aa9 in __ldb_cmmu (addr=240640, mmu_idx=1) at /build/buildd/qemu-kvm-0.12.3+noroms/softmmu_template.h:131
        res = <value optimized out>
        index = 55297
        tlb_addr = <value optimized out>
        addend = <value optimized out>
#6 0x0000000000515a78 in cpu_arm_exec (env1=<value optimized out>) at /build/buildd/qemu-kvm-0.12.3+noroms/cpu-exec.c:630
        reg_AREG0 = <value optimized out>
        saved_AREG0 = 0xcda270
        reg_AREG1 = <value optimized out>
        saved_AREG1 = 0x0
        reg_AREG2 = <value optimized out>
        saved_AREG2 = 0x431bde82d7b634db
        ret = <value optimized out>
        interrupt_request = <value optimized out>
        tb = <value optimized out>
        next_tb = 3584
#7 0x000000000040dfd3 in qemu_cpu_exec (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>)
    at /build/buildd/qemu-kvm-0.12.3+noroms/vl.c:4073
No locals.
#8 tcg_cpu_exec (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>) at /build/buildd/qemu-kvm-0.12.3+noroms/vl.c:4102
---Type <return> to continue, or q <return> to quit---
        ret = -364584960
#9 main_loop (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>) at /build/buildd/qemu-kvm-0.12.3+noroms/vl.c:4226
        r = <value optimized out>
#10 main (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>) at /build/buildd/qemu-kvm-0.12.3+noroms/vl.c:6238
        gdbstub_dev = 0x0
        boot_devices_bitmap = 0
        i = <value optimized out>
        snapshot = 0
        initrd_filename = ...

Read more...

Revision history for this message
Oliver Grawert (ogra) wrote :

hanging again while trying ubuntu-desktop in rootstock, here a backtrace from the hanging process

Revision history for this message
Arvind Singh (aaroh) wrote :

I am seeing similar issue on Lucid.
Rootstock version: rootstock-0.1.99.3
Here is my command line:
sudo ./rootstock --fqdn beagleboard --login ubuntu --password beagle --imagesize 4G --swapsize 512 --components "main,universe,multiverse" --seed openssh-server,build-essential,apache2,postgresql-server-dev-8.4,postgresql-contrib,sqlite3,libsqlite3-dev --dist lucid --serial ttyS2 --kernel-image http://rcn-ee.net/deb/kernel/beagle/lucid/v2.6.32.11-l13/linux-image-2.6.32.11-l13_1.0lucid_armel.deb

Fails with:

/bin/installer: line 55: 157 Segmentation fault apt-get -y install openssh-server build-essential apache2 postgresql-server-dev-8.4 postgresql-contrib sqlite3 libsqlite3-dev
E: Second stage build in Virtual Machine failed !
E: Please see the log to see what went wrong.
I: Cleaning up...
../rootstock: line 54: 5051 Killed qemu-system-arm $QEMUOPTS -append "${APPEND}" > $QEMUFIFO 2>&1
....
I: A logfile was saved as /home/asingh/met/v_4_1_0/ubuntu-sd/rootstock-201005260742.log
I: done ...
mkimage: Can't open ./vmlinuz-*: No such file or directory

This succeeds if I just give 2 packages as seed.

This was and is all working in Jaunty without any issues.

Revision history for this message
Oliver Grawert (ogra) wrote :

@arvind this is not related to this bug, please open a new one

Revision history for this message
Arvind Singh (aaroh) wrote :

Sure Thanks.

Revision history for this message
Anthony Liguori (anthony-codemonkey) wrote :

beagleboard is not in upstream QEMU. Please do not mark bugs as affects upstream QEMU unless you've actually reproduced the problem with upstream QEMU.

Changed in qemu:
status: New → Invalid
Revision history for this message
Oliver Grawert (ogra) wrote :

@Anthony, this bug has nothing to do with beagleboards it happens if qemu is run on x86 systems

Revision history for this message
Oliver Grawert (ogra) wrote :

@Anthony please see all the above comments before judging and please reopen it upstream again

Revision history for this message
Oliver Grawert (ogra) wrote :

and just for reference so there isnt coming up any confusion again, the used qemu call that breaks is for a versatilbepb machine using the versatile kernel from http://ports.ubuntu.com/ubuntu-ports/dists/lucid/main/installer-armel/current/images/versatile/netboot/vmlinuz

Revision history for this message
Anthony Liguori (anthony-codemonkey) wrote :

If someone reproduces the bug against upstream qemu, feel free to refile the bug with the appropriate information.

Revision history for this message
Ricardo Salveti (rsalveti) wrote :

I'm now implementing the support for creating a rootstock rootfs without requiring root, and I also got stuck at a segmentation fault, just after executing the debootstrap' second stage.

I'm running the qemu-system-arm from qemu-kvm-extras 0.12.4+noroms-0ubuntu4, at maverick.

My qemu command line:
qemu-system-arm -M versatilepb -cpu cortex-a8 -kernel qemu-vmlinuz -no-reboot -nographic -drive file=qemu-armel-rootstock.img,aio=native,cache=none -m 256 -append 'console=ttyAMA0,115200n8 root=/dev/sda rw mem=256M init=/bin/installer'
Uncompressing Linux................................................................................................................................................................................................. done, booting the kernel.
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
...
...
I: Configuring initramfs-tools...
I: Base system installed successfully.
I: Starting basic services in VM
[ 912.758337] udev: starting version 151
Adding `local diversion of /usr/sbin/invoke-rc.d to /usr/sbin/invoke-rc.d.rootstock'
[ 913.623614] eth0: link up
Internet Systems Consortium DHCP Client V3.1.3
Copyright 2004-2009 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/eth0/52:54:00:12:34:56
Sending on LPF/eth0/52:54:00:12:34:56
Sending on Socket/fallback
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 7
DHCPOFFER of 10.0.2.15 from 10.0.2.2
DHCPREQUEST of 10.0.2.15 on eth0 to 255.255.255.255 port 67
DHCPACK of 10.0.2.15 from 10.0.2.2
bound to 10.0.2.15 -- renewal in 34170 seconds.
Generating locales...
  en_GB.UTF-8... done
Segmentation fault (core dumped)

[436282.681876] qemu-system-arm[4435]: segfault at ffffffffcd7797cc ip ffffffffcd7797cc sp 00007fffa42e3070 error 14

For testing and debugging purposes, I'm also uploading the rootstock image to my server (if you want to test, wait at least 2 hours and make sure to check the md5).

Image:
 * rootfs: http://rsalveti.net/pub/ubuntu/rootstock/qemu-armel-rootstock.img (md5 356b497edddec08ff19f2ef545b4e207)
 * kernel: http://ports.ubuntu.com/ubuntu-ports/dists/lucid/main/installer-armel/current/images/versatile/netboot/vmlinuz

Going to get some sleep now but tomorrow will try qemu with gdb and also with the current upstream version.

Revision history for this message
Ricardo Salveti (rsalveti) wrote :

For the seg fault bug I've created the bug 604872.

Meanwhile I'll try to reproduce the hang problem with Maverick and upstream Qemu.

Revision history for this message
Ricardo Salveti (rsalveti) wrote :

After some tests I was able to reproduce the issue on Maverick and with upstream Qemu (08218b3527301760393b0b4ec732fcdfb7ff6cda).

Also tried stressing the disk with fsstress and dd for many hours, but couldn't reproduce the issue. The easier way to reproduce it is with dpkg, but hard to find the exact cause.

My cmd line:
./rootstock --fqdn beagleboard --login ubuntu --password temppwd --seed ubuntu-netbook -i 3G --dist lucid --serial ttyS2 --components "main universe multiverse" --keepimage

The only thing I changed was the path of qemu-system-arm inside rootstock, to get the upstream one.

Revision history for this message
Ricardo Salveti (rsalveti) wrote :

Here is where it's getting stuck now:
....
Selecting previously deselected package libimobiledevice0.
Unpacking libimobiledevice0 (from .../libimobiledevice0_0.9.7-1ubuntu1_armel.deb) ...
Selecting previously deselected package libtalloc2.
Unpacking libtalloc2 (from .../libtalloc2_2.0.1-1_armel.deb) ...
Selecting previously deselected package libwbclient0.
Unpacking libwbclient0 (from .../libwbclient0_2%3a3.4.7~dfsg-1ubuntu3_armel.deb) ...

Thierry Carrez (ttx)
Changed in qemu-kvm (Ubuntu Lucid):
milestone: lucid-updates → none
Changed in qemu-kvm (Ubuntu):
milestone: lucid-updates → none
Revision history for this message
Dr. The Fugitive (drbrando007) wrote :

This should help a little:
I can confirm that this is a problem in rootstock, not just inside qemu or qemu/kvm.
Reproducible on host system.
System output from uname :

user@lucid:~/kvm$ uname -a
Linux lucid 2.6.32-25-generic #44-Ubuntu SMP Fri Sep 17 20:05:27 UTC 2010 x86_64 GNU/Linux

Rootstock command used :

user@lucid:~/kvm$ sudo rootstock --fqdn nexus --login user --password psswd --imagesize 8G --seed ubuntu-netbook,build-essential,openssh-server,nano

This has failed three times in a row, hanging upon :

Unpacking iso-codes (from .../iso-codes_3.12.1-1_all.deb) ...

This essential just hangs the bash shell, terminal still responsive to ctrl-C, utilizes 100% of 1 (one) of my dual cores.
Is there any open issues on rootstock related to this?

Thanks-
DrTheFugitive

Revision history for this message
Ricardo Salveti (rsalveti) wrote :

What fails is qemu-arm-static, as rootstock uses it internally to install the packages.

Revision history for this message
Dr. The Fugitive (drbrando007) wrote :

@Ricardo Salveti

Ok, I am going to get a release candidate for 10.10 Meerkat, install it and try to rootstock ubuntu-netbook for lucid from there. I will inform this log of my progress.

Revision history for this message
Dr. The Fugitive (drbrando007) wrote :

Allright, it appears that iso-codes will build successefully on this kernel:

user@meerkatuname:~/kvm$ uname -a
Linux merelynatural 2.6.35-22-generic #33-Ubuntu SMP Sun Sep 19 20:32:27 UTC 2010 x86_64 GNU/Linux

Output as follows:

Setting up x11-xkb-utils (7.5+1) ...
Setting up libxklavier16 (5.0-0ubuntu1) ...

Setting up libgnomekbd-common (2.30.0-0ubuntu2) ...

Setting up iso-codes (3.12.1-1) ...
Setting up libgnomekbd4 (2.30.0-0ubuntu2) ...

Build still fails however, it seems it will still run into the much aggreived Mono issue, bash hangs here:

Setting up python-desktopcouch (0.6.4-0ubuntu3) ...

Setting up mono-2.0-gac (2.4.4~svn151842-1ubuntu4) ...
Setting up mono-gac (2.4.4~svn151842-1ubuntu4) ...
* Installing 6 assemblies from libart2.0-cil into Mono

Am back to the drawing board, am trying to avoid ubuntu-minimal and aptitude installation of ubuntu-netbook due to sdhc constantly crapping out after a real small number of write cycles (way too early, am arguing right now with manufacturers for warranty replacements). SDHC is going to be the real benefactor of rootstock.

TheFugitive

Changed in qemu-kvm (Ubuntu):
assignee: Dustin Kirkland (kirkland) → nobody
Changed in qemu-kvm (Ubuntu Lucid):
assignee: Dustin Kirkland (kirkland) → nobody
Revision history for this message
Dr. The Fugitive (drbrando007) wrote :
Download full text (6.2 KiB)

running this on a real arm board, snapdragon htc phone
this is a chroot apt-get install ubuntu-netbook. here is the results of my 'kill -SIGINT' for hanging mono processess, this does not appear to be an issue related to kvm, since this has been done outside the virtual machine:

Setting up mono-2.0-gac (2.4.4~svn151842-1ubuntu4) ...
Setting up mono-gac (2.4.4~svn151842-1ubuntu4) ...
* Installing 6 assemblies from libart2.0-cil into Mono
E: Installation of libart2.0-cil with /usr/share/cli-common/runtimes.d/mono failed
* Installing 1 assembly from libflickrnet2.2-cil into Mono
E: Installation of libflickrnet2.2-cil with /usr/share/cli-common/runtimes.d/mono failed
* Installing 6 assemblies from libgconf2.0-cil into Mono
Use of uninitialized value $_ in scalar chomp at /usr/share/cli-common/runtimes.d/mono line 144.
Use of uninitialized value $fullname in concatenation (.) or string at /usr/share/cli-common/runtimes.d/mono line 113.
E: installing Assembly /usr/lib/cli/gconf-sharp-2.0/gconf-sharp.dll failed
E: Installation of libgconf2.0-cil with /usr/share/cli-common/runtimes.d/mono failed
* Installing 1 assembly from libglade2.0-cil into Mono
E: Installation of libglade2.0-cil with /usr/share/cli-common/runtimes.d/mono failed
* Installing 1 assembly from libglib2.0-cil into Mono
Use of uninitialized value $_ in scalar chomp at /usr/share/cli-common/runtimes.d/mono line 144.
Use of uninitialized value $fullname in concatenation (.) or string at /usr/share/cli-common/runtimes.d/mono line 113.
E: installing Assembly /usr/lib/cli/glib-sharp-2.0/glib-sharp.dll failed
E: Installation of libglib2.0-cil with /usr/share/cli-common/runtimes.d/mono failed
* Installing 1 assembly from libgmime2.4-cil into Mono
E: Installation of libgmime2.4-cil with /usr/share/cli-common/runtimes.d/mono failed
* Installing 1 assembly from libgnome-keyring1.0-cil into Mono
E: Installation of libgnome-keyring1.0-cil with /usr/share/cli-common/runtimes.d/mono failed
* Installing 6 assemblies from libgnome-vfs2.0-cil into Mono
Use of uninitialized value $_ in scalar chomp at /usr/share/cli-common/runtimes.d/mono line 144.
Use of uninitialized value $fullname in concatenation (.) or string at /usr/share/cli-common/runtimes.d/mono line 113.
E: installing Assembly /usr/lib/cli/gnome-vfs-sharp-2.0/gnome-vfs-sharp.dll failed
E: Installation of libgnome-vfs2.0-cil with /usr/share/cli-common/runtimes.d/mono failed
* Installing 2 assemblies from libgnome2.24-cil into Mono
E: Installation of libgnome2.24-cil with /usr/share/cli-common/runtimes.d/mono failed
* Installing 1 assembly from libgnomepanel2.24-cil into Mono
E: Installation of libgnomepanel2.24-cil with /usr/share/cli-common/runtimes.d/mono failed
* Installing 5 assemblies from libgtk2.0-cil into Mono
E: Installation of libgtk2.0-cil with /usr/share/cli-common/runtimes.d/mono failed
* Installing 1 assembly from liblaunchpad-integration1.0-cil into Mono
E: Installation of liblaunchpad-integration1.0-cil with /usr/share/cli-common/runtimes.d/mono failed
* Installing 3 assemblies from libmono-addins-gui0.2-cil into Mono
Use of uninitialized value $_ in scalar chomp at /usr/share/cli-common/runtimes.d/mono line 144.
Use of unini...

Read more...

Revision history for this message
Dr. The Fugitive (drbrando007) wrote :

This is just an issue with mono, apparently does not appear on omap boards. This will appear on a physical machine as well as virtual, I have so far successfully rootstock ubuntu-minimal (meerkat) and used apt-get install --no-install-recommends ubuntu-netbook to build thus far. Am working on a kernel for my device(qualcomm snapdragon), will report results.

Revision history for this message
Oliver Grawert (ogra) wrote :

this bug has nothing to do with mono at all, please do not confuse it with the (already open and already commented) mono tasks.

this bug is about qemu hanging if a big task of packages is installed, even if there are no mono packages among them. note in the initial logs that it hangs in various places, usually when unpacking iso-codes. please dont make more mess out of this bug than it already is, you commented the same on three bugs now which are all different.

Revision history for this message
Dr. The Fugitive (drbrando007) wrote :

agreed, guilty of shotgun reporting. will try to reproduce on qemu less the mono-package installs before submitting to this tracker again.

Thank you for the feedback.

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Moving this bug over to the qemu-linaro package, which now provides qemu-system-arm

affects: qemu-kvm (Ubuntu) → qemu-linaro (Ubuntu)
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for qemu-linaro (Ubuntu) because there has been no activity for 60 days.]

Changed in qemu-linaro (Ubuntu):
status: Incomplete → Expired
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for qemu-linaro (Ubuntu Lucid) because there has been no activity for 60 days.]

Changed in qemu-linaro (Ubuntu Lucid):
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.