linux-image-2.6.24-18-xen breaks mono

Linux srv1 2.6.24-19-xen #1 SMP Wed Jun 4 17:03:44 UTC 2008 x86_64 GNU/Linux

Same deal here with 2.6.24-19-xen on AMD64 - No mono applications work, f-spot example below.

# f-spot
TYPE: 1
**
** ERROR:(mini-amd64.c:192):amd64_patch: assertion failed: (amd64_is_imm32 (disp))
Stacktrace:

Native stacktrace:

        f-spot [0x51bb67]
        /lib/libpthread.so.0 [0x7f39dde0e7d0]
        /lib/libc.so.6(gsignal+0x35) [0x7f39dd84f095]
        /lib/libc.so.6(abort+0x110) [0x7f39dd850af0]
        /usr/lib/libglib-2.0.so.0(g_assertion_message+0x117) [0x7f39de279bc7]
        /usr/lib/libglib-2.0.so.0 [0x7f39de27a062]
        f-spot [0x42dbb8]
        f-spot [0x4e6229]
        f-spot [0x50849b]
        f-spot [0x508cd9]
        f-spot [0x5091d3]
        f-spot(mono_exception_from_name_two_strings+0xdd) [0x48c38d]
        f-spot(mono_runtime_init+0x13e) [0x48e64e]
        f-spot [0x4e31e5]
        f-spot(mono_main+0x341) [0x4169b1]
        /lib/libc.so.6(__libc_start_main+0xf4) [0x7f39dd83b1c4]
        f-spot(realloc+0x381) [0x416109]

Debug info from gdb:

(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
[New Thread 0x7f39dead77a0 (LWP 32191)]
[New Thread 0x7f39deae4950 (LWP 32192)]
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
0x00007f39dde0d5cb in read () from /lib/libpthread.so.0
  2 Thread 0x7f39deae4950 (LWP 32192) 0x00007f39dde0de81 in nanosleep ()
   from /lib/libpthread.so.0
  1 Thread 0x7f39dead77a0 (LWP 32191) 0x00007f39dde0d5cb in read ()
   from /lib/libpthread.so.0

Thread 2 (Thread 0x7f39deae4950 (LWP 32192)):
#0 0x00007f39dde0de81 in nanosleep () from /lib/libpthread.so.0
#1 0x00000000004b8f4f in ?? ()
#2 0x00007f39dde063f7 in start_thread () from /lib/libpthread.so.0
#3 0...

I have been suggested a work around for mono to work with the bugged kernel:

1. Download mono source
2. Edit file mini/mini-amd64.h adding #define MONO_ARCH_NOMAP32BIT at the end of the file, just before the last #endif
3. Compile mono using ./configure --prefix=/usr if you want to replace the current mono installation and make you life easier with GAC
4. If the case, add in GAC the assemblies not fund by the new mono installation (http://www.mono-project.com/Assemblies_and_the_GAC)

Note that this is a workaround for the kernel bug outlined in my previous post, not a fix for mono.

Hope it helps

This seems to affect the 'platinum' build machine when uploading to a PPA, but oddly not any of the other amd64 build machines.

(eg. log at http://launchpadlibrarian.net/15836353/buildlog_ubuntu-hardy-amd64.galaxium-svn_0.8~svn1054-0ubuntu1~hardy1~ppa1_FAILEDTOBUILD.txt.gz )

The Ubuntu Kernel Team is planning to move to the 2.6.27 kernel for the upcoming Intrepid Ibex 8.10 release. As a result, the kernel team would appreciate it if you could please test this newer 2.6.27 Ubuntu kernel. There are one of two ways you should be able to test:

1) If you are comfortable installing packages on your own, the linux-image-2.6.27-* package is currently available for you to install and test.

--or--

2) The upcoming Alpha5 for Intrepid Ibex 8.10 will contain this newer 2.6.27 Ubuntu kernel. Alpha5 is set to be released Thursday Sept 4. Please watch http://www.ubuntu.com/testing for Alpha5 to be announced. You should then be able to test via a LiveCD.

Please let us know immediately if this newer 2.6.27 kernel resolves the bug reported here or if the issue remains. More importantly, please open a new bug report for each new bug/regression introduced by the 2.6.27 kernel and tag the bug report with 'linux-2.6.27'. Also, please specifically note if the issue does or does not appear in the 2.6.26 kernel. Thanks again, we really appreicate your help and feedback.

The Intrepid Ibex 8.10 Beta release was most recently announced - http://www.ubuntu.com/testing/intrepid/beta . It contains the 2.6.27 Ubuntu kernel. It would be great if you could test and verify if this is still an issue. The status is being set to Incomplete until we receive further feedback. Thanks.

*This is an automated response*

This bug report is being closed because we received no response to the previous request for information. Please reopen this if it is still an issue in the actively developed pre-release of Jaunty Jackalope 9.04 - http://cdimage.ubuntu.com/releases/jaunty . To reopen the bug report simply change the Status of the "linux" task back to "New".

I can reproduce this bug on 2.6.24-23-xen; i.e. the latest available kernel available for our current LTS. I don't think the bug should be closed because no one's been able to test it on later releases. Regardless of it's status in those release, it's still a bug in 8.04.

this can be reproduced in launchapd buildd system.
Eg.: https://launchpad.net/~do-testers/+archive/+build/842475

When will that be fixed? It still makes building of some programs in PPA buildd service impossible.

Here's a patch to fix this. It works for me; after rebuilding the
kernel with the patch, I'm able to install mono applications on the
64-bit Xen hardy kernel. Can we please get this into an SRU? I'm
happy to test any upload to hardy-proposed.

Thanks for the patch James. The bug should probablt be fixed in Jaunty (if it is still a problem there) before porting the patch to the older releases. Could someone test if this is still an issue in Intrepid and Jaunty, or if it is Hardy specific? Which kernel do the PPA build daemons run, as it would be really great to get this fixed there?

As far as I can see, the -xen kernel has been summarily dropped post Hardy, so whether or not it's fixed in later releases seems somewhat moot.

The PPA buildds run the latest Hardy Xen kernel available, which is why I'm so keen to get an SRU, precisely so that we can fix this problem for mono PPA users.

Fix committed to Hardy http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-hardy.git;a=commitdiff;h=6b2514fb662158f2def03a618c5c671c95f636a8

Accepted into hardy-proposed; please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

I can confirm the packages in hardy-proposed work for me:

Linux osmium-host 2.6.24-24-xen #1 SMP Fri Feb 20 11:19:37 UTC 2009 x86_64 GNU/Linux

ii linux-image-xen 2.6.24.24.26
ii linux-image-2.6.24-24-xen 2.6.24-24.50
ii linux-ubuntu-modules-2.6.24-24-xen 2.6.24-24.38

can successfully install mono packages.

What's the plan for upgrading our buildds ?

All the buildds that require the fix have been updated to the kernel from -proposed and are happily running it. Still hoping to see this moved to -updates, obviously, but I realize that requires feedback on a lot of other bug fixes that I can't test as easily as this one.

This bug was fixed in the package linux - 2.6.24-24.53

---------------
linux (2.6.24-24.53) hardy-proposed; urgency=low

  [Stefan Bader]

  * Rebuild of 2.6.24-24.51 with 2.6.24-23.52 security patches applied.

linux (2.6.24-24.51) hardy-proposed; urgency=low

  [Alessio Igor Bogani]

  * rt: Updated PREEMPT_RT support to rt27
    - LP: #324275

  [Steve Beattie]

  * fix apparmor memory leak on deleted file ops
    - LP: #329489

  [Upstream Kernel Changes]

  * KVM: MMU: Add locking around kvm_mmu_slot_remove_write_access()
    - LP: #335097, #333409
  * serial: 8250: fix shared interrupts issues with SMP and RT kernels
    - LP: #280821
  * 8250.c: port.lock is irq-safe
    - LP: #280821
  * ACPI: Clear WAK_STS on resume
    - LP: #251338

linux (2.6.24-24.50) hardy-proposed; urgency=low

  [Alok Kataria]

  * x86: add X86_FEATURE_HYPERVISOR feature bit
    - LP: #319945
  * x86: add a synthetic TSC_RELIABLE feature bit
    - LP: #319945
  * x86: vmware: look for DMI string in the product serial key
    - LP: #319945
  * x86: Hypervisor detection and get tsc_freq from hypervisor
    - LP: #319945
  * x86: Use the synthetic TSC_RELIABLE bit to workaround virtualization
    anomalies.
    - LP: #319945
  * x86: Skip verification by the watchdog for TSC clocksource.
    - LP: #319945
  * x86: Mark TSC synchronized on VMware.
    - LP: #319945

  [Colin Ian King]

  * SAUCE: Bluetooth USB: fix kernel panic during suspend while streaming
    audio to bluetooth headset
    - LP: #331106

  [James Troup]

  * XEN: Enable architecture specific get_unmapped_area_topdown
    - LP: #237724

  [Stefan Bader]

  * Xen: Fix FTBS after Vmware TSC updates.
    - LP: #319945

  [Upstream Kernel Changes]

  * r8169: fix RxMissed register access
    - LP: #324760
  * r8169: Tx performance tweak helper
    - LP: #326891
  * r8169: use pci_find_capability for the PCI-E features
    - LP: #326891
  * r8169: add 8168/8101 registers description
    - LP: #326891
  * r8169: add hw start helpers for the 8168 and the 8101
    - LP: #326891
  * r8169: additional 8101 and 8102 support
    - LP: #326891
  * Fix memory corruption in console selection
    - LP: #329007

linux (2.6.24-23.52) hardy-security; urgency=low

  [Stefan Bader]
  * rt: Fix FTBS caused by shm changes
    - CVE-2009-0859

  [Steve Beattie]

  * fix apparmor memory leak on deleted file ops
    - LP: #329489

  [Upstream Kernel Changes]

  * NFS: Remove the buggy lock-if-signalled case from do_setlk()
    - CVE-2008-4307
  * sctp: Avoid memory overflow while FWD-TSN chunk is received with bad
    stream ID
    - CVE-2009-0065
  * net: 4 bytes kernel memory disclosure in SO_BSDCOMPAT gsopt try #2
    - CVE-2009-0676
  * sparc: Fix mremap address range validation.
    - CVE-2008-6107
  * copy_process: fix CLONE_PARENT && parent_exec_id interaction
    - CVE-2009-0028
  * security: introduce missing kfree
    - CVE-2009-0031
  * eCryptfs: check readlink result was not an error before using it
    - CVE-2009-0269
  * dell_rbu: use scnprintf() instead of less secure sprintf()
    - CVE-2009-0322
  * drivers/net/skfp: if !capable(CAP_NET_ADMIN): inverted logic
    - CVE-2009-0675
  * Ext4: Fix online res...

I am having the same issue with Jaunty Jackalope, log ahows as below:
Are there any fixes which are Jaunty specific rather than Hardy?

Thanks.

**
ERROR:mini-amd64.c:199:amd64_patch: assertion failed: (amd64_is_imm32 (disp))
Stacktrace:

Native stacktrace:

        /usr/bin/mono [0x429e65]
        /lib/libpthread.so.0 [0x7fec57c24080]
        /lib/libc.so.6(gsignal+0x35) [0x7fec57650fb5]
        /lib/libc.so.6(abort+0x183) [0x7fec57652bc3]
        /usr/lib/libglib-2.0.so.0(g_assertion_message+0x113) [0x7fec5829d4f3]
        /usr/lib/libglib-2.0.so.0 [0x7fec5829da82]
        /usr/bin/mono [0x437e98]
        /usr/bin/mono [0x540b9c]
        /usr/bin/mono [0x566640]
        /usr/bin/mono [0x5679f1]
        /usr/bin/mono [0x56812e]
        /usr/bin/mono [0x4bd42f]
        /usr/bin/mono(mono_runtime_init+0x14f) [0x4c05af]
        /usr/bin/mono [0x53cf3c]
        /usr/bin/mono(mono_main+0x311) [0x417831]
        /lib/libc.so.6(__libc_start_main+0xe6) [0x7fec5763c5a6]
        /usr/bin/mono [0x416ef9]

Debug info from gdb:

=================================================================
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================

[Mon Jul 13 16:37:48 2009] [error] Failed to connect to mod-mono-server after several attempts to spawn the process.