8.10 intrepid alpha 6 - root file system mounted read only (XFS) with2.6.27-4-generic

Hi Jason,

Can you also attach you dmesg output when booting with the 2.6.27-4 kernel? Thanks.

Not easily. Remember that the root file system is mounted read only which is also preventing me from mounting any other file system. I'll attach my /var/log/syslog and /var/log/messages to the bug report.

 Jason L. Froebe

Tech log http://www.froebe.net/blog
MyDatabases (free) magazine: http://froebe.net/blog/category/databases/my-databases/
Froebe Fibers http://www.froebe-fibers.com

The opinions expressed within are the sole rantings of a raving lunatic and in no way reflect the rantings, fits, tantrums, errors, corrections, allocutions, or aimless thoughts of Sybase or its employees or of TeamSybase or ISUG. Any resemblence to reasonable thought, or any official or published opinion of Sybase, TeamSybase or ISUG is merely coincidental, and should be totally ignored.

----- Original Message ----
From: Leann Ogasawara <email address hidden>
To: <email address hidden>
Sent: Wednesday, September 24, 2008 5:58:43 PM
Subject: [Bug 273601] Re: 8.10 intrepid alpha 6 - root file system mounted read only (XFS) with2.6.27-4-generic

Hi Jason,

Can you also attach you dmesg output when booting with the 2.6.27-4
kernel? Thanks.

--
8.10 intrepid alpha 6 - root file system mounted read only (XFS) with2.6.27-4-generic
https://bugs.launchpad.net/bugs/273601
You received this bug notification because you are a direct subscriber
of the bug.

first boot in messages and syslog show booting up with 2.6.24. The following boot shows 2.6.27-4. I included both for comparison.

I can confirm this bug.
It first showed up after the upgrade from hardy - i seemed to have fixed it by adding rw to the fstab mount options.
I added a new graphics card last night - all went well - shut down system. It now mounts the root filesystem as read only.
Also as Jason said, it mounts correctly with 2.6.24-21
my mount options are almost the same as Jasons
xfs rw,noatime,nodiratime,nobarrier,logbufs=8
my root is on a raid0 partition
its strange that all was well before the gfx upgrade and ive noted errors about uvesafb - not sure if it has anything to do with this
can anyone help me?

I just upgraded today and can confirm this.

Confirmed with linux-image-2.6.27-7-generic.

This issue is happening to me also with Intrepid Ibex and linux-image-2.6.27-7-generic.

Seems to be a problem with logbsize or logbufs options, see here:

http://groups.google.com/group/linux.kernel/browse_thread/thread/3e7c824e4b988547

officially fixed in vanilla 2.6.27.3:
commit c068663ae65e507814545b59a8e2090f48a85613
Author: Christoph Hellwig <email address hidden>
Date: Sun Oct 12 14:30:44 2008 +0200

    xfs: fix remount rw with unrecognized options

    commit 6c5e51dae2c37127e00be392f40842e08077e96a upstream

    When we skip unrecognized options in xfs_fs_remount we should just break
    out of the switch and not return because otherwise we may skip clearing
    the xfs-internal read-only flag. This will only show up on some
    operations like touch because most read-only checks are done by the VFS
    which thinks this filesystem is r/w. Eventually we should replace the
    XFS read-only flag with a helper that always checks the VFS flag to make
    sure they can never get out of sync.

    Bug reported and fix verified by Marcel Beister on #xfs.
    Bug fix verified by updated xfstests/189.

    Signed-off-by: Christoph Hellwig <email address hidden>
    Acked-by: Eric Sandeen <email address hidden>
    Signed-off-by: Timothy Shimmin <email address hidden>
    Signed-off-by: Linus Torvalds <email address hidden>
    Signed-off-by: Greg Kroah-Hartman <email address hidden>

casperado, thanks so much for the upstream reference. Unfortunately the Intrepid kernel is frozen right now since we are so close to release. However, we should be picking up this patch as a Stable Release Update for Intrepid. The kernel team intends to keep the Intrepid kernel in sync with the usptream stable kernels so I'm assuming this should be fixed with the first post Intrepid kernel release. So if you can just give us a bit of time we'll get this in. For more information about the Stable Release Update process please see https://wiki.ubuntu.com/StableReleaseUpdates . I'll go ahead and open the Intrepid nomination and milestone it so we don't forget. Thanks.

Leann, thanks for the reply on my bug that's a dup of this. I thought I'd already posted my workaround, but it turns out I only posted it on bug 253786 (/dev/.static/dev is left read-only). Since I found a workaround, this bug hasn't caused me any problems.

Anyway, you can convince the init scripts to remount the bind mount read-write as well as the main root filesystem with this in fstab
/dev /dev/.static/dev bind remount,rw 0 0

 That avoids triggering the bug with XFS root filesystems. And it makes /dev/.static/dev read-write, so MAKEDEV works, so I'll be keeping this in my fstab even after the XFS bug is fixed. see bug 253786 for more discussion on ro/rw, and how it gets that way in the first place.

 If you don't want to break the kernel freeze, you could possibly add this workaround to update-manager's collection of quirks. (e.g. test if xfs_info / shows that the root filesystem is xfs, then append a line to /etc/fstab.)

 Hmm, reading the fix description, XFS has a per-FS read-only flag? That doesn't sound like it would interact well with various combinations of read-only and read-write bind mounts... Could be a security hole if you think you have a ro bind mount for a chroot, but some FS ops are allowed. I'll test this and report a bug if I remember and have time, and it's not already known.

http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-intrepid.git;a=commit;h=34d6c6d0d08f10d1663247d4b225fe4148728520

https://edge.launchpad.net/ubuntu/+source/linux/2.6.27-8.17

Hello,

I was bitten by this bug too (very strange, the kernel reported the FS as read-write, and touch and rm would work , but chmod gave 'read only filesystem'). Getting rid of XFS flags in /etc/fstab was a workaround for me too.

The people on #xfs freenode helped me out here, and they noted that, apart from this patch, ubuntu intrepid is also missing a very important data corruption fix for XFS.

<hch> and
          http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=co
          mmitdiff;h=73f6aa4d44ab6157badc456ddfa05b31e58de5f0 while you're at
          it
<hch> without the latter one you can get really bad corruption if you rely on
          barriers
<hch> looks like not many people use xfs on ubuntu..

(...)

<hch> but the barrier fix would be extremely useful too
<hch> otherwise we'll get lots of complaints from ubuntu users about data
          corruption

should i log a separate bugreport about this ?

thanks!
greetings,
Frank

> should i log a separate bugreport about this ?

 I think that's already in Ubuntu's released Intrepid kernel, not just the proposed-updates kernel. See the changelog.Debian.gz

linux (2.6.27-7.13) intrepid; urgency=low
...
  * Fix barrier fail detection in XFS
...
 -- Tim Gardner <email address hidden> Sun, 19 Oct 2008 10:06:21 -0600

 Which was a day after Linux 2.6.27.2 was released: from ChangeLog-2.6.27.2:
commit 6bcd6d778419101dd96cbbdf03eeab8d779b1d66
Author: Greg Kroah-Hartman <email address hidden>
Date: Sat Oct 18 10:57:22 2008 -0700

    Linux 2.6.27.2
...
commit 769b0455c1ec257b9e5067129accab1a6052de4c
Author: Christoph Hellwig <email address hidden>
Date: Fri Oct 10 17:28:29 2008 +1100

    Fix barrier fail detection in XFS

> I think that's already in Ubuntu's released Intrepid kernel

 oops, no it isn't, the patch you posted a line-wrapped URL to is in 2.6.27.7. (No mention of it in ChangeLog-2.6.27.7, though. I though the Changelogs were auto-generated, but obviously patches can be in patch 2.6.27.7.gz without an entry mentioning xfs in the Changelog.)

 Anyway, I expect it will get packaged for intrepid-proposed in due time, like previous 2.6.27 stable releases. I agree that pushing it to intrepid-updates with more priority than usual would make sense.

On Fri, Nov 21, 2008 at 03:53:03AM -0000, Peter Cordes wrote:

> Anyway, I expect it will get packaged for intrepid-proposed in due
> time, like previous 2.6.27 stable releases. I agree that pushing it to
> intrepid-updates with more priority than usual would make sense.

Yes, this is already in the git repository and has been uploaded to
proposed as 2.6.26-10.20. It should release fairly shortly.

I just updated to 2.6.27-9, and my system is still booting as read-only. It's been a month since Intrepid was released.

I just installed the 2.6.26-10.20 kernel from intrepid-proposed repository and can confirm that the issue on the root filing system being mounted read only, is resolved for me. Thanks.

2.6.27-10 from intrepid-proposed doesn't mount the root file system at all on my system:

Begin: Loading essential drivers... ...
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Waiting for root file system... ...
Done.
Begin: Running /scripts/local-premount ...
19+0 records in
19+0 records out
kinit: name_to_dev_t(/dev/disk/by-uuid/874b696f-fd43-4c6d-9493-a8a01941bbc0) = dev(8,4)
kinit: trying to resume from /dev/disk/by-uuid/874b696f-fd43-4c6d-9493-a8a01941bbc0
kinit: No resume image, doing normal boot...
Done.
mount: mounting /dev/md1 on /root failed: Device or resource busy
Begin: Runnning /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
mount: mounting /root/dev on /dev/.static/dev failed: No such file or directory
Done.
mount: mounting /sys on /root/sys failed: No such file or directory
mount: mounting /proc on /root/proc failed: No such file or directory
Target filesystem doesn't have /sbin/init.
No init found. Try passing init= bootarg.

It looks like that was just mdadm taking too long to assemble the raid. Once I resolved that, the file system mounts fine. This bug is resolved for me.

Per a decision made by the Ubuntu Kernel Team, bugs will longer be assigned to the ubuntu-kernel-team in Launchpad as part of the bug triage process. The ubuntu-kernel-team is being unassigned from this bug report. Refer to https://wiki.ubuntu.com/KernelTeamBugPolicies for more information. Thanks.