overlayfs v1: renaming existing file uses chardev whiteout (should be symlink)

Bug #1410480 reported by Ryan Tandy
228
This bug affects 31 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Fix Released
High
Andy Whitcroft

Bug Description

On a recent vivid livecd, after I rename a file that exists in the lower filesystem, the whiteout entry is visible.

root@ubuntu:~# ls -l /etc/fstab
-rw-r--r-- 1 root root 63 Jan 13 11:56 /etc/fstab
root@ubuntu:~# mv /etc/fstab /etc/fstab.renamed
root@ubuntu:~# ls -l /etc/fstab*
c--------- 1 root root 0, 0 Jan 13 19:57 /etc/fstab
-rw-r--r-- 1 root root 63 Jan 13 11:56 /etc/fstab.renamed
root@ubuntu:~#

It still happens if I trigger a copy-up first by editing the file.

Files that exist only in the upper filesystem (created after booting the livecd) work properly.

On utopic, everything works properly. I have not tested a 3.17 kernel.

ProblemType: Bug
DistroRelease: Ubuntu 15.04
Package: linux-image-3.18.0-8-generic 3.18.0-8.9
ProcVersionSignature: Ubuntu 3.18.0-8.9-generic 3.18.1
Uname: Linux 3.18.0-8-generic x86_64
ApportVersion: 2.15.1-0ubuntu2
Architecture: amd64
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/hwC0D0', '/dev/snd/pcmC0D2c', '/dev/snd/pcmC0D1p', '/dev/snd/pcmC0D0c', '/dev/snd/pcmC0D0p', '/dev/snd/controlC0', '/dev/snd/by-path', '/dev/snd/hwC1D3', '/dev/snd/hwC1D2', '/dev/snd/hwC1D1', '/dev/snd/hwC1D0', '/dev/snd/pcmC1D9p', '/dev/snd/pcmC1D8p', '/dev/snd/pcmC1D7p', '/dev/snd/pcmC1D3p', '/dev/snd/controlC1', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CasperVersion: 1.347
Date: Tue Jan 13 19:42:41 2015
IwConfig:
 eth0 no wireless extensions.

 lo no wireless extensions.
LiveMediaBuild: Ubuntu 15.04 "Vivid Vervet" - Alpha amd64 (20150113)
Lsusb:
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
 Bus 002 Device 003: ID 046d:c31c Logitech, Inc. Keyboard K120 for Business
 Bus 002 Device 002: ID 413c:3012 Dell Computer Corp. Optical Wheel Mouse
 Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
MachineType: To Be Filled By O.E.M. To Be Filled By O.E.M.
ProcEnviron:
 TERM=rxvt-unicode-256color
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB: 0 nouveaufb
ProcKernelCmdLine: initrd=hostname/vivid/desktop/amd64/casper/initrd.lz boot=casper netboot=nfs nfsroot=10.0.2.1:/var/lib/tftpboot/hostname/vivid/desktop/amd64 quiet splash file=/cdrom/preseed/hostname.seed url=http://10.0.2.64/preseed/ubiquity.cfg text BOOT_IMAGE=hostname/vivid/desktop/amd64/casper/vmlinuz.efi
PulseList:
 Error: command ['pacmd', 'list'] failed with exit code 1: Home directory not accessible: Permission denied
 No PulseAudio daemon running, or not running as session daemon.
RelatedPackageVersions:
 linux-restricted-modules-3.18.0-8-generic N/A
 linux-backports-modules-3.18.0-8-generic N/A
 linux-firmware 1.140
RfKill:

SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 06/07/2010
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: 080015
dmi.board.asset.tag: To Be Filled By O.E.M.
dmi.board.name: M61PMP/M61PMP-K
dmi.board.vendor: FOXCONN
dmi.board.version: 1.0
dmi.chassis.asset.tag: To Be Filled By O.E.M.
dmi.chassis.type: 3
dmi.chassis.vendor: To Be Filled By O.E.M.
dmi.chassis.version: To Be Filled By O.E.M.
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr080015:bd06/07/2010:svnToBeFilledByO.E.M.:pnToBeFilledByO.E.M.:pvrToBeFilledByO.E.M.:rvnFOXCONN:rnM61PMP/M61PMP-K:rvr1.0:cvnToBeFilledByO.E.M.:ct3:cvrToBeFilledByO.E.M.:
dmi.product.name: To Be Filled By O.E.M.
dmi.product.version: To Be Filled By O.E.M.
dmi.sys.vendor: To Be Filled By O.E.M.

Revision history for this message
Ryan Tandy (rtandy) wrote :
Revision history for this message
Ryan Tandy (rtandy) wrote :

Not true of all directories, actually. /root is special somehow. (detach_mounts is suspicious... probably the initramfs mounted the real root at /root, or something like that?) Renaming /mnt or /opt works properly and no whiteout is left behind.

summary: - 3.18 overlayfs doesn't rename existing files/dirs properly
+ 3.18 overlayfs: renaming existing file leaves whiteout visible
Ryan Tandy (rtandy)
description: updated
Revision history for this message
Brad Figg (brad-figg) wrote : Status changed to Confirmed

This change was made by a bot.

Changed in linux (Ubuntu):
status: New → Confirmed
Revision history for this message
Ryan Tandy (rtandy) wrote : Re: 3.18 overlayfs: renaming existing file leaves whiteout visible

OK, this is interesting.

root@plonk:~# mkdir /lower /upper /work /ovl
root@plonk:~# touch /lower/testfile
root@plonk:~# mkdir /lower/testdir
root@plonk:~# mount -t overlay -o lowerdir=/lower,upperdir=/upper,workdir=/work overlay /ovl
root@plonk:~# mv /ovl/testfile /ovl/renamed-testfile
root@plonk:~# ls -l /ovl
total 4
-rw-r--r-- 1 root root 0 Jan 13 13:15 renamed-testfile
drwxr-xr-x 2 root root 4096 Jan 13 13:15 testdir
root@plonk:~# umount /ovl
root@plonk:~# rm -r /upper/* /work/*
root@plonk:~# mount -t overlayfs -o lowerdir=/lower,upperdir=/upper,workdir=/work overlay /ovl
root@plonk:~# mv /ovl/testfile /ovl/renamed-testfile
root@plonk:~# ls -l /ovl
total 4
-rw-r--r-- 1 root root 0 Jan 13 13:15 renamed-testfile
drwxr-xr-x 2 root root 4096 Jan 13 13:15 testdir
c--------- 1 root root 0, 0 Jan 13 13:16 testfile
root@plonk:~# uname -srm
Linux 3.18.0-8-generic x86_64

That is, "mount -t overlay" works fine, but "mount -t overlayfs" has the bug.

I tried the 3.18.2 and 3.19-rc4 mainline kernels. "mount -t overlay" works fine there as well, and they don't understand "mount -t overlayfs".

Revision history for this message
Ryan Tandy (rtandy) wrote :

Reading what seems to be the relevant commit: http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-vivid.git;a=commitdiff;h=9a742a48114e416e9a4cd6e8615ba0d261dda599

If I'm reading that right, then a v1 overlayfs (mounted as "overlayfs") should be using the old-style (symlink) whiteouts, right? Why am I seeing char devices at all?

Revision history for this message
Ryan Tandy (rtandy) wrote :

Unlinking a file or directory and renaming a file all result in a symlink whiteout, it's just renaming a file that does the wrong thing.

Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.19 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.19-rc4-vivid/

Changed in linux (Ubuntu):
importance: Undecided → Medium
Revision history for this message
Ryan Tandy (rtandy) wrote :

Hi,

this is an Ubuntu-specific bug, the overlayfs v1 compat patch is not included upstream.

(For completeness, I did boot the 3.19-rc4 mainline kernel and verify that, per comment 4.)

FWIW, I think (but am not sure how to confirm) that it's falling into vfs_rename with flags including RENAME_WHITEOUT...

Thanks.

tags: added: kernel-unable-to-test-upstream
summary: - 3.18 overlayfs: renaming existing file leaves whiteout visible
+ overlayfs_v1: renaming existing file uses chardev whiteout (should be
+ symlink)
Andy Whitcroft (apw)
Changed in linux (Ubuntu):
importance: Medium → High
assignee: nobody → Andy Whitcroft (apw)
milestone: none → ubuntu-15.01
Andy Whitcroft (apw)
summary: - overlayfs_v1: renaming existing file uses chardev whiteout (should be
+ overlayfs v1: renaming existing file uses chardev whiteout (should be
symlink)
Revision history for this message
Andy Whitcroft (apw) wrote :

Ok confirmed, when the source is already copied up, and we rename it we have to whiteout the source. This is done using RENAME_WHITEOUT which leaves a V2 style whiteout in place.

Changed in linux (Ubuntu):
status: Confirmed → In Progress
Revision history for this message
Ubuntu QA Website (ubuntuqa) wrote :

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1410480

tags: added: iso-testing
Revision history for this message
Andy Whitcroft (apw) wrote :

Ok preliminary patches are in testing. As this is severely affecting the installer, I have separatly switched that up to use overlay (V2 format) directly under Bug #1412411.

Revision history for this message
Andy Whitcroft (apw) wrote :

Patches applied, pending upload after A2 freeze lifts.

Changed in linux (Ubuntu):
status: In Progress → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package linux - 3.18.0-11.12

---------------
linux (3.18.0-11.12) vivid; urgency=low

  [ Leann Ogasawara ]

  * Release Tracking Bug
    - LP: #1414085

  [ Andy Whitcroft ]

  * SAUCE: arm64 -- psci tell the compiler which registers we are needing
    values in
    - LP: #1414002
 -- Leann Ogasawara <email address hidden> Fri, 23 Jan 2015 09:10:09 -0800

Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.