libvirt inside lxd container cannot start virbr0 (Unable to set bridge virbr0 forward_delay: Permission denied)
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
libvirt (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
Bionic |
Won't Fix
|
Undecided
|
Unassigned | ||
Cosmic |
Won't Fix
|
Undecided
|
Unassigned |
Bug Description
Ubuntu 18.04 lxd container, running on Ubuntu 18.04 host (kernel 4.15.0-38-generic)
Inside the container, I installed libvirt-bin. However it fails to start the predefined 'default' network:
root@bionic:/etc# virsh net-start default
error: Failed to start network default
error: Unable to set bridge virbr0 forward_delay: Permission denied
root@bionic:/etc# echo $?
1
root@bionic:/etc# virsh net-list
Name State Autostart Persistent
-------
root@bionic:/etc# virsh net-list --all
Name State Autostart Persistent
-------
default inactive yes yes
Here is the config:
root@bionic:/etc# cat /etc/libvirt/
<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
virsh net-edit default
or other application using the libvirt API.
-->
<network>
<name>
<uuid>
<forward mode='nat'/>
<bridge name='virbr0' stp='on' delay='0'/>
<mac address=
<ip address=
<dhcp>
<range start='
</dhcp>
</ip>
</network>
Using "virsh net-edit default" to remove "delay='0'" does not make a difference; it gets reinserted and the same error occurs.
/var/log/syslog shows:
Nov 12 11:59:11 bionic networkd-
Nov 12 11:59:11 bionic systemd-
Nov 12 11:59:11 bionic libvirtd[225]: 2018-11-12 11:59:11.257+0000: 251: error : virNetDevBridge
Nov 12 11:59:11 bionic systemd-
Nov 12 11:59:11 bionic networkd-
Nov 12 11:59:11 bionic networkd-
Nov 12 11:59:11 bionic networkd-
Nov 12 11:59:11 bionic networkd-
Nov 12 11:59:11 bionic networkd-
Nov 12 11:59:11 bionic networkd-
Nov 12 11:59:11 bionic networkd-
Nov 12 11:59:11 bionic networkd-
Attaching strace to libvirtd, this is what I see:
...
[pid 225] <... recvmsg resumed> {msg_name=
[pid 250] ioctl(23, SIOCGIFINDEX, {ifr_name=
[pid 225] recvmsg(13, <unfinished ...>
[pid 250] <... ioctl resumed> , }) = 0
[pid 225] <... recvmsg resumed> {msg_name=
[pid 250] close(23 <unfinished ...>
[pid 225] poll([{fd=4, events=POLLIN}, {fd=6, events=POLLIN}, {fd=9, events=POLLIN}, {fd=10, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=17, events=POLLIN}, {fd=18, events=POLLIN}, {fd=20, events=POLLIN}], 11, 4996 <unfinished ...>
[pid 250] <... close resumed> ) = 0
[pid 250] ioctl(22, SIOCBRADDIF) = 0
[pid 250] close(22) = 0
[pid 250] socket(AF_UNIX, SOCK_DGRAM, 0) = 22
[pid 250] fcntl(22, F_GETFD) = 0
[pid 250] fcntl(22, F_SETFD, FD_CLOEXEC) = 0
[pid 250] ioctl(22, SIOCGIFFLAGS, {ifr_name=
[pid 250] ioctl(22, SIOCSIFFLAGS, {ifr_name=
[pid 250] close(22) = 0
[pid 250] access(
[pid 250] socket(AF_UNIX, SOCK_DGRAM, 0) = 22
[pid 250] fcntl(22, F_GETFD) = 0
[pid 250] fcntl(22, F_SETFD, FD_CLOEXEC) = 0
[pid 250] access(
[pid 250] openat(AT_FDCWD, "/sys/class/
[pid 250] gettid() = 250
[pid 250] write(2, "2018-11-12 12:02:07.815+0000: 250: error : virNetDevBridge
...
WORKAROUND: "lxc config set bionic security.privileged yes && lxc restart bionic"
However, I don't think that privileged mode should be necessary. If I turn off privileged mode, I can still create and edit bridges by hand, including setting the forwarding delay:
root@bionic:~# brctl show
bridge name bridge id STP enabled interfaces
root@bionic:~# brctl addbr testbr0
root@bionic:~# brctl show
bridge name bridge id STP enabled interfaces
testbr0 8000.000000000000 no
root@bionic:~# brctl setfd testbr0 0
root@bionic:~# brctl showstp testbr0 | grep "forward delay"
forward delay 0.00 bridge forward delay 0.00
root@bionic:~# cat /sys/class/
0
root@bionic:~# brctl setfd testbr0 1
root@bionic:~# brctl showstp testbr0 | grep "forward delay"
forward delay 1.00 bridge forward delay 1.00
root@bionic:~# cat /sys/class/
100
However, writing to the /sys filesystem directly does not work:
root@bionic:~# echo 0 > /sys/class/
bash: /sys/class/
root@bionic:~#
In fact, it looks like "brctl setfd" is failing silently to access the /sys entry, as shown by strace, but is falling back to using an ioctl which succeeds.
root@bionic:~# strace -f brctl setfd testbr0 1
...
socket(AF_UNIX, SOCK_STREAM, 0) = 4
brk(NULL) = 0x55e926464000
brk(0x55e926485000) = 0x55e926485000
openat(AT_FDCWD, "/sys/class/
ioctl(4, SIOCDEVPRIVATE, 0x7fff63a06da0) = 0
exit_group(0) = ?
+++ exited with 0 +++
root@bionic:~# echo $?
0
This suggests that the proper solution is for libvirt to do something similar.
ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: libvirt0:amd64 4.0.0-1ubuntu8.5
ProcVersionSign
Uname: Linux 4.15.0-38-generic x86_64
ApportVersion: 2.20.9-0ubuntu7.4
Architecture: amd64
Date: Mon Nov 12 11:44:59 2018
ProcEnviron:
TERM=xterm-
PATH=(custom, no user)
LANG=C.UTF-8
SourcePackage: libvirt
UpgradeStatus: No upgrade log present (probably fresh install)
Hi,
to me it seems this is not a bug, but an issue with the default config.
To run virtualization in a LXD container - which by default is unprivileged for security you have to make some changes.
We will not change LXD/Libvirt defaults for that afaik, but the following is my recommendation as a container profile addition to get KVM+Libvirt running fine in a container.
config: kernel_ modules: openvswitch, nbd,ip_ tables, ip6_tables, kvm privileged: "true"
boot.autostart: "true"
linux.
security.nesting: "true"
security.
description: ""
devices:
eth0:
mtu: "9000"
name: eth0
nictype: bridged
parent: lxdbr0
type: nic
kvm:
path: /dev/kvm
type: unix-char
mem:
path: /dev/mem
type: unix-char
tun:
path: /dev/net/tun
type: unix-char
name: kvm
used_by: []
You can create that with "lxc profile new kvm" and then launch those guests that need it with default+kvm profile, while leaving the other secure and unprivileged. daily:c/ amd64 c --profile default --profile kvm
$ lxc launch ubuntu-
I hope that helps to understand, but IMHO it is not a bug.