Updating systemd kills network on bionic

Bug #1782709 reported by Uli.Heller
32
This bug affects 6 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Expired
Medium
Unassigned
systemd (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

I do have an ubuntu bionic server installation without a graphical interface. The server
has two active network adapters, both connected to the internet. One is used for outgoing
internet traffic, the other for incoming. The incoming adapter lives on a public network segment
(something like 88.236.133.104/29). I do have multiple servers within this segment.

Typically, I update the server regularly, meaning:

* ssh to the incoming adapter: ssh 88.236.133.108
* apt-get update
* apt-get upgrade
* apt-get dist-upgrade
* apt-get clean
* reboot

It used to work ok.

BUT: Today, "apt-get upgrade" seem to take really long when updating systemd.
After a could of minutes, I realized that the ssh session became inactive. I couldn't type
into it, I had to abort it via ~. .

* ssh from the internet to the incoming adapter was not working anymore - ssh 88.236.133.108
* ssh via another server within the public network segment worked
    * ssh 88.236.133.106 -> ssh 88.236.133.108
* After running "netplan apply" everything was fine again

Here my netplan configuration (I changed the ip addresses):

network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      dhcp4: yes
    eno2:
      dhcp4: no
      addresses: [88.236.133.108/29]
      #gateway4: 88.236.133.105
      routes:
       - to: 0.0.0.0/0
         via: 88.236.133.105
         table: 120
      routing-policy:
       - from: 88.236.133.108/32
         table: 120
       - to: 88.236.133.108/32
         table: 120

Unfortunately, I don't have the console output avalable anymore.
Shall I provide additional info?

Best regards, Uli
---
ProblemType: Bug
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Jul 20 08:10 seq
 crw-rw---- 1 root audio 116, 33 Jul 20 08:10 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.9-0ubuntu7.2
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
DistroRelease: Ubuntu 18.04
InstallationDate: Installed on 2018-07-14 (5 days ago)
InstallationMedia: Ubuntu-Server 18.04 LTS "Bionic Beaver" - Release amd64 (20180426)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
MachineType: IBM IBM System x -[794582G]-
Package: linux (not installed)
PciMultimedia:

ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=de_DE.UTF-8
 SHELL=/bin/bash
ProcFB: 0 mgadrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.15.0-23-generic root=UUID=a41b3020-8359-4fe2-8fc3-1c97011f71ec ro
ProcVersionSignature: Ubuntu 4.15.0-23.25-generic 4.15.18
RelatedPackageVersions:
 linux-restricted-modules-4.15.0-23-generic N/A
 linux-backports-modules-4.15.0-23-generic N/A
 linux-firmware 1.173.1
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
Tags: bionic
Uname: Linux 4.15.0-23-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:

_MarkForUpload: True
dmi.bios.date: 05/07/2014
dmi.bios.vendor: IBM Corp.
dmi.bios.version: -[D6E162AUS-1.20]-
dmi.board.asset.tag: (none)
dmi.board.name: 00D4062
dmi.board.vendor: IBM
dmi.board.version: (none)
dmi.chassis.asset.tag: none
dmi.chassis.type: 23
dmi.chassis.vendor: IBM
dmi.chassis.version: none
dmi.modalias: dmi:bvnIBMCorp.:bvr-[D6E162AUS-1.20]-:bd05/07/2014:svnIBM:pnIBMSystemx-[794582G]-:pvr00:rvnIBM:rn00D4062:rvr(none):cvnIBM:ct23:cvrnone:
dmi.product.family: System x
dmi.product.name: IBM System x -[794582G]-
dmi.product.version: 00
dmi.sys.vendor: IBM

Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1782709

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
tags: added: bionic
Revision history for this message
Uli.Heller (uli-heller) wrote :

root@samson:~# apport-collect 1782709
ERROR: The python3-launchpadlib package is not installed. This functionality is not available.

Do I really have to install this package?

Changed in linux (Ubuntu):
status: Incomplete → Confirmed
tags: added: apport-collected
description: updated
Revision history for this message
Uli.Heller (uli-heller) wrote : CRDA.txt

apport information

Revision history for this message
Uli.Heller (uli-heller) wrote : CurrentDmesg.txt

apport information

Revision history for this message
Uli.Heller (uli-heller) wrote : Lspci.txt

apport information

Revision history for this message
Uli.Heller (uli-heller) wrote : Lsusb.txt

apport information

Revision history for this message
Uli.Heller (uli-heller) wrote : ProcCpuinfo.txt

apport information

Revision history for this message
Uli.Heller (uli-heller) wrote : ProcCpuinfoMinimal.txt

apport information

Revision history for this message
Uli.Heller (uli-heller) wrote : ProcInterrupts.txt

apport information

Revision history for this message
Uli.Heller (uli-heller) wrote : ProcModules.txt

apport information

Revision history for this message
Uli.Heller (uli-heller) wrote : UdevDb.txt

apport information

Revision history for this message
Uli.Heller (uli-heller) wrote : WifiSyslog.txt

apport information

Revision history for this message
Uli.Heller (uli-heller) wrote :

I examined /var/log/dpkg.log. There is no pause in there. The complete upgrade process looks pretty normal to my. So I guess it is just a network configuration issue during the upgrade that prevented my terminal session from getting updated.

Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

Do you have a way to reproduce this issue, or was it a one time event?

Changed in linux (Ubuntu):
status: Confirmed → Incomplete
importance: Undecided → Medium
Revision history for this message
Uli.Heller (uli-heller) wrote :
Download full text (7.2 KiB)

Today, just another "apt-get upgrade":

root@samson:~# apt-get upgrade
Paketlisten werden gelesen... Fertig
Abhängigkeitsbaum wird aufgebaut.
Statusinformationen werden eingelesen.... Fertig
Paketaktualisierung (Upgrade) wird berechnet... Fertig
Die folgenden Pakete werden aktualisiert (Upgrade):
  base-files console-setup console-setup-linux grub-common grub-efi-amd64 grub-efi-amd64-bin grub-efi-amd64-signed grub2-common keyboard-configuration
  libnss-systemd libpam-systemd libperl5.26 libsystemd0 libudev1 mdadm perl perl-base perl-modules-5.26 python3-cryptography python3-software-properties
  python3-update-manager screen snapd software-properties-common systemd systemd-sysv udev unattended-upgrades update-manager-core
29 aktualisiert, 0 neu installiert, 0 zu entfernen und 0 nicht aktualisiert.
Es müssen 30,9 MB an Archiven heruntergeladen werden.
Nach dieser Operation werden 176 kB Plattenplatz zusätzlich benutzt.
Möchten Sie fortfahren? [J/n]
Holen:1 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 base-files amd64 10.1ubuntu2.1 [58,0 kB]
Holen:2 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libperl5.26 amd64 5.26.1-6ubuntu0.2 [3.531 kB]
Holen:3 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 perl amd64 5.26.1-6ubuntu0.2 [201 kB]
Holen:4 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 perl-base amd64 5.26.1-6ubuntu0.2 [1.390 kB]
Holen:5 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 perl-modules-5.26 all 5.26.1-6ubuntu0.2 [2.762 kB]
Holen:6 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnss-systemd amd64 237-3ubuntu10.3 [105 kB]
Holen:7 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsystemd0 amd64 237-3ubuntu10.3 [204 kB]
Holen:8 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpam-systemd amd64 237-3ubuntu10.3 [108 kB]
Holen:9 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 systemd amd64 237-3ubuntu10.3 [2.896 kB]
Holen:10 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 udev amd64 237-3ubuntu10.3 [1.100 kB]
Holen:11 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libudev1 amd64 237-3ubuntu10.3 [53,8 kB]
Holen:12 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 systemd-sysv amd64 237-3ubuntu10.3 [11,7 kB]
Holen:13 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 console-setup-linux all 1.178ubuntu2.3 [982 kB]
Holen:14 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 console-setup all 1.178ubuntu2.3 [105 kB]
Holen:15 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 keyboard-configuration all 1.178ubuntu2.3 [372 kB]
Holen:16 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-update-manager all 1:18.04.11.4 [34,1 kB]
Holen:17 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 update-manager-core all 1:18.04.11.4 [8.500 B]
Holen:18 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 grub-efi-amd64 amd64 2.02-2ubuntu8.2 [46,8 kB]
Holen:19 http://de.archive.ubuntu.com/ubuntu bionic-updates/main amd64 grub2-common amd64 2.02-2ubuntu8.2 [531 kB]
Holen:20 http://de.archive.ubuntu.com/...

Read more...

Revision history for this message
Shelby Merrick (forkineye) wrote :

I recently installed a fresh system with 18.04 as well and have experienced this exact issue when attempting to "apt dist-upgrade" from ssh. In the two instances where systemd was being updated (a few weeks ago and today), my ssh connection was dropped and I'm unable to re-connect to the server. I started the upgrade from a screen session this time and will investigate further when I can get to the console this evening.

Revision history for this message
DEXTER (mydexterid) wrote :
Download full text (7.6 KiB)

Just happened to me too:
root@gw:/etc# apt upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  eatmydata libeatmydata1 python3-blinker python3-jinja2 python3-json-pointer python3-jsonpatch python3-jsonschema python3-jwt python3-markupsafe python3-oauthlib
Use 'apt autoremove' to remove them.
The following packages will be upgraded:
  apport busybox-initramfs busybox-static docker.io initramfs-tools initramfs-tools-bin initramfs-tools-core landscape-common libidn11 libnss-systemd libpam-modules libpam-modules-bin libpam-runtime libpam-systemd libpam0g libpci3 libseccomp2 libsystemd0
  libudev1 libx11-6 libx11-data libxcb1 pciutils python3-apport python3-gi python3-problem-report systemd systemd-sysv udev unattended-upgrades
30 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 54.0 MB of archives.
After this operation, 37.2 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpam0g amd64 1.1.8-3.6ubuntu2.18.04.1 [55.1 kB]
Get:2 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpam-modules-bin amd64 1.1.8-3.6ubuntu2.18.04.1 [36.7 kB]
Get:3 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpam-modules amd64 1.1.8-3.6ubuntu2.18.04.1 [242 kB]
Get:4 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnss-systemd amd64 237-3ubuntu10.15 [105 kB]
Get:5 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsystemd0 amd64 237-3ubuntu10.15 [205 kB]
Get:6 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpam-systemd amd64 237-3ubuntu10.15 [108 kB]
Get:7 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 systemd amd64 237-3ubuntu10.15 [2,899 kB]
Get:8 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 udev amd64 237-3ubuntu10.15 [1,100 kB]
Get:9 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libudev1 amd64 237-3ubuntu10.15 [54.2 kB]
Get:10 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpam-runtime all 1.1.8-3.6ubuntu2.18.04.1 [37.1 kB]
Get:11 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 initramfs-tools all 0.130ubuntu3.7 [9,592 B]
Get:12 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 initramfs-tools-core all 0.130ubuntu3.7 [48.1 kB]
Get:13 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 initramfs-tools-bin amd64 0.130ubuntu3.7 [12.8 kB]
Get:14 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 busybox-initramfs amd64 1:1.27.2-2ubuntu3.1 [166 kB]
Get:15 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 systemd-sysv amd64 237-3ubuntu10.15 [12.0 kB]
Get:16 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libseccomp2 amd64 2.3.1-2.1ubuntu4.1 [39.1 kB]
Get:17 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libidn11 amd64 1.33-2.1ubuntu1.1 [45.9 kB]
Get:18 http://hu.archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-gi amd64 3.26.1-2ubuntu1 [153 kB]
Get:19 http://hu.archiv...

Read more...

Revision history for this message
DEXTER (mydexterid) wrote :

And logging in on the console the apt process was waiting on for some answers:
ps aux | grep apt
root 13335 0.1 6.6 116500 67388 pts/1 S+ 19:30 0:00 apt upgrade
root 15048 0.0 0.3 11856 3628 pts/3 S+ 19:31 0:00 /bin/bash /usr/bin/ucf --three-way --debconf-ok /usr/share/unattended-upgrades/20auto-upgrades-disabled /etc/apt/apt.conf.d/20auto-upgrades
root 15090 0.0 0.4 24460 4976 pts/3 S+ 19:31 0:00 whiptail --backtitle Package configuration --title Configuring unattended-upgrades --output-fd 11 --nocancel --default-item keep the local version currently installed --menu A new version (/usr/share/unattended-upgrades/20auto-upgrades-disabled) of configuration file /etc/apt/apt.conf.d/20auto-upgrades is available, but the version installed currently has been locally modified. What do you want to do about modified configuration file 20auto-upgrades? 17 211 7 -- install the package maintainer's version keep the local version currently installed show the differences between the versions show a side-by-side difference between the versions show a 3-way difference between available versions do a 3-way merge between available versions start a new shell to examine the situation
root 16771 0.0 0.1 13136 1148 tty1 S+ 19:41 0:00 grep --color=auto apt

Revision history for this message
DEXTER (mydexterid) wrote :

I could only resolve the ssh issue with killing the apt process, and restarted networking.
This is a pretty bad issue imho. Upgrading packages through ssh should not disconnect the session, but even if it is, upgrading the systemd package should not result in not being able to log back in ever again!

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in systemd (Ubuntu):
status: New → Confirmed
Revision history for this message
DEXTER (mydexterid) wrote :

Also I see this in the logs, but maybe not related (why can't I edit my comments?)

Mar 17 19:31:55 localhost systemd-networkd-wait-online[14189]: Event loop failed: Connection timed out
Mar 17 19:31:55 localhost apt-helper[14171]: E: Sub-process /lib/systemd/systemd-networkd-wait-online returned an error code (1)
Mar 17 19:31:55 localhost systemd[1]: apt-daily.service: Control process exited, code=exited status=100
Mar 17 19:31:55 localhost systemd[1]: apt-daily.service: Failed with result 'exit-code'.
Mar 17 19:31:55 localhost systemd[1]: Failed to start Daily apt download activities.

Revision history for this message
DEXTER (mydexterid) wrote :

Some more info:
 - The machine is a Virtualbox ubuntu bionic, and I have a snapshot before the upgrade, so I can reproduce it
 - I'm using ifconfig instead of netplan, because I need a ppp interface (for pppoe)
 - I have an /etc/network/interfaces file
 - it seems that the interface through which I'm using ssh loses it's IP address (defined in the interfaces file as static)
 - I also did these after installing bionic:
"systemctl disable systemd-networkd-wait-online.service
systemctl mask systemd-networkd-wait-online.service" because it was holding up the boot process for a long period

Revision history for this message
rjr162 (ronrossman) wrote :

I'd like to also confirm this bug, which we've been experiencing for the past few months since we first setup an 18.04 image for our deployment portal.

Initially I thought the cause MAY have been from us using ifupdown (we have to as our vCenter instance hasn't been upgraded to 6.5.x or 6.7 where Ubuntu 18.04 with NetPlan is officially supported).

I've checked logs and it seems that *every time* apt upgrade is ran and one of the systemd components are upgraded, the network dies.

You can do "ip addr" and see the interface has no addresses assigned.
A simple restart of networking and the IPs come back. This literally just happened this morning to two of our 18.04 LTS systems and the systemd packages that were updated were:

libsystemd0
systemd-sysv
libpam-systemd
systemd
libnss-systemd

The rest of the packages that were updated:
grub-common
grub2-common
udev
grub-pc
apache2-data
libudev1
grub-pc-bin
apache2-bin
libunistring2
apache2
apache2-utils

I also had another report on Feb 20th, and it also involved the following updates:
Start-Date: 2019-02-20 04:41:09
Commandline: /usr/bin/apt-get -o quiet=1 upgrade -y -o APT::Get::Show-Upgraded=true
Upgrade: libsystemd0:amd64 (237-3ubuntu10.12, 237-3ubuntu10.13), udev:amd64 (237-3ubuntu10.12, 237-3ubuntu10.13), libudev1:amd64 (237-3ubuntu10.12, 237-3ubuntu10.13), systemd-sysv:amd64 (237-3ubuntu10.12, 237-3ubuntu10.13), libpam-systemd:amd64 (237-3ubuntu10.12, 237-3ubuntu10.13), systemd:amd64 (237-3ubuntu10.12, 237-3ubuntu10.13), libnss-systemd:amd64 (237-3ubuntu10.12, 237-3ubuntu10.13), pciutils:amd64 (1:3.5.2-1ubuntu1, 1:3.5.2-1ubuntu1.1), libpci3:amd64 (1:3.5.2-1ubuntu1, 1:3.5.2-1ubuntu1.1)
End-Date: 2019-02-20 04:42:19

The interesting thing is, looking at my notes, in my case it MAY BE from the ifupdown which should "just work", as this 2nd VM also has that setup. I looked and it updated this morning the same as the first machine but I didn't have a report of it being offline. Now on this 2nd machine, I did also do an "apt remove netplan.io" as a "hope this solves the issue and it's some conflict or issue between netplan.io and ifupdown OR a conflict between systemd and netplan.io and since there's no report of it being down and I just verified networking is fine.... is it something between systemd and netplan?

Revision history for this message
James Gray (james-1) wrote :

Can confirm the same behaviour on my systems too. Updating systemd recently caused the entire routing table to be dropped (I had console access and was able to confirm no routes - not even loopback worked). In normal operation I see:

    >ip route show
    default via 10.0.0.1 dev eno1 src 10.0.0.60 metric 202
    10.0.0.0/24 dev eno1 proto dhcp scope link src 10.0.0.60 metric 202

However, when systemd is updated, the route table is blank although interfaces maintain their addresses etc. When working remotely, I need to get remote hands to reboot the system to restore normal functionality which obviously extends down time. We have implemented procedures to watch the route table and if it disappears, triggers a reboot but this is problematic as it could happen before the update process is finished.

Brad Figg (brad-figg)
tags: added: ubuntu-certified
Revision history for this message
Druggo Yang (druggo) wrote :

same issue, upgrade systemd cause network outage about 5 second,

syslog show as below:

Dec 18 10:03:29 clark systemd[1]: Stopped Wait for Network to be Configured.
Dec 18 10:03:29 clark systemd[1]: Stopping Wait for Network to be Configured...
Dec 18 10:03:29 clark systemd[1]: Stopping Network Service...
Dec 18 10:03:29 clark systemd[1]: Stopped Network Service.
Dec 18 10:03:29 clark systemd[1]: Starting Network Service...
Dec 18 10:03:29 clark systemd[1]: Started Network Service.
Dec 18 10:03:29 clark systemd[1]: Stopping Network Name Resolution...
Dec 18 10:03:29 clark systemd[1]: Starting Wait for Network to be Configured...

Dec 18 10:03:28 clark kernel: [2977260.887987] printk: systemd: 35 output lines suppressed due to ratelimiting
Dec 18 10:03:29 clark kernel: [2977261.277470] br0: port 1(eno1) entered disabled state
Dec 18 10:03:32 clark kernel: [2977264.881553] tg3 0000:01:00.0 eno1: Link is up at 1000 Mbps, full duplex
Dec 18 10:03:32 clark kernel: [2977264.881593] tg3 0000:01:00.0 eno1: Flow control is off for TX and off for RX
Dec 18 10:03:32 clark kernel: [2977264.881602] tg3 0000:01:00.0 eno1: EEE is disabled
Dec 18 10:03:32 clark kernel: [2977264.881667] br0: port 1(eno1) entered blocking state
Dec 18 10:03:32 clark kernel: [2977264.881699] br0: port 1(eno1) entered forwarding state

Revision history for this message
Brandon Jackson (bluegman991) wrote :

Having the exact same issue with ubuntu 20.04. Has anyone found a work around for this rather annoying issue?

Revision history for this message
DEXTER (mydexterid) wrote :

I have:

apt-get install ifupdown

systemctl stop systemd-networkd.socket systemd-networkd networkd-dispatcher systemd-network-wait-online
systemctl disable systemd-networkd.socket systemd-networkd networkd-dispatcher systemd-network-wait-online
systemctl mask systemd-networkd.socket systemd-networkd networkd-dispatcher systemd-network-wait-online

best.

Revision history for this message
Dan Streetman (ddstreet) wrote :

please reopen if this is still an issue

Changed in systemd (Ubuntu):
status: Confirmed → Invalid
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]

Changed in linux (Ubuntu):
status: Incomplete → Expired
Revision history for this message
Dominic Raferd (dominic-timedicer) wrote :

I changed my networking setup to use netplan and the problem disappeared.

Revision history for this message
Force (force000) wrote :

I have the same problem and every time I upgrade systemd, it kills my network access to the server. I therefore had to disable automatic updates and always run a shutdown -r 10 command before upgrading

Revision history for this message
Dominic Raferd (dominic-timedicer) wrote (last edit ):

Despite my comment above, we continue to have this problem when using netplan. We are running Ubuntu 22.04 Server, fully updated. See below the /etc/netplan/config.yaml (external IP quatrains have been changed) (sorry launchpad loses all indents when you edit!). The setup is that enp8s1 provides internet access (with metric 49) and enp7s0 provides local lan access *and* (with metric 51) fallback internet access:

network:
  version: 2
  renderer: networkd
  ethernets:
    enp8s1:
      dhcp4: false
      wakeonlan: true
      addresses: [111.94.100.80/24]
      nameservers:
        addresses: [111.94.100.1]
      # makes this the default route - deprecated
      #gateway4: 111.94.100.1
      link-local: []
      routes:
      - to: default
        via: 111.94.100.1
        metric: 49
        on-link: true
    enp7s0:
      dhcp4: false
      wakeonlan: true
      addresses: [192.168.20.88/24]
      nameservers:
        addresses: [192.168.20.1]
      link-local: []
      routes:
      - to: default
        via: 192.168.20.1
        metric: 51
        on-link: true
        # to avoid conflict with enp8s1 default route, but makes no difference
        #table: 76
      - to: 192.168.20.0/24
        via: 192.168.20.1
        metric: 48

When working properly:
# ip route show to default
default via 111.94.100.1 dev enp8s1 proto static metric 49 onlink
default via 192.168.20.1 dev enp7s0 proto static metric 51 onlink

But after systemd reconfiguration we have two default routes with a metric of 0! We have to delete one of them to restore the correct setup (as above) and get internet access via enp8s1 again. My workaround is continuous monitoring of syslog for a line indicating that the network has been (re)configured by systemd and then running some bash code like this (extract):

if [[ -n $(ip route show to default metric 0 via 192.168.20.1) ]]; then
  NUM_DEFAULT_ROUTES=$(ip route show to default|wc -l)
  if [[ $NUM_DEFAULT_ROUTES -gt 2 ]]; then
    ip route delete to default metric 0 via 192.168.20.1
  fi
fi

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.