Ubuntu
systemd package

Update systemd for ubuntu 18.04 with fix for interaction between OnFailure= and Restart=

Bug #1849261 reported by bugproxy
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
High
Canonical Foundations Team
systemd (Ubuntu)
Fix Released
Undecided
Skipper Bug Screeners
Xenial
Won't Fix
Medium
Unassigned
Bionic
Fix Released
Medium
Dan Streetman
Disco
Fix Released
Undecided
Unassigned
Eoan
Fix Released
Undecided
Unassigned

Bug Description

[impact]

services configured with both OnFailure= and Restart= directives will both start the failure service(s), as well as restarting itself. The documentation (and later versions) indicate that OnFailure= services will only be started once the service actually does stop (due to failure) once its restart limit has been exceeded.

[test case]

create service files as:

ubuntu@lp1849261-b:/etc/systemd/system$ cat test.service
[Unit]
OnFailure=failure.service

[Service]
ExecStart=/bin/sh -c '/bin/sleep 10; /bin/false'
Restart=always

ubuntu@lp1849261-b:/etc/systemd/system$ cat failure.service
[Service]
ExecStart=/bin/true

perform daemon-reload, and then start test service:

ubuntu@lp1849261-b:/etc/systemd/system$ sudo systemctl daemon-reload
ubuntu@lp1849261-b:/etc/systemd/system$ sudo systemctl start test

wait for a while for the test service to exit - at least 10 seconds, as that's what we configured in the service, although I have waited longer in this example. Then check the journal for both test and failure services:

ubuntu@lp1849261-b:/etc/systemd/system$ journalctl -b -u test
-- Logs begin at Fri 2019-12-13 00:52:50 UTC, end at Fri 2019-12-13 01:18:34 UTC. --
Dec 13 01:17:43 lp1849261-b systemd[1]: Started test.service.
Dec 13 01:17:53 lp1849261-b systemd[1]: test.service: Main process exited, code=exited, status=1/FAILURE
Dec 13 01:17:53 lp1849261-b systemd[1]: test.service: Failed with result 'exit-code'.
Dec 13 01:17:53 lp1849261-b systemd[1]: test.service: Triggering OnFailure= dependencies.
Dec 13 01:17:53 lp1849261-b systemd[1]: test.service: Service hold-off time over, scheduling restart.
Dec 13 01:17:53 lp1849261-b systemd[1]: test.service: Scheduled restart job, restart counter is at 1.
Dec 13 01:17:53 lp1849261-b systemd[1]: Stopped test.service.
Dec 13 01:17:53 lp1849261-b systemd[1]: Started test.service.
Dec 13 01:18:03 lp1849261-b systemd[1]: test.service: Main process exited, code=exited, status=1/FAILURE
Dec 13 01:18:03 lp1849261-b systemd[1]: test.service: Failed with result 'exit-code'.
Dec 13 01:18:03 lp1849261-b systemd[1]: test.service: Triggering OnFailure= dependencies.
Dec 13 01:18:04 lp1849261-b systemd[1]: test.service: Service hold-off time over, scheduling restart.
Dec 13 01:18:04 lp1849261-b systemd[1]: test.service: Scheduled restart job, restart counter is at 2.
Dec 13 01:18:04 lp1849261-b systemd[1]: Stopped test.service.
Dec 13 01:18:04 lp1849261-b systemd[1]: Started test.service.
Dec 13 01:18:14 lp1849261-b systemd[1]: test.service: Main process exited, code=exited, status=1/FAILURE
Dec 13 01:18:14 lp1849261-b systemd[1]: test.service: Failed with result 'exit-code'.
Dec 13 01:18:14 lp1849261-b systemd[1]: test.service: Triggering OnFailure= dependencies.
Dec 13 01:18:14 lp1849261-b systemd[1]: test.service: Service hold-off time over, scheduling restart.
Dec 13 01:18:14 lp1849261-b systemd[1]: test.service: Scheduled restart job, restart counter is at 3.
Dec 13 01:18:14 lp1849261-b systemd[1]: Stopped test.service.
Dec 13 01:18:14 lp1849261-b systemd[1]: Started test.service.
Dec 13 01:18:24 lp1849261-b systemd[1]: test.service: Main process exited, code=exited, status=1/FAILURE
Dec 13 01:18:24 lp1849261-b systemd[1]: test.service: Failed with result 'exit-code'.
Dec 13 01:18:24 lp1849261-b systemd[1]: test.service: Triggering OnFailure= dependencies.
Dec 13 01:18:24 lp1849261-b systemd[1]: test.service: Service hold-off time over, scheduling restart.
Dec 13 01:18:24 lp1849261-b systemd[1]: test.service: Scheduled restart job, restart counter is at 4.
Dec 13 01:18:24 lp1849261-b systemd[1]: Stopped test.service.
Dec 13 01:18:24 lp1849261-b systemd[1]: Started test.service.
Dec 13 01:18:34 lp1849261-b systemd[1]: test.service: Main process exited, code=exited, status=1/FAILURE
Dec 13 01:18:34 lp1849261-b systemd[1]: test.service: Failed with result 'exit-code'.
Dec 13 01:18:34 lp1849261-b systemd[1]: test.service: Triggering OnFailure= dependencies.
Dec 13 01:18:34 lp1849261-b systemd[1]: test.service: Service hold-off time over, scheduling restart.
Dec 13 01:18:34 lp1849261-b systemd[1]: test.service: Scheduled restart job, restart counter is at 5.
Dec 13 01:18:34 lp1849261-b systemd[1]: Stopped test.service.
Dec 13 01:18:34 lp1849261-b systemd[1]: Started test.service.

ubuntu@lp1849261-b:/etc/systemd/system$ journalctl -b -u failure
-- Logs begin at Fri 2019-12-13 00:52:50 UTC, end at Fri 2019-12-13 01:19:15 UTC. --
Dec 13 01:17:53 lp1849261-b systemd[1]: Started failure.service.
Dec 13 01:18:03 lp1849261-b systemd[1]: Started failure.service.
Dec 13 01:18:14 lp1849261-b systemd[1]: Started failure.service.
Dec 13 01:18:24 lp1849261-b systemd[1]: Started failure.service.
Dec 13 01:18:34 lp1849261-b systemd[1]: Started failure.service.
Dec 13 01:18:44 lp1849261-b systemd[1]: Started failure.service.
Dec 13 01:18:55 lp1849261-b systemd[1]: Started failure.service.
Dec 13 01:19:05 lp1849261-b systemd[1]: Started failure.service.
Dec 13 01:19:15 lp1849261-b systemd[1]: Started failure.service.

note that the 'failure' service should not have been started at all.

[regression potential]

this changes how service failures are handled, specifically when OnFailure= services are started. However, this changes the behavior to be in line with the documentation, and the current behavior is unlikely to be anyone's desired behavior - both starting the failure service *and* restarting the failed service. However, this will cause a regression for anyone *expecting* both the OnFailure= service(s) to be started, as well as the failing service to be restarted.

Any other regression would likely occur during the state change of a unit, most likely during stop/exit of a service.

[scope]

This is needed only for Bionic and Xenial (but marking WontFix for Xenial).

The patches to fix this:
7f66b026bbdda6bd3ddf7f617bf2c3a475e4a665
2ad2e41a72ec19159c0746a78e15ff880fe32a63
are included already in Disco and later.

For Xenial, I'm marking this as WontFix because it has additional backporting requirements; at least commit deb4e7080db and possibly more. Without anyone specifically requesting this for Xenial, I think the possibility for regression in Xenial due to the larger backport is too high to proactively apply it.

[other info]

original description:
---

Within Ubunutu 18.04 systemd 237-3ubuntu10.31: amd64 arm64 armhf i386 ppc64el s390x is available,

The pull request for 240 is now requested to backport to 237 for Ubuntu 18.04, which is integrated within the IBM product.

Pull request: https://github.com/systemd/systemd/pull/9158

See original description
bugproxy (bugproxy)
tags: added: architecture-s39064 bugnameltc-182011 severity-high targetmilestone-inin1804
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → systemd (Ubuntu)
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
importance: Undecided → High
assignee: nobody → Canonical Foundations Team (canonical-foundations)
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

We will not be upgrading systemd to a new upstream release in 18.04.

W.r.t.
https://github.com/systemd/systemd/pull/9158
https://github.com/systemd/systemd/issues/8398

I thought we tried to backport that before, let me ping a few people if that was the case or not.

Balint Reczey (rbalint)
Changed in systemd (Ubuntu):
status: New → Fix Released
Changed in systemd (Ubuntu Eoan):
status: New → Fix Released
Changed in systemd (Ubuntu Disco):
status: New → Fix Released
Steve Langasek (vorlon)
summary: - Update systemd for ubuntu 18.04 with requested fix.
+ Update systemd for ubuntu 18.04 with fix for interaction between
+ OnFailure= and Restart=
description: updated
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: New → Triaged
Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Bionic):
status: New → In Progress
assignee: nobody → Dan Streetman (ddstreet)
importance: Undecided → Medium
tags: added: bionic ddstreet systemd
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Triaged → In Progress
Dan Streetman (ddstreet)
description: updated
Changed in systemd (Ubuntu Xenial):
status: New → In Progress
assignee: nobody → Dan Streetman (ddstreet)
importance: Undecided → Medium
tags: added: xenial
Dan Streetman (ddstreet)
description: updated
tags: removed: xenial
Changed in systemd (Ubuntu Xenial):
status: In Progress → Won't Fix
assignee: Dan Streetman (ddstreet) → nobody
Francis Ginther (fginther)
tags: added: id-5de127a738dabd05006e38e8
Dimitri John Ledkov (xnox)
Changed in systemd (Ubuntu Bionic):
status: In Progress → Triaged
Changed in ubuntu-z-systems:
status: In Progress → Triaged
Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Bionic):
status: Triaged → In Progress
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Triaged → In Progress
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.34 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in systemd (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-bionic
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (systemd/237-3ubuntu10.34)

All autopkgtests for the newly accepted systemd (237-3ubuntu10.34) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

php7.2/7.2.24-0ubuntu0.18.04.2 (armhf)
openssh/1:7.6p1-4ubuntu0.3 (arm64, armhf, ppc64el, amd64, s390x, i386)
dovecot/1:2.2.33.2-1ubuntu4.5 (armhf)
gvfs/1.36.1-0ubuntu1.3.3 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

This SRU needs to be reuploaded, due to security update that trumped this in progress SRU.

Changed in systemd (Ubuntu Bionic):
status: Fix Committed → In Progress
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.39 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in systemd (Ubuntu Bionic):
status: In Progress → Fix Committed
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (systemd/237-3ubuntu10.39)

All autopkgtests for the newly accepted systemd (237-3ubuntu10.39) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

php7.2/7.2.24-0ubuntu0.18.04.2 (armhf)
gvfs/1.36.1-0ubuntu1.3.3 (ppc64el)
lxc/3.0.3-0ubuntu1~18.04.1 (amd64)
systemd/237-3ubuntu10.39 (i386)
netplan.io/0.98-0ubuntu1~18.04.1 (i386, amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Dan Streetman (ddstreet) wrote :
Download full text (4.4 KiB)

bionic verification:

root@lp1849261-b:~# dpkg -l systemd | grep ii
ii systemd 237-3ubuntu10.39 amd64 system and service manager
root@lp1849261-b:~# systemctl status test failure
● test.service
   Loaded: loaded (/etc/systemd/system/test.service; static; vendor preset: enab
   Active: inactive (dead)

● failure.service
   Loaded: loaded (/etc/systemd/system/failure.service; static; vendor preset: e
   Active: inactive (dead)
root@lp1849261-b:~# systemctl start test
root@lp1849261-b:~# sleep 10 ; systemctl status test failure
● test.service
   Loaded: loaded (/etc/systemd/system/test.service; static; vendor preset: enab
   Active: active (running) since Sun 2020-02-16 14:42:27 UTC; 2s ago
 Main PID: 1338 (sh)
    Tasks: 2 (limit: 4702)
   CGroup: /system.slice/test.service
           ├─1338 /bin/sh -c /bin/sleep 10; /bin/false
           └─1342 /bin/sleep 10

Feb 16 14:42:27 lp1849261-b systemd[1]: Started test.service.

● failure.service
   Loaded: loaded (/etc/systemd/system/failure.service; static; vendor preset: e
   Active: inactive (dead)
root@lp1849261-b:~# journalctl -b -u test --no-pager
-- Logs begin at Fri 2019-12-13 00:52:50 UTC, end at Sun 2020-02-16 14:43:08 UTC. --
Feb 16 14:42:06 lp1849261-b systemd[1]: Started test.service.
Feb 16 14:42:16 lp1849261-b systemd[1]: test.service: Main process exited, code=exited, status=1/FAILURE
Feb 16 14:42:16 lp1849261-b systemd[1]: test.service: Failed with result 'exit-code'.
Feb 16 14:42:16 lp1849261-b systemd[1]: test.service: Service hold-off time over, scheduling restart.
Feb 16 14:42:16 lp1849261-b systemd[1]: test.service: Scheduled restart job, restart counter is at 1.
Feb 16 14:42:16 lp1849261-b systemd[1]: Stopped test.service.
Feb 16 14:42:16 lp1849261-b systemd[1]: Started test.service.
Feb 16 14:42:26 lp1849261-b systemd[1]: test.service: Main process exited, code=exited, status=1/FAILURE
Feb 16 14:42:26 lp1849261-b systemd[1]: test.service: Failed with result 'exit-code'.
Feb 16 14:42:27 lp1849261-b systemd[1]: test.service: Service hold-off time over, scheduling restart.
Feb 16 14:42:27 lp1849261-b systemd[1]: test.service: Scheduled restart job, restart counter is at 2.
Feb 16 14:42:27 lp1849261-b systemd[1]: Stopped test.service.
Feb 16 14:42:27 lp1849261-b systemd[1]: Started test.service.
Feb 16 14:42:37 lp1849261-b systemd[1]: test.service: Main process exited, code=exited, status=1/FAILURE
Feb 16 14:42:37 lp1849261-b systemd[1]: test.service: Failed with result 'exit-code'.
Feb 16 14:42:37 lp1849261-b systemd[1]: test.service: Service hold-off time over, scheduling restart.
Feb 16 14:42:37 lp1849261-b systemd[1]: test.service: Scheduled restart job, restart counter is at 3.
Feb 16 14:42:37 lp1849261-b systemd[1]: Stopped test.service.
Feb 16 14:42:37 lp1849261-b systemd[1]: Started test.service.
Feb 16 14:42:47 lp1849261-b systemd[1]: test.service: Main process exited, code=exited, status=1/FAILURE
Feb 16 14:42:47 lp1849261-b systemd[1]: test.service: Failed with result 'exit-code'.
Feb 16 14:42:47 lp1849261-b systemd[1]: test.service: Service hold-off time over, scheduling restart.
Feb 16 14:42:47 lp1849261-b systemd[1]: test.service: Scheduled restart job...

Read more...

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for systemd has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 237-3ubuntu10.39

---------------
systemd (237-3ubuntu10.39) bionic; urgency=medium

  [ Dariusz Gadomski ]
  * d/p/lp1762391/0001-Call-getgroups-to-know-size-of-supplementary-groups-.patch,
    d/p/lp1762391/0002-user-util-tweak-to-in_gid.patch,
    d/p/lp1762391/0003-user-util-Add-helper-functions-for-gid-lists-operati.patch,
    d/p/lp1762391/0004-execute-Restore-call-to-pam_setcred.patch,
    d/p/lp1762391/0005-execute-Detect-groups-added-by-PAM-and-merge-them-wi.patch,
    d/p/lp1762391/0006-test-Add-tests-for-gid-list-ops.patch,
    d/p/lp1762391/0007-execute-add-const-to-array-parameters-where-possible.patch,
    d/p/lp1762391/0008-execute-allow-pam_setcred-to-fail-ignore-errors.patch:
    - Restore call to pam_setcred (LP: #1762391)

  [ Ioanna Alifieraki ]
  * d/p/lp1860548/0001-Revert-Replace-use-of-snprintf-with-xsprintf.patch,
    d/p/lp1860548/0002-job-truncate-unit-description.patch:
    - use snprintf instead of xsprintf (LP: #1860548)

  [ Dan Streetman ]
  * d/p/lp1833193-network-update-address-when-static-address-was-alrea.patch:
    - Update lft when static addr was cfg by dhcp (LP: #1833193)
  * d/p/lp1849261/0001-core-when-we-can-t-enqueue-OnFailure-job-show-full-e.patch,
    d/p/lp1849261/0002-core-don-t-trigger-OnFailure-deps-when-a-unit-is-goi.patch:
    - Only trigger OnFailure= if Restart= is not in effect (LP: #1849261)
  * d/p/lp1671951-network-set-ipv6-mtu-after-link-up-or-device-mtu-cha.patch:
    - set ipv6 mtu at correct time (LP: #1671951)
  * d/p/lp1845909/0001-networkd-honour-LinkLocalAddressing.patch,
    d/p/lp1845909/0002-networkd-fix-link_up-12505.patch,
    d/p/lp1845909/0003-network-do-not-send-ipv6-token-to-kernel.patch,
    d/p/lp1845909/0004-network-rename-linux_configure_after_setting_mtu-to-linux.patch,
    d/p/lp1845909/0005-network-add-link-setting_genmode-flag.patch,
    d/p/lp1845909/0006-network-if-ipv6ll-is-disabled-enumerate-tentative-ipv6-ad.patch,
    d/p/lp1845909/0007-network-drop-foreign-config-after-addr_gen_mode-has-been-.patch,
    d/p/lp1845909/0008-network-drop-IPv6LL-address-when-LinkLocalAddressing.patch:
    - if LinkLocalAddressing=no prevent creation of ipv6ll (LP: #1845909)
  * d/p/lp1859862-network-Do-not-disable-IPv6-by-writing-to-sysctl.patch:
    - enable ipv6 when needed (LP: #1859862)
  * d/p/lp1836695-networkd-Add-back-static-routes-after-DHCPv4-lease-e.patch:
    - (re)add static routes after getting dhcp4 addr (LP: #1836695)
  * d/t/storage:
    - fix buggy test (LP: #1831459)
    - without scsi_debug, skip test (LP: #1847816)

 -- Dan Streetman <email address hidden> Thu, 06 Feb 2020 10:00:49 -0500

Changed in systemd (Ubuntu Bionic):
status: Fix Committed → Fix Released
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2020-02-26 08:06 EDT-------
IBM Bugzilla status-> closed, Fix Released by Canonical

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.