focal security update 2.34.6-0ubuntu0.20.04.1 cannot be automatically installed due to new dependency

Hello Steve, thanks for the report; can you run a manual:

    sudo apt update && sudo apt upgrade

and report back the apt output, which will give a better idea of what exactly is holding back the upgrade?

Thanks

Unfortunately I've already done that on the two affected machines and didn't make a note of the output. I will try to dig out the dpkg logs. As I said, the extra dependency on libopengl0 seemed to be the issue. It's also just possible I took a snapshot or backup so I can roll back and retry - I will have a look.

I seem to recall one machine had had non-security updates disabled after they had previously been enabled, and I initially suspected that had caused the problem, but then it occurred on another machine where that wasn't true.

OK, here is dpkg.log section from one machine:

2022-03-05 14:45:14 startup archives unpack
2022-03-05 14:45:14 install libopengl0:amd64 <none> 1.3.2-1~ubuntu0.20.04.1
2022-03-05 14:45:14 status triggers-pending libc-bin:amd64 2.31-0ubuntu9.2
2022-03-05 14:45:14 status half-installed libopengl0:amd64 1.3.2-1~ubuntu0.20.04.1
2022-03-05 14:45:14 status unpacked libopengl0:amd64 1.3.2-1~ubuntu0.20.04.1
2022-03-05 14:45:14 upgrade libwebkit2gtk-4.0-37:amd64 2.34.4-0ubuntu0.20.04.1 2.34.6-0ubuntu0.20.04.1
2022-03-05 14:45:14 status half-configured libwebkit2gtk-4.0-37:amd64 2.34.4-0ubuntu0.20.04.1
2022-03-05 14:45:14 status unpacked libwebkit2gtk-4.0-37:amd64 2.34.4-0ubuntu0.20.04.1
2022-03-05 14:45:14 status half-installed libwebkit2gtk-4.0-37:amd64 2.34.4-0ubuntu0.20.04.1
2022-03-05 14:45:17 status unpacked libwebkit2gtk-4.0-37:amd64 2.34.6-0ubuntu0.20.04.1
2022-03-05 14:45:18 upgrade libjavascriptcoregtk-4.0-18:amd64 2.34.4-0ubuntu0.20.04.1 2.34.6-0ubuntu0.20.04.1
2022-03-05 14:45:18 status half-configured libjavascriptcoregtk-4.0-18:amd64 2.34.4-0ubuntu0.20.04.1
2022-03-05 14:45:18 status unpacked libjavascriptcoregtk-4.0-18:amd64 2.34.4-0ubuntu0.20.04.1
2022-03-05 14:45:18 status half-installed libjavascriptcoregtk-4.0-18:amd64 2.34.4-0ubuntu0.20.04.1
2022-03-05 14:45:19 status unpacked libjavascriptcoregtk-4.0-18:amd64 2.34.6-0ubuntu0.20.04.1
2022-03-05 14:45:19 startup packages configure
2022-03-05 14:45:19 configure libjavascriptcoregtk-4.0-18:amd64 2.34.6-0ubuntu0.20.04.1 <none>
2022-03-05 14:45:19 status unpacked libjavascriptcoregtk-4.0-18:amd64 2.34.6-0ubuntu0.20.04.1
2022-03-05 14:45:19 status half-configured libjavascriptcoregtk-4.0-18:amd64 2.34.6-0ubuntu0.20.04.1
2022-03-05 14:45:19 status installed libjavascriptcoregtk-4.0-18:amd64 2.34.6-0ubuntu0.20.04.1
2022-03-05 14:45:19 configure libopengl0:amd64 1.3.2-1~ubuntu0.20.04.1 <none>
2022-03-05 14:45:19 status unpacked libopengl0:amd64 1.3.2-1~ubuntu0.20.04.1
2022-03-05 14:45:19 status half-configured libopengl0:amd64 1.3.2-1~ubuntu0.20.04.1
2022-03-05 14:45:19 status installed libopengl0:amd64 1.3.2-1~ubuntu0.20.04.1
2022-03-05 14:45:19 configure libwebkit2gtk-4.0-37:amd64 2.34.6-0ubuntu0.20.04.1 <none>
2022-03-05 14:45:19 status unpacked libwebkit2gtk-4.0-37:amd64 2.34.6-0ubuntu0.20.04.1
2022-03-05 14:45:19 status half-configured libwebkit2gtk-4.0-37:amd64 2.34.6-0ubuntu0.20.04.1
2022-03-05 14:45:19 status installed libwebkit2gtk-4.0-37:amd64 2.34.6-0ubuntu0.20.04.1
2022-03-05 14:45:19 trigproc libc-bin:amd64 2.31-0ubuntu9.2 <none>
2022-03-05 14:45:19 status half-configured libc-bin:amd64 2.31-0ubuntu9.2
2022-03-05 14:45:19 status installed libc-bin:amd64 2.31-0ubuntu9.2

OK, have manually rolled back the system to previous state (the old versions of the packages were still available on my apt-cacher-ng server), and run unattended-upgrades in debug mode - file attached. I guess the key lines are:

sanity check failed for: {'libjavascriptcoregtk-4.0-18=2.34.6-0ubuntu0.20.04.1', 'libopengl0=1.3.2-1~ubuntu0.20.04.1', 'libwebkit2gtk-4.0-37=2.34.6-0ubuntu0.20.04.1'} : pkg libopengl0 is not in an allowed origin
falling back to adjusting libjavascriptcoregtk-4.0-18's dependencies
sanity check failed for: {'nautilus-share=0.7.3-2ubuntu3', 'libjavascriptcoregtk-4.0-18=2.34.6-0ubuntu0.20.04.1', 'gnome-session-flashback=1:3.36.5-0ubuntu1', 'atril=1.24.0-1', 'gnome-todo=3.28.1-5', 'gnucash=1:3.8b-1ubuntu1', 'gnome-calendar=3.36.2-0ubuntu1', 'xubuntu-desktop=2.233', 'ubuntu-unity-desktop=0.2', 'evolution-data-server=3.36.4-0ubuntu1', 'metacity=1:3.36.1-1', 'update-manager=1:20.04.10.7', 'indicator-bluetooth=0.0.6+17.10.20170605-0ubuntu3', 'libfolks-eds25=0.13.2-1', 'gdm3=3.36.3-0ubuntu0.20.04.3', 'update-notifier=3.192.30.7', 'mutter=3.36.9-0ubuntu0.20.04.1', 'gnome-user-docs=3.36.2+git20200704-0ubuntu0.1', 'yelp=3.36.0-1', 'rhythmbox-plugins=3.4.4-1ubuntu2', 'libedataserverui-1.2-2=3.36.4-0ubuntu1', 'libgoa-backend-1.0-1=3.36.0-1ubuntu1', 'ubuntu-session=3.36.0-2ubuntu1', 'ubuntu-docs=20.04.3', 'gir1.2-webkit2-4.0=2.34.6-0ubuntu0.20.04.1', 'libatrilview3=1.24.0-1', 'unity-control-center=15.04.0+19.10.20190921-0ubuntu3', 'gnome-control-center=1:3.36.5-0ubuntu1', 'zenity=3.32.0-5', 'gnome-online-accounts=3.36.0-1ubuntu1', 'ubuntu-release-upgrader-gtk=1:20.04.33', 'gnome-shell=3.36.7-0ubuntu0.20.04.1', 'apturl=0.5.2ubuntu19', 'shotwell=0.30.10-0ubuntu0.1', 'geary=3.36.1-1', 'libyelp0=3.36.0-1', 'libwebkit2gtk-4.0-37=2.34.6-0ubuntu0.20.04.1'} : pkg libgoa-backend-1.0-1 is marked to be deleted

I'm unclear on exactly how u-a is supposed to work, it's possible this is an algorithmic bug there I suppose?

The new dependency on libopengl0 is expected. The new version of WebKitGTK fixed opengl detection and the new dependency is now required.

I am going to re-assign this bug to unattended-upgrades. If it's not willing to install new dependencies, it definitely should get fixed as some security updates will always get held back.

Digging a bit further - this machine was manually dist-upgraded on 30-May-2021 (it has -updates enabled, but is set to install only security updates automatically.) That update pulled in libglvnd 1.3.2-1~ubuntu0.20.04.1 (source for libegl1, libglvnd0, etc.)

To upgrade to webkit2gtk 2.34.6-0ubuntu0.20.04.1, u-a must install libopengl0, either 1.3.1-1 from the main archive or 1.3.2-1~ubuntu0.20.04.1 from -updates. However, -updates is not a trusted source for u-a when configured like this, so the only candidate is 1.3.1-1. Unfortunately libopengl0 1.3.1-1 depends on libglvnd0=1.3.1-1, but 1.3.2-1~ubuntu0.20.04.1 is already installed, so u-a would have to downgrade it which is I guess a decision well beyond its pay-grade.

I suppose the solution is to find a way to lose the new dependency, obvious answer would be to backport the fixes in webkit2gtk 2.34.6 to 2.34.4? I assume there is some sort of policy on adding new dependencies in security updates?

I suppose there's an argument to be made that if the user is prepared to periodically manually install non-security updates, then they should be prepared to check for held back security updates too. I tend to work from the command-line so don't know what the GUI interface(s) allow and indicate in this scenario.

The other option in u-a might be to split Unattended-Upgrade::Allowed-Origins into "Automatic origins" and "permitted origins", so only packages in the former will be automatically installed, but upgraded dependencies could be pulled from the latter if required?

Thanks for doing the digging to confirm the cause; I suspect unattended-upgrades should be modified to perform something similar to apt upgrade, rather than apt-get upgrade, and bring in new dependencies when necessary. A lot of systems never have interactive users any more.

Thanks

> To upgrade to webkit2gtk 2.34.6-0ubuntu0.20.04.1, u-a must install libopengl0, either 1.3.1-1 from the main archive or 1.3.2-1~ubuntu0.20.04.1 from -updates. However, -updates is not a trusted source for u-a when configured like this, so the only candidate is 1.3.1-1. Unfortunately libopengl0 1.3.1-1 depends on libglvnd0=1.3.1-1, but 1.3.2-1~ubuntu0.20.04.1 is already installed, so u-a would have to downgrade it which is I guess a decision well beyond its pay-grade.

Oh, I believe this is the actually issue here. Unattended-upgrades only updates from the -security pocket, so a security update can't rely on something that is only in -updates.

I will look into pushing the binary packages from libglvnd to the -security pocket so that the upgrade can be handled properly.

I just rebuilt and released libglvnd binaries into the -security pocket. This includes the libopengl0 binary package. This should allow unattended-upgrades to install libopengl0 automatically when pulling in the webkitgtk update.

@mdeslaur: fix seems to be good. I had the login nag about "2 updates could not be installed automatically.", but I ran sudo unattended-upgrade once which cleared also that.

Thanks!