[snap] suggestion: alert users when the snap has been refreshed while running

This is an attempt at implementing this: https://git.launchpad.net/~chromium-team/chromium-browser/+git/snap-from-source/commit/?id=a7dc89ea1996665cb528701c2c23a5e17e5508c2

Implemented in a much more lightweight manner using notify-send: https://git.launchpad.net/~chromium-team/chromium-browser/+git/snap-from-source/commit/?id=82ee1ce51514ee197ee6fd908c9f0af881f1f2ac.

I just noticed hundreds of chromium.launcher processes running on my machine with `sleep 60` and found that it's because of this fix. Looks like this background process is started but nerver stopped? nor does it checks if it's already running. This is especially visible in my case because I'm running automated tests starting chrome using capybara, and it'll be the same for everybody doing integration tests with chrome I guess.

Maybe a good fix would be to check if this background process is already running to not start it again?

https://user-images.githubusercontent.com/201687/79634864-773b1300-816d-11ea-98f5-1a677aca60dd.png

I can confirm this undesirable side effect. The background process that checks whether an update was installed should die when the corresponding chromium instance is terminated.

It also occurs with a freshly installed chromium.

It can be easily reproduced:

```
$ pgrep chromium | xargs kill # kill any running chromium (beware!)
$ pgrep chromium | wc -l
0 # There are non running
$ /snap/bin/chromium --product-version # command quits immediately.
$ pgrep chromium | wc -l
1
$ /snap/bin/chromium # opens a browser.
# Quit browser manually.
$ pgrep chromium | wc -l
2
$ ps faux | grep chromium # check any processes running.
\_/bin/sh /snap/chromium/1100/bin/chromium.launcher --product-version
\_/bin/sh /snap/chromium/1100/bin/chromium.launcher
```

This shows that a process remains active after closing or running.
The `ps faux` shows the leftover running processes and clearly shows they no longer have a parent: it seems the parent process forks, then quits, but leaves its child running.

Above comment comes from a bug that this behaviour introduced in a project that uses chrome: https://github.com/titusfortner/webdrivers/issues/168

So, besides it leaving around running processes (slightly messy), it causes bugs (hanging) in scripts and tools that wait for chrome+children to exit.

While it's not a beautiful solution, the attached patch does terminate the lingering process for me.

The sleep started by the symlink check is not killed, but will terminate after 60 seconds.

The attachment "1864901_termination_check_current_symlink.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

Thanks for your patch Lothar. I think your solution would work, although I went for a slightly different approach: https://git.launchpad.net/~chromium-team/chromium-browser/+git/snap-from-source/commit/?id=eb7aefe53b2c4710c030e04947ae66ced8fe9d3b.

(I am aware that pids can get recycled so it's not 100% correct, but it should work in most cases, and avoids loosing the exec call in the common code path (no temp profile)).

Comments and suggestions welcome, of course.

I get a notification if snap updates chromium in background.
But clicking the notification does nothing - only the notifcation disappears.
No restart of all chromium instances - so not really useful...

Indeed it's only an informative notification, it won't take any action by itself.

For the record I recently noticed the 30 leaked processes started taking all my CPU (all at the same time, maybe when an update was downloaded). They were all running in infinite loop and I couldn't see the "sleep 60" in their children any more in htop. I didn't investigate more as I had to quickly kill them to recover my machine. This is really too experimental and dangerous.

This feature caused me leaking processes.
See Bug: https://bugs.launchpad.net/ubuntu/+source/chromium-browser/+bug/1874973

Olivier, I wasn't able to test your patch. But I like your approach better than mine.

The problem with the numerous background sleep processes should be fixed in the stable channel now. Please shout here if you're still seeing it (note that existing background processes before refreshing to the latest version won't be cleaned up).

I'm still seeing the problem.

tarantula!root# snap refresh
All snaps up to date.

tarantula!root# snap list | grep chromium
chromium 81.0.4044.122 1119 latest/stable canonical* -

revision 1119 is not up-to-date, the latest for amd64 is now 1135.

tarantula!root# snap refresh
All snaps up to date.

How do I get the "latest" ?

That may sound like a dumb question, but are you connected to the network?

What's the output of the following command?

    snap info --abs-time chromium

tarantula!root# snap refresh
core18 20200427 from Canonical✓ refreshed

tarantula!root# snap refresh
All snaps up to date.

tarantula!root# snap list | grep chromium
chromium 81.0.4044.122 1119 latest/stable canonical* -

tarantula!root# snap info --abs-time chromium
name: chromium
summary: Chromium web browser, open-source version of Chrome
publisher: Canonical✓
store-url: https://snapcraft.io/chromium
contact: https://bugs.launchpad.net/ubuntu/+source/chromium-browser/+bugs?field.tag=snap
license: unset
description: |
  An open-source browser project that aims to build a safer, faster, and more stable way for all
  Internet users to experience the web.
commands:
  - chromium.chromedriver
  - chromium
snap-id: XKEcBqPM06H1Z7zGOdG5fbICuf8NWK5R
tracking: latest/stable
refresh-date: 2020-04-25T11:23:52+03:00
channels:
  latest/stable: 81.0.4044.129 2020-04-29T17:25:51Z (1135) 161MB -
  latest/candidate: 81.0.4044.129 2020-04-28T18:04:52Z (1135) 161MB -
  latest/beta: 83.0.4103.23 2020-04-25T02:07:58Z (1125) 163MB -
  latest/edge: 84.0.4128.3 2020-04-29T05:50:21Z (1136) 164MB -
installed: 81.0.4044.122 (1119) 161MB -

tarantula!root# ps axu | grep chr
root 561745 0.0 0.0 6436 728 pts/2 S+ 15:20 0:00 grep chr

tarantula!root# snap refresh chromium
error: cannot refresh "chromium": snap "chromium" has running apps (chromium)

Had to restart snapd to update chromium snap.

The issues with dependant scripts like chromerunner are not solved either.

Were external scripts considered when building this "solution" at all? External scripts, running chromium, obviously, want the process they spawn to exit -including any children that that process might spawn-.

Breaking this, is bad.

If, for some reason, different launch-patterns are required, such as update processes, CLI-processes, scripts or user-interaction, could we not introduce *standalone* scripts for that?

E.g. "update-chromium.sh" that would do all the hackish-run-in-background-with-sleep. And avoid cluttering the generic runner with that?
Or, if that *has* to be included in the generic launching, can we then introduce a `chromium-cli` or somesuch that does *not* have all these hacks for during updates in place?

The sleep process will outlive the launcher process at most 60 seconds, after which it will exit.

Note that this is only a temporary helper, until background refreshes of snaps are properly notified to running snaps by snapd through a well defined API.

Here is a solution suggestion why not storing version independent files such as Downloads, profile and other in common directory since it doesn't change between versions and keep version dependent files in version directories?

Just like how virtual machines and containers work, storage is linked to new VM for files which scopes are beyond VM lifespan and per VM temporary files lives in VM itself.

I tested and snap allows files in /home/user/snap/chromium/common to be accessed via chromium. What will be the down side of creating .config, Download, Documents, Videos etc. in that. From my previous experience i know that simply copying chromium directory in .config is okay to migrate profile to new chromium version.

This issue causes `ubuntu-bug chromium-browser` to hang.

Additionally, isn’t this an issue that’s best solved within snapd itself?

@Ebuzer: Downloads are already stored by default outside of the versioned directory, in the user's XDG downloads directory (typically $HOME/Downloads). The rationale for versioning the profile directory is that the storage format isn't backwards compatible, so downgrading a snap to a previous revision would potentially render a profile unusable.

@Roman: this is indeed something that, longer-term, should be addressed in snapd itself. This is merely a stop-gap measure to mitigate the problem until this happens. Note that ubuntu-bug hanging sounds unrelated to this particular issue. Can you elaborate on what you're trying to do, and how it hangs?

> The sleep process will outlive the launcher process at most 60 seconds, after which it will exit.

A problem I'm seeing is that the sleep process is sometimes *blocking* progression of other scripts e.g. if you pipe the output of a chromium process to something else:

```
$ snap refresh
All snaps up to date.

$ snap list | grep chromium
chromium 81.0.4044.138 1143 latest/stable canonical* -

$ time chromium --product-version
81.0.4044.138

real 0m0.206s
user 0m0.107s
sys 0m0.110s

$ time chromium --product-version | tee /dev/null
81.0.4044.138

real 1m0.161s # <--- !!!
user 0m0.122s
sys 0m0.078s
```

I came here from https://github.com/titusfortner/webdrivers/issues/168 as mentioned in #5, and the situation there is slightly worse, for reasons that aren't clear to me. That Ruby code is attempting to read the Chromium version through a pipe, very much like the following code, and this blocks *indefinitely*, not just for 60 seconds:
```
$ irb
irb(main):001:0> pipe = IO.popen(['chromium','--product-version'])
=> #<IO:fd 7>
irb(main):002:0> pipe.read
 … hangs indefinitely …
```

I can confirm both Roman's observation in comment #25, and Nick's in comment #27.
Seeing that there are real problems with the implementation, I'm going to revert it for now, until I can work out a better solution (or, ideally, a better mechanism is implemented in snapd itself).

Reverted in https://git.launchpad.net/~chromium-team/chromium-browser/+git/snap-from-source/commit/?id=d3a863f6e544867e6596601c4f5f5373539e56af.

Note that as a mitigation to bug #1616650, the chromium profile directory, which was previously versioned in $SNAP_USER_DATA is now under $SNAP_USER_COMMON, so problems should be much less visible to end users.