Binary package hint: squid
I'm running Ubuntu 10.04.2 LTS with latest version of squid 2.7.STABLE7-1ubuntu12.2. This is a default install - there have been no changes to /etc/squid.conf.
When I shutdown or reboot the system, I get the following error:
* Unmounting local filesystems...
umount2: Device or resource busy
umount: /var: device is busy.
(In some cases useful info about processes that use
the device is found by lsof(8) or fuser(1))
umount2: Device or resource busy
umount2: Device or resource busy
umount: /dev/mapper/vg01-usr busy - remounted read-only
[fail]
When the system starts up again, I see the following message:
/dev/mapper/vg01-var: recovering journal
It appears the /var partition was not cleanly unmounted during shutdown. Obviously, there is a risk of filesystem corruption here. FYI, i have /var and other important mountpoints as separate partitions (using LVM and ext4).
To confirm my suspicions of squid still running during shutdown, I checked /var/log/squid/cache.log and there is no indication that it received the signal to terminate. In fact, cache.log hints at this problem while the system is being brought up again:
Rebuilding storage in /var/spool/squid (DIRTY)
Only when I manually run "stop squid", then cache.log would report that squid stopped successfully:
Preparing for shutdown after 0 requests
Waiting 30 seconds for active connections to finish
FD 14 Closing HTTP connection
Shutting down...
FD 15 Closing ICP connection
Closing unlinkd pipe FD 12
... <skipped> ...
logfileClose: closing log /var/log/squid/store.log
logfileClose: closing log /var/log/squid/access.log
Squid Cache (Version 2.7.STABLE7): Exiting normally.
To test whether /var was locked by squid during shutdown, I added a script to run "ls /var/run | grep squid" and "lsof | grep var" before filesystems are unmounted. And voila!
ls /var/run | grep squid:
squid.pid
lsof | grep var:
squid 971 proxy cwd DIR 251,4 4096 130613 /var/spool/squid
squid 971 proxy 5u REG 251,4 13617 128678 /var/log/squid/cache.log
squid 971 proxy 7w REG 251,4 0 128724 /var/log/squid/access.log
squid 971 proxy 9w REG 251,4 0 128789 /var/log/squid/store.log
squid 971 proxy 11w REG 251,4 48 130308 /var/spool/squid/swap.state
Hence, the next system startup would result in /var recovering journal again.
Finally, I tried running "stop squid" before issuing the shutdown command and it successfully unmounts /var and I do not get /var recovering journal on the next system startup.
Thank you for taking the time to report this bug and helping to make Ubuntu better.
I was able to confirm this issue in the latest development release as well.