apt-get hangs with "Waiting for headers" for exactly two minutes on dl.google.com apt line

Bug #528925 reported by Pedro Côrte-Real
142
This bug affects 28 people
Affects Status Importance Assigned to Milestone
APT
Unknown
Unknown
apt (Ubuntu)
Incomplete
Low
daniele

Bug Description

Binary package hint: apt

I installed google chrome in karmic and later lucid. Google chrome adds the following apt line to update itself:

deb http://dl.google.com/linux/deb/ stable main

When running apt-get update with this line installed it will hang for exactly two minutes saying "Waiting for headers". I've ran a wireshark dump of the connection and it seems to be stopping right after receiving Release.gpg. After some kind of timeout is reached it seems to close the connection and then download everything else and finish without problems.

Attached is the wireshark log of all this. Seems to be some kind of bug in how apt's http code handles the google server. Both wget and curl fetch the file without problems. Why does apt even need its own implementation of an HTTP client?

Revision history for this message
Pedro Côrte-Real (pedrocr) wrote :
Revision history for this message
Pedro Côrte-Real (pedrocr) wrote :

For some reason there is no apt-dbg package so I compiled a new apt package with unstripped binaries. gdb gives me this backtrace of the http method when the connection is hung:

0 0x00007f20dfbe5c53 in select () from /lib/libc.so.6
#1 0x0000000000404b14 in HttpMethod::Go (this=0x7fff544f4ab0,
    ToFile=<value optimized out>, Srv=0x2080aa0) at http.cc:789
#2 0x00000000004060e0 in ServerState::RunHeaders (this=0x2080aa0)
    at http.cc:396
#3 0x0000000000407ecc in HttpMethod::Loop (this=0x7fff544f4ab0)
    at http.cc:1151
#4 0x0000000000409fcb in main () at http_main.cc:19

I haven't spent too much time looking at the code but one thing that came to mind was that the file was 189 bytes only so that maybe when it gets to the select it is waiting for extra data after the whole file has already been transmitted and it somehow has missed that.

Revision history for this message
Torsten Spindler (tspindler) wrote :

I can reproduce the problem on Lucid. A two minute hang when adding the line to sources.list.d/google.list.

Changed in apt (Ubuntu):
status: New → Confirmed
Revision history for this message
Adrian Bridgett (adrian-bridgett) wrote :

I also see this on debian unstable using apt-cacher-ng FYI. hangs for a bit, then finishes

apt-cacher-ng: Version: 0.4.4-1~bpo50+1
apt: Version 0.7.25.3

...
Get: 68 http://ftp.uk.debian.org sid/main 2010-03-10-0900.30.pdiff [1,826B]
99% [Waiting for headers]
99% [Waiting for headers]
99% [Waiting for headers]
Ign http://dl.google.com stable/non-free Translation-en_GB
Ign http://dl.google.com stable/main Translation-en_GB
Hit http://dl.google.com stable Release
Ign http://dl.google.com stable/non-free Packages/DiffIndex
Ign http://dl.google.com stable/main Packages/DiffIndex
Hit http://dl.google.com stable/non-free Packages
Hit http://dl.google.com stable/main Packages
Fetched 300kB in 2min 1s (2,460B/s)

Revision history for this message
gozdal (gozdal) wrote :

It seems to be a problem with HTTP/1.1 pipelining on Google's side. APT sends 6 requests in pipeline to the server and receives only one response:

GET /linux/deb/dists/stable/Release.gpg HTTP/1.1
Host: dl.google.com
Connection: keep-alive
If-Modified-Since: Sat, 20 Mar 2010 00:00:00 GMT
User-Agent: Ubuntu APT-HTTP/1.3 (0.7.23.1ubuntu2)

GET /linux/deb/dists/stable/non-free/i18n/Translation-pl.bz2 HTTP/1.1
Host: dl.google.com
Connection: keep-alive
User-Agent: Ubuntu APT-HTTP/1.3 (0.7.23.1ubuntu2)

GET /linux/deb/dists/stable/main/i18n/Translation-pl.bz2 HTTP/1.1
Host: dl.google.com
Connection: keep-alive
User-Agent: Ubuntu APT-HTTP/1.3 (0.7.23.1ubuntu2)

GET /linux/deb/dists/testing/Release.gpg HTTP/1.1
Host: dl.google.com
Connection: keep-alive
If-Modified-Since: Sat, 20 Mar 2010 00:00:00 GMT
User-Agent: Ubuntu APT-HTTP/1.3 (0.7.23.1ubuntu2)

GET /linux/deb/dists/testing/non-free/i18n/Translation-pl.bz2 HTTP/1.1
Host: dl.google.com
Connection: keep-alive
User-Agent: Ubuntu APT-HTTP/1.3 (0.7.23.1ubuntu2)

GET /linux/deb/dists/testing/main/i18n/Translation-pl.bz2 HTTP/1.1
Host: dl.google.com
Connection: keep-alive
User-Agent: Ubuntu APT-HTTP/1.3 (0.7.23.1ubuntu2)

HTTP/1.1 200 OK
Last-Modified: Sat, 20 Mar 2010 00:00:00 GMT
Accept-Ranges: bytes
Content-Length: 189
Content-Type: application/octet-stream
ETag: 12798
Vary: *
Date: Sun, 21 Mar 2010 00:05:52 GMT
Server: downloads
X-XSS-Protection: 0

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQBLpAB7oECDD3+sWZERAjWDAJ0U/RrTcNgDBndhZhzpfO8IkGTxrgCeM4g2
MTcZmYqhpJ6jreLHBW4H/F0=
=u/IL
-----END PGP SIGNATURE-----

As a workaround adding

Acquire::http::Pipeline-Depth "0";

to to a file in /etc/apt.conf.d fixes the problem for me.

Revision history for this message
Stuart Colville (muffinresearch) wrote :

gozdal's workaround to disable http/1.1 pipelining does the job, thanks!

Note: The apt conf dir is /etc/apt/apt.conf.d/

Revision history for this message
Nuno Lucas (ntlucas) wrote :

I can also confirm that the workaround works for me.

On the other hand it seems the update is slower than before, so while it's a nice workaround the real fix should be to make apt to fallback to this automatically if the previous method times out.

Revision history for this message
spinkham (steve-pinkham) wrote :

The google Chromium bug, which is linked above, claims the real fix on their end should be out in the next few days.

Revision history for this message
Darkmike (mikefaille) wrote :

This workaround work for me too.

Revision history for this message
sobi3ch (sobi3ch) wrote :

You guys mean /etc/apt/apt.conf.d/05aptitude right?

Revision history for this message
Nuno Lucas (ntlucas) wrote :

sobi3ch: Any file will do. It will only matter if the same configuration is changed on a file read latter (with a higher number).

Revision history for this message
Victor Vargas (kamus) wrote :

According to upstream comments this issue is solved, please could somebody check if this behaviour is still occurring under latest release of Ubuntu Maverick? Thanks.

Changed in apt (Ubuntu):
importance: Undecided → Low
status: Confirmed → Incomplete
Revision history for this message
Wayne Scott (wsc9tt) wrote :

I was seeing this problem and it was certainly fixed months ago.
The problem was an incorrectly configured web server at google.

However I am still running Luicd, but I don't see why Maverick should behave
any differently.

Revision history for this message
Lucifer (lucifer666inferno) wrote :

In Maverick i have the same problem with the google repository.

Revision history for this message
David Clayton (dcstar) wrote :

This problem affects my 10.10 VMs as all package activities seem to stall waiting for non-existent _AU translation files.

Once whatever timeout is concluded, downloading and updating works but I still have to wait a couple of minutes before things get going.

The pipeline workaround cures this problem for me.

Revision history for this message
vasilisc (vasilisc) wrote :

/etc/apt/apt.conf
Acquire::http::Timeout "300";

daniele (dacelli)
Changed in apt (Ubuntu):
assignee: nobody → daniele (dacelli)
Revision history for this message
Jo Vermeulen (jozilla) wrote :

Still occurs on 12.04 for me.

Revision history for this message
Kyle M Weller (kylew) wrote :

i can confirm this on 12.04

Revision history for this message
Cris G (selfmedicate0440) wrote :

Still occuring as of 8/25/14 on 14.04.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.