Comment 21 for bug 380504

On Thu, Mar 25, 2010 at 02:21:08PM -0000, Leonard Richardson wrote:
> > I generally get them one or two times a day on scripts that are run
> > hourly, so say less than 5% of the time.
>
> How long do you have to wait before the 5xx error goes away? Do you just
> try again an hour later?

Pretty much. I've seen a few cases where I'll get a group of 2-3 errors
across several scripts. What's funny is that they often are not the
same error. Like, I'll get an IncompleteRead error followed by one or
more 503 or 502 errors within an hour or two later. This makes me think
that while different error codes are being reported, that they are
interrelated in some fashion.

For example, here are the cron reports I've received over the past day
or so, for several different scripts cronned to run hourly:

# Crontab
 20 * * * * /home/bryce/src/Arsenal/arsenal/hooks/data-xorg-packagebugs
 22 * * * * /home/bryce/src/Arsenal/arsenal/hooks/data-upgrade
 24 * * * * /home/bryce/src/Arsenal/arsenal/hooks/data-bugtotals
 26 * * * * /home/bryce/src/Arsenal/arsenal/hooks/data-milestone-bugs

Date: Wed, 24 Mar 2010 10:04:39 -0700
Subject: Cron <bryce@chideok> /home/bryce/src/Arsenal/arsenal/hooks/data-bugtotals

...
httplib.IncompleteRead: IncompleteRead(15687 bytes read, 40586 more expected)

Date: Wed, 24 Mar 2010 19:05:43 -0700
Subject: Cron <bryce@chideok> /home/bryce/src/Arsenal/arsenal/hooks/data-patches

Exception: code 0
            reason 'HTTP Error 503: Service Unavailable
Response headers:

Date: Wed, 24 Mar 2010 21:53:45 -0700
Subject: Cron <bryce@chideok> /home/bryce/src/Arsenal/arsenal/hooks/data-bugtotals

httplib.IncompleteRead: IncompleteRead(57991 bytes read, 4914 more expected)

Date: Wed, 24 Mar 2010 21:58:53 -0700
Subject: Cron <bryce@chideok> /home/bryce/src/Arsenal/arsenal/hooks/data-bugtotals

Exception: code 0
            reason 'HTTP Error 502: Bad Gateway
Response headers:

Date: Thu, 25 Mar 2010 01:16:02 -0700
Subject: Cron <bryce@chideok> /home/bryce/src/Arsenal/arsenal/hooks/data-bugtotals

Exception: code 0
            reason 'HTTP Error 502: Bad Gateway
Response headers:

Date: Thu, 25 Mar 2010 01:16:44 -0700
Subject: Cron <bryce@chideok> /home/bryce/src/Arsenal/arsenal/hooks/data-milestone-bugs

Exception: code 0
            reason 'HTTP Error 502: Bad Gateway
Response headers:

Date: Thu, 25 Mar 2010 06:31:52 -0700
Subject: Cron <bryce@chideok> /home/bryce/src/Arsenal/arsenal/hooks/data-bugtotals

Exception: code 0
            reason 'HTTP Error 503: Service Unavailable
Response headers:

Since I get back html error pages in the emails I assume these are
errors on the launchpad side and not just network flakiness. Is that
an accurate assumption? I mean, if I'm the only person seeing these
issues maybe I should be looking for a faulty router or something?

Bryce