Upload rejected with "unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType'"

Bug #589068 reported by Michael Nelson
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
High
Michael Nelson

Bug Description

fta reported the following rejection this morning:

http://paste.ubuntu.com/443871/

The relevant bit being:
unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType'

Strangely, that rejection email is not displayed in the log, but at the exact time of the email we see:
{{{
2010-06-03 05:00:32 DEBUG Initialising connection.
2010-06-03 05:00:32 DEBUG Beginning processing
2010-06-03 05:00:32 DEBUG Could not fix the lockfile permission: [Errno 1] Operation not permitted: '/srv/launchpad.net/ppa-queue/incoming/.lock'
2010-06-03 05:00:32 DEBUG Checked in /srv/launchpad.net/ppa-queue/incoming, found ['upload-20100603-055540-000491']
2010-06-03 05:00:32 DEBUG Considering upload upload-20100603-055540-000491
2010-06-03 05:00:32 DEBUG Moving upload directory /srv/launchpad.net/ppa-queue/incoming/upload-20100603-055540-000491 to /srv/launchpad.net/ppa-queue/faile
d/upload-20100603-055540-000491
2010-06-03 05:00:32 DEBUG Moving distro file /srv/launchpad.net/ppa-queue/incoming/upload-20100603-055540-000491.distro to /srv/launchpad.net/ppa-queue/fai
led/upload-20100603-055540-000491.distro
2010-06-03 05:00:32 DEBUG Rolling back any remaining transactions.
}}}

(and there are quite a few more "Could not fix the lockfile permission" messages following.).

I'd take a guess that this is related to a piece of untested code in process-upload that is looking for the old build attributes (datecreated, datebuilt) rather than the new (date_created, date_finished etc.).

Related branches

Revision history for this message
Fabien Tassin (fta) wrote :

today, the exact same 3 uploads failed (i mean, same src package name, same dist, but newer version).
No other src package in my numerous PPA uploads seem to be affected.

So in ~ubuntu-mozilla-daily/ppa, xulrunner-1.9.2/{jaunty,hardy} and firefox/karmic are repeatedly failing.
I'm not sure why those 3 are different though, they are supposed to be exactly the same as the others.

Changed in soyuz:
assignee: nobody → Michael Nelson (michael.nelson)
importance: Medium → High
tags: added: soyuz-upload
Changed in soyuz:
milestone: none → 10.06
Revision history for this message
falkTX (Old) (falk-t-j) wrote :

I'm also affected by this bug.

Here's some more information:

I have a debian source package (ladish-git) and I've uploaded it to 2 different PPAs.
1 - https://launchpad.net/~ladi-team/+archive/ppa
2 - https://launchpad.net/~falk-t-j/+archive/lucid-latest

It fails to upload on the 2nd PPA, but fine on the first one
(link to package on 1st PPA - https://launchpad.net/~ladi-team/+archive/ppa/+sourcepub/1167978/+listing-archive-extra)

I tried to go the the 1st PPA then copy it to the 2nd one (using Lucid version, rebuild sources)
That gives me this error:
" Sorry, something just went wrong in Launchpad.
We’ve recorded what happened, and we’ll fix it as soon as possible. Apologies for the inconvenience.
(Error ID: OOPS-1621L614) "
which links to - https://lp-oops.canonical.com/oops.py/?oopsid=1621L614

Hope this is helpful

Changed in kxstudio:
status: New → Confirmed
importance: Undecided → High
Revision history for this message
Michael Nelson (michael.nelson) wrote :

Thanks falkTX - the oops is very helpful, as it's the same issue, but with a stack-trace :)

Based on the oops, it seems we have some completed builds (BuildStatus.FULLYBUILT), who's date_started is not set. Checking the destination PPA, here is the culprit:

https://launchpad.net/~falk-t-j/+archive/lucid-latest/+build/1768191

Note that that build was built on the same day as its amd64 counterpart:
https://launchpad.net/~falk-t-j/+archive/lucid-latest/+build/1768190

which has both date_started and date_finished.

Revision history for this message
falkTX (Old) (falk-t-j) wrote :

oh, this means the package was actually accepted, it's just the mail that was misleading...

anyway, thanks for working on this

Revision history for this message
Michael Nelson (michael.nelson) wrote :

OK, after racking my brains with wgrant, I found that all the examples of successful builds without timestamps that we know of, finished around: 20100602-1154

For example:
https://launchpad.net/~falk-t-j/+archive/lucid-latest/+build/1768191
https://edge.launchpad.net/~ubuntu-mozilla-daily/+archive/ppa/+build/1768594
https://edge.launchpad.net/~ubuntu-mozilla-daily/+archive/ppa/+build/1768591
https://edge.launchpad.net/~ubuntu-mozilla-daily/+archive/ppa/+build/1768601

which indicates that when the rollout started, perhaps data was not properly replicated.

But anyway, it means we can fix this error message itself trivially :)

Revision history for this message
Michael Nelson (michael.nelson) wrote :

Running some queries on staging show that as expected, there are 9 builds that completed since 2010-06-01 where date_started is None.

Changed in soyuz:
status: Triaged → In Progress
Revision history for this message
Michael Nelson (michael.nelson) wrote :

Fix committed with r10981 devel.

Changed in soyuz:
status: In Progress → Fix Committed
Revision history for this message
Ursula Junque (ursinha) wrote : Bug fixed by a commit
tags: added: qa-needstesting
Revision history for this message
Michael Nelson (michael.nelson) wrote :

QA'd on dogfood by

1. updating to db-devel r9444
2. Applying a patch to reverse the fix: http://pastebin.ubuntu.com/448192/
3. Manually remove the date_started of i386 build of hello 2.4-4ppa1: https://dogfood.launchpad.net/~michael.nelson/+archive/pocketsphinx/+build/1693756
4. Uploading 2.4-4ppa2
5. ppa-run.sh and watch process upload fail with the expected error: http://pastebin.ubuntu.com/448190/

Then,

1. Revert patch to pristine 9444
2. Upload 2.4-4ppa2 again.
3. ppa-run.sh and watch process upload continue as expected: http://pastebin.ubuntu.com/448194/

tags: added: qa-ok
removed: qa-needstesting
Revision history for this message
falkTX (Old) (falk-t-j) wrote :

I still get the error:
"Rejected:
unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType'"

As before, the same package compiles fine in 1 PPA, but not in the other.

Successful build:
https://launchpad.net/~ladi-team/+archive/ppa/+sourcepub/1172311/+listing-archive-extra

No build:
https://launchpad.net/~falk-t-j/+archive/lucid-latest

Revision history for this message
Michael Nelson (michael.nelson) wrote :

Hi falkTX,

Yes, the fix has not landed in production, so you will continue to see the error for uploads for the specific package until the next release (unless it's cherry-picked), or we manually fix the missing date_started (which we should do, given the circumstances).

However, if you use the edge system to copy the package as you mentioned at:

https://bugs.launchpad.net/soyuz/+bug/589068/comments/2

it should copy now without oopsing, but *only* if all your requests are via the edge system:

https://edge.launchpad.net/~ladi-team/+archive/ppa

Revision history for this message
falkTX (Old) (falk-t-j) wrote :

Thanks!
It works!

It's not perfect, but it's a nice workaround.

Revision history for this message
falkTX (Old) (falk-t-j) wrote :

I can now say that the bug seems fixed.

Recent upload has been successful:
https://launchpad.net/~falk-t-j/+archive/lucid-latest/+sourcepub/1180109/+listing-archive-extra

(this is the PPA that had problems)

Revision history for this message
Fabien Tassin (fta) wrote :

not fixed for me. the exact same 3 src packages have been rejected today.
I guess the fix didn't land in production yet. did it?

Revision history for this message
Julian Edwards (julian-edwards) wrote :

The fix is not in production until the status changes to "Fix Released". This should be in about a week.

Revision history for this message
Michael Nelson (michael.nelson) wrote : Re: [Bug 589068] Re: Upload rejected with "unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType'"

On Wed, Jun 23, 2010 at 9:26 AM, Fabien Tassin <email address hidden> wrote:
> not fixed for me. the exact same 3 src packages have been rejected today.
> I guess the fix didn't land in production yet. did it?

No, it'll be in the 10.06 rollout. I haven't looked into why falkTX
can now upload his package, but it's not related to this fix.

Changed in soyuz:
status: Fix Committed → Fix Released
Revision history for this message
falkTX (Old) (falk-t-j) wrote :

changing this to fixed too

Changed in kxstudio:
status: Confirmed → Fix Released
Revision history for this message
Fabien Tassin (fta) wrote :

yep, confirmed fixed. Thanks!

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.