Enable a builder to present a generalised history

Bug #491330 reported by Michael Nelson
14
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
Medium
Michael Nelson

Bug Description

Note: the soyuz aspect of this bug is complete, but other apps will need to update to the new db-schema before their builds will appear.

Currenty our user interface assumes that the build history of a builder consists solely of binary package builds:

https://edge.launchpad.net/builders/rothera/+history

This is because, currently IBuilder implements the IHasBuildRecords interface providing getBuildRecords(), which filters by binary build specific attributes, such as IBuild.buildstate, IBuild.distroarchseries and importantly, IBuild.builder, and returns a result set of IBuild.

We could update this to support IBuildBase instead (so that SourcePackageRecipeBuilds will also be included), and there are two main options for considering that (see bug 536700), but this will not help us present TranslationTemplatesBuildJobs that do not currently have an associated build.

So the options that we've discussed are:

1. use Job records for the history, as they obviously common for all jobs running on the build farm. Currently they are deleted as soon as the job completes successfully, but we could instead purge them after a set time if performance is an issue. (Julian is keen to get rid of this)

2. As on bug 536700, ensure that all build farm jobs have an associated build (and extract the common build attributes into a base BuildFarmJob table.

So far, we all agree it would be worthwhile spending the time to refactor the whole BuildFarmJob system, removing Job, and simplifying the rest of the related tables (see the attached build-related.png).

{{{
10:25 < bigjools> noodles775, wgrant, jtv-otp: Job on its own is too generic for a build farm job. I'd even be happy to ditch our use of Job and migrate to BuildFarmJob completely.
10:25 < wgrant> bigjools: Yeah, quite possibly.
10:25 < lifeless> call it task
10:25 < lifeless> that way you can really confuse people
10:25 < bigjools> I've never been really happy with using Job
10:25 < jtv-otp> bigjools: that whole subsystem needs a cleanup... not sure I'd enjoy yet more duplication
10:26 < bigjools> I felt press-ganged into using it
10:26 < wgrant> It never seemed like a really good idea to use it.
10:26 < wgrant> We do not use much of it.
10:27 < bigjools> I continually fail to see the duplication between Job and build farm jobs
10:27 < bigjools> someone wanted the buildd manager to become more like a job runner
10:27 < bigjools> it's not going to happen
}}}

More conversation at: http://irclogs.ubuntu.com/2010/03/11/%23launchpad-dev.html#t08:42

description: updated
Revision history for this message
Michael Nelson (michael.nelson) wrote :

Fixed in r60 of the production configs.

Revision history for this message
Michael Nelson (michael.nelson) wrote :

Er, wrong bug.

Changed in soyuz:
milestone: none → 10.01
tags: added: buildfarm
Changed in soyuz:
assignee: nobody → Michael Nelson (michael.nelson)
description: updated
summary: - Update Builder history UI for generalised builders
+ Enable a builder to present a generalised history
Changed in soyuz:
milestone: 10.01 → 10.02
Changed in soyuz:
milestone: 10.02 → 10.03
tags: added: wellington
description: updated
Changed in soyuz:
assignee: Michael Nelson (michael.nelson) → nobody
Revision history for this message
Michael Nelson (michael.nelson) wrote :
description: updated
Revision history for this message
Julian Edwards (julian-edwards) wrote :

We'll discuss this at the sprint next week.

Changed in soyuz:
milestone: 10.03 → pending
Revision history for this message
Michael Nelson (michael.nelson) wrote :

Marking as a duplicate of bug 536700 as the same solution will make both the Builder history and PPA builds views trivial.

Revision history for this message
Michael Nelson (michael.nelson) wrote :

I've just finished 536700 and am doing the builder history as a separate branch, so separate bug is good :)

Changed in soyuz:
assignee: nobody → Michael Nelson (michael.nelson)
milestone: pending → 10.06
Revision history for this message
Michael Nelson (michael.nelson) wrote :

This landed with devel r11041. Tested on edge vs production. This page already has performance issues that weren't being addressed by this branch at all, but surprisingly the query count has been reduced (unintended side-effect).

Edge at r11042: https://edge.launchpad.net/builders/palmer/+history?batch=50
    At least 286 queries issued in 6.75 seconds

Production (10.05): https://launchpad.net/builders/palmer/+history?batch=50
    At least 334 queries issued in 7.46 seconds

tags: added: qa-ok
description: updated
Changed in soyuz:
status: Triaged → Fix Committed
Changed in soyuz:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.