Publisher configuration needs redesign

Bug #45270 reported by Celso Providelo
2
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
Medium
William Grant

Bug Description

I'm not sure if it's mentioned in other bugs, but let's ensure we have an specific description of this issue.

Currently publisher system retrieves configuration options (like archive path, components to be published and other) from a text field in IDistribution and IDistroRelease, called 'lucilleconfig'. The text is formatted for python standard config module and is parsed on the fly.

There are some cheap alternatives to improves it:

 * Replace current DB field by a zope XML configuration section
 * Wrap the text field into a reliable and documented object with correct permissions and tests.

The former looks the right thing to do, but it may compromise the scalability of the system, since this configuration is particularly related to the distributions/distroreleases themselves (Gina uses this approach currently, so not a big deal)

The last sounds like a band-aid, but can also reach the required reliability and security we want.

Related branches

Revision history for this message
Celso Providelo (cprov) wrote :

Since we sort the design, some tests which have hardcoded archive path need to be revisited and fixed, they are, for instance, doc/soyuz-upload.txt and scripts/ftest/test_ftpmaster.py (there are other)

Changed in launchpad-publisher:
assignee: nobody → cprov
status: Unconfirmed → Confirmed
Revision history for this message
Daniel Silverstone (dsilvers) wrote :

The configuration is per distribution and per distrorelease. We should not expect it to be in the launchpad.conf

By all means make it an XML configuration blob which its in the column in the DB.

Alternatively we can just use proper columns instead of a nasty text field.

Revision history for this message
Celso Providelo (cprov) wrote : Re: [Bug 45270] Re: Publisher configuration needs redesign

Daniel Silverstone wrote:
> The configuration is per distribution and per distrorelease. We should
> not expect it to be in the launchpad.conf

Gina already does it, however I agree that gina works in a more
controlled environment than the distro/distrorelease.

> By all means make it an XML configuration blob which its in the column
> in the DB.

XML & config represent the same issue, I don't think people would be
able to edit/fine-tune it in place.

> Alternatively we can just use proper columns instead of a nasty text
> field.

Could be, but anyway we don't need that bunch of attributes in the DB.

See an example of IDistribution.lucilleconfig

[publishing]
pendingremovalduration=5
root=/var/tmp/archive
archiveroot=/var/tmp/archive/ubuntutest
poolroot=/var/tmp/archive/ubuntutest/pool
distsroot=/var/tmp/archive/ubuntutest/dists
overrideroot=/var/tmp/archive/ubuntutest-overrides
cacheroot=/var/tmp/archive/ubuntutest-cache
miscroot=/var/tmp/archive/ubuntutest-misc

The only attribute we need in IDistribution is the
pendingremovalduration, which could also defaults to a zope config value.

'root' sounds like a machine dependendent value to me and should fit in
the zope config as well, local_archive_path.

Maybe we still need to handle specific publication paths fo each
distribuiton (mirroring and permission issues), so, I'd not mind to have
 an IDistribution.archive_path, which would be joined with
local_archive_path to build former 'root'.

And then a set of path.join s for poolroot, distroot, overrideroot,
cacheroot and miscroot, which could also be provided as properties in
IDistribution. I can't see any use-case for customized cache/misc/pool.

And the IDistroRelease.lucilleconfig:

[publishing]
components = main restricted universe multiverse

Isn't it the same as IDistroRelease.components() ?

[]
--
Celso Providelo <email address hidden>
Canonical Ltd - http://www.canonical.com

Curtis Hovey (sinzui)
tags: added: tech-debt
Curtis Hovey (sinzui)
Changed in soyuz:
assignee: Celso Providelo (cprov) → nobody
tags: added: soyuz-publish
Revision history for this message
William Grant (wgrant) wrote :

Paths will now be calculated from archivepublisher.root, components from ComponentSelection, and the stay of execution from a new config key.

Changed in soyuz:
assignee: nobody → William Grant (wgrant)
status: Triaged → In Progress
Revision history for this message
Robert Collins (lifeless) wrote :

On Fri, Oct 15, 2010 at 10:32 PM, William Grant <email address hidden> wrote:
> Paths will now be calculated from archivepublisher.root, components from
> ComponentSelection, and the stay of execution from a new config key.

Stay of execution? If thats just 'how long to keep things' I suggest
using a flag for it, with a hard coded fallback.

Revision history for this message
Julian Edwards (julian-edwards) wrote :

On Friday 15 October 2010 10:56:01 you wrote:
> On Fri, Oct 15, 2010 at 10:32 PM, William Grant <email address hidden>
wrote:
> > Paths will now be calculated from archivepublisher.root, components from
> > ComponentSelection, and the stay of execution from a new config key.
>
> Stay of execution? If thats just 'how long to keep things' I suggest
> using a flag for it, with a hard coded fallback.

Given that we've never changed it, I say just hard-code it.

Revision history for this message
Launchpad QA Bot (lpqabot) wrote : Bug fixed by a commit
tags: added: qa-needstesting
Changed in launchpad:
status: In Progress → Fix Committed
William Grant (wgrant)
tags: added: qa-ok
removed: qa-needstesting
Curtis Hovey (sinzui)
Changed in launchpad:
milestone: none → 11.01
Curtis Hovey (sinzui)
Changed in launchpad:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.