Generating zcml overrides writes to global state

Bug #663620 reported by Robert Collins
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Triaged
Low
Unassigned

Bug Description

the override file that generate_overrides writes to is global state; this can result in tests stomping on each other.

We could address this (given how static it is) by indirecting via python, which can know the config name.

Doing this should allow removing the overrides file altogether in fact.

Revision history for this message
Gary Poster (gary) wrote :

+1 on getting rid of the dynamically generated overrides.

"indirecting via python": IOW, instead of loading site.zcml always, we'd load different top-level zcmls, like ftesting.zcml or dev.zcml? Or something else?

Changed in launchpad-foundations:
status: New → Triaged
importance: Undecided → Low
Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 663620] Re: generate overrides writes to global state

On Thu, Oct 21, 2010 at 4:13 AM, Gary Poster <email address hidden> wrote:
> +1 on getting rid of the dynamically generated overrides.
>
> "indirecting via python":  IOW, instead of loading site.zcml always,
> we'd load different top-level zcmls, like ftesting.zcml or dev.zcml?  Or
> something else?

So, heres the full scenario for locality-of-reference:
test process
 - opens an ephemeral smtp port
 - starts a script (e.g. a job processor)
   - runs execute_zcml_for_scripts
     - configures its mailer to connect the smtp port

That might work, though the problem is that we need to in
'execute_zcml_for_scripts'
Currently this works via two things:
 - the port is statically allocated in testrunner-appserver/*.zcml
 - a global (statically named) overrides file includes the port in it

the thing we need to do is:
 - provide the port number from the test process to the job processor
(or appserver or *whatever* is being run)
 - not write to a well known file - either don't write to a file at
all, or write to a ephemeral file and hand off info about that via an
environment variable.

Revision history for this message
Gary Poster (gary) wrote : Re: generate overrides writes to global state

zcml can be read/included with globs, so an ephemeral file would work (if it is there, it is read; if not, no biggie, because it was to be found in a glob). That's the best idea I have with the info I have so far. example: <include files="package-includes/*-meta.zcml" /> (from site.zcml)

<includeOverrides file="test-*-meta.zcml" /> would be equivalent (or whatever glob pattern you want, wherever)

If we were going to use environment variables we could have...any old part of the system look at that environment variable. I'd do it outside of the zcml stuff if possible.

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 663620] Re: generate overrides writes to global state

On Thu, Oct 21, 2010 at 11:45 AM, Gary Poster <email address hidden> wrote:
> zcml can be read/included with globs, so an ephemeral file would work
> (if it is there, it is read; if not, no biggie, because it was to be
> found in a glob).  That's the best idea I have with the info I have so
> far. example: <include files="package-includes/*-meta.zcml" /> (from
> site.zcml)
>
> <includeOverrides file="test-*-meta.zcml" /> would be equivalent (or
> whatever glob pattern you want, wherever)
>
> If we were going to use environment variables we could have...any old
> part of the system look at that environment variable.  I'd do it outside
> of the zcml stuff if possible.

Me too, but this stuff is *already in* zcml, and I'm not sure how to
move it out of zcml.

ephemeral files are fine, but we need *unique ephemeral*, not
ephemeral-well-known.

Revision history for this message
Robert Collins (lifeless) wrote : Re: generate overrides writes to global state

We chatted on the phone; unique top level variants of ftesting.zcml and script-testing.zcml are the go

summary: - generate overrides writes to global state
+ Generating zcml overrides writes to global state
Revision history for this message
Robert Collins (lifeless) wrote :

This won't affect LXC based parallel testing as that uses an aufs or union filesystem to make all writes in a single test thread be localised.

Gary Poster (gary)
tags: removed: paralleltest
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.