Trial's test runner manages to break resource sharing in OptimisingTestSuite

Bug #297563 reported by James Henstridge
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Twisted
New
Unknown
testresources
Invalid
Critical
Unassigned

Bug Description

This bug is due to evilness in Twisted's TrialRunnner (tested using twisted 8.1.0).

Before passing control to the test suite, TrialRunner.run(test) calls "twisted.trial.unittest.decorate(test, ITestCase)", which pokes around inside the suite, replacing all the tests with versions adapted to the ITestCase interface.

None of the ResourcedTestCase tests implement this interface, so they get replaced by a _PyUnitTestCaseAdapter wrapper class, which lacks useful attributes like the "resources" attribute.

The upshot is that OptimisingTestSuite has no information to work with so can't sort tests in a meaningful fashion and can't share resources between tests.

It would be nice if testresources could work with such a test runner, since the trial test runner is required for some twisted related tests. Given the evilness of what it is doing, I'd understand if it can't be worked around though.

Revision history for this message
Robert Collins (lifeless) wrote :

I think this isn't a testresources bug per se; I'd be inclined to fix it by teaching the decorator to pass through unknown attribute lookups to the decorated objects. JML, do you concur?

Changed in testresources:
importance: Undecided → Critical
Revision history for this message
Glyph Lefkowitz (glyph) wrote :

My personal impression (take this with a grain of salt, I don't know too much about testresources yet) is that teaching the decorator to pass through unknown attribute lookups will fix this particular issue, but make the ultimate failure condition much more painful; especially given the proclivity of test-infrastructure code to make liberal use of potentially conflicting names like "name", "id", "count", etc.

The idea of somehow making the decorator less intrusive is a good one, though. If passing through attribute lookups could be combined with some mechanism for namespacing of additional attributes that might eliminate my concern...?

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 297563] Re: Trial's test runner manages to break resource sharing in OptimisingTestSuite

On Tue, 2008-11-18 at 09:14 +0000, Glyph Lefkowitz wrote:
> My personal impression (take this with a grain of salt, I don't know too
> much about testresources yet) is that teaching the decorator to pass
> through unknown attribute lookups will fix this particular issue, but
> make the ultimate failure condition much more painful; especially given
> the proclivity of test-infrastructure code to make liberal use of
> potentially conflicting names like "name", "id", "count", etc.

No more than subclassing or decorating elsewhere in python; while there
is no super-robust answer in python core, I don't see that we need to
solve it any more in other code, except where problems are actually
occuring.

> The idea of somehow making the decorator less intrusive is a good one,
> though. If passing through attribute lookups could be combined with
> some mechanism for namespacing of additional attributes that might
> eliminate my concern...?

I think I need a restating of your concern, perhaps as a mini story, to
understand it. My attempt follows:

When code from source A looks for 'test.FOO' and a test from source B
has 'FOO', allowing A to find test.FOO will break if B's FOO is
different, as as such the twisted test decorator should prevent A from
accessing test.FOO.

I think this is bogus, A and B will either work well together, or will
not. twisted's implementation detail (that it decorates test cases)
neither makes the cooperation between A and B better nor worse (if it
passes attribute lookups to the decorated object). Of course, it can
make it worse if it prohibits such lookups.

-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

Revision history for this message
Glyph Lefkowitz (glyph) wrote :

After an IRC discussion with Mr. Collins, and an evaluation of all the decorators that are currently in use, I agree that forwarding attribute access is probably the right way to address this specific problem. The decorators in question don't really define any interesting attributes, and the mechanism is not in wide enough use to pose a serious problem with conflicting attributes; if that were to happen, making individual decorators be as non-intrusive as possible so they won't conflict with anything isn't a terribly difficult problem.

In any event, the main point is that coming up with a more general mechanism for test-run extensibility is out of scope for fixing this issue.

Changed in twisted:
status: Unknown → New
Revision history for this message
Robert Collins (lifeless) wrote :

As per discussion with twisted upstream, this is not an issue for testresources to fix per-se.

Changed in testresources:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.