Comment 9 for bug 98437

I had an encounter with this one, when I tried to browser test Quickinstaller Control Panel of Plone. The Quickinstaller will redirect to the referrer send with the posted form request when a Product get's installed. I took a dive into the matter and here is what I found out (quoting a blog post I wrote on that matter; excuse the prose.):

There are various places inside the Zope testing framework working together to cause this weird behaviour. First there is zope.app.testing.functional which sets HTTP_REFERER to the nonsense value 'localhost'. As defined by RFC 2616, Section 14.36, the referrer is either a absolute or a relative URI. In this case it would be a relative URI, but one that simply makes no sense. Even if it was 'http://localhost/' it would not, because in most cases this would not be the correct referrer! This header value should be better left unset. After all, a web service might reckon with a suppressed referrer, but not with a misleading.

Removing those references will not solve the matter, however. The true problem is that zope.testbrowser.browser will actually never send a referrer for any submitted form. Only when it follows links, a referrer gets send along.

Zope's testbrowser builds on top the programmatic web browsing framework Mechanize. It wraps this framework completely, and unfortunately differently. The difference which is relevant here is the representation of HTML forms. Zope testbrowser offers objects for each HTML element needed for browser, i.e. links and various controls; also for forms. With the Mechanize library however the user selects a form with Browser and then fills it. The test-browser by-passes this part of the Mechanize API completely. It generates the HTTP request for posting the form with ClientForm.HTMLForm, then gives it to Browser.open of Mechanize. While this operation ultimately handles all web access, it does not set a referrer. It simply cannot: there is no way to tell whether the last call to open was related to the current call. That is why Mechanize sets the referrer only on clicks, submits and when following links, as mechanize.useragent.HTTPRefererProcessor explains.

Fixing this seems pretty easy, though I gave it no thorough testing yet. All one has to do is change zope.testbrowser.Browser._clickSubmit from

>>> self.mech_browser.open(form.click(
... id=control.id, name=control.name, label=label, coord=coord))

to

>>> self.mech_browser.form = form
>>> self.mech_browser.submit(id=control.id, name=control.name,
... label=label, coord=coord)

The only problem I see is concurrency safety, though I doubt that has any relevance here.