Launchpad itself

cannot access attachments of private bugs any more

Bug #620458 reported by Martin Pitt on 2010-08-19

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Launchpad itself	Fix Released	High	Abel Deuring	Launchpad itself 11.01
	httplib2	Unknown	Unknown	auto-code.google.com-18 #62

Bug Description

For some weeks the Apport retracers have started crashing as soon as they try to access an attachment of a private bug. This has started to happen independently of the data center firewall configuration (since I can reproduce it locally as well) or python-launchpadlib and dependency version bumps (since I get the same on the local checkout on ronne, which hasn't been updated in ages). It seems that this regressed in Launchpad itself, but I'm also filing it against python-launchpadlib for a potential improvement of the error message, since it is very unclear.

Simple reproducer:

$ lp-shell
>>> b=lp.bugs[614676]
>>> b.attachments[0].data
<lazr.restfulclient.resource.HostedFile object at 0x1d33a90>
>>> f = b.attachments[0].data.open()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/lib/python2.6/dist-packages/lazr/restfulclient/resource.py", line 357, in open
    return HostedFileBuffer(self, mode, content_type, filename)
  File "/usr/lib/python2.6/dist-packages/lazr/restfulclient/resource.py", line 953, in __init__
    self.url, return_response=True)
  File "/usr/lib/python2.6/dist-packages/lazr/restfulclient/_browser.py", line 287, in get
    response, content = self._request(url, extra_headers=headers)
  File "/usr/lib/python2.6/dist-packages/lazr/restfulclient/_browser.py", line 242, in _request
    str(url), method=method, body=data, headers=headers)
  File "/usr/lib/python2.6/dist-packages/lazr/restfulclient/_browser.py", line 211, in _request_and_retry
    url, method=method, body=body, headers=headers)
  File "/usr/lib/python2.6/dist-packages/httplib2/__init__.py", line 1129, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/python2.6/dist-packages/lazr/restfulclient/_browser.py", line 130, in _request
    redirections, cachekey)
  File "/usr/lib/python2.6/dist-packages/httplib2/__init__.py", line 946, in _request
    (response, content) = self.request(location, redirect_method, body=body, headers = headers, redirections = redirections - 1)
  File "/usr/lib/python2.6/dist-packages/httplib2/__init__.py", line 1129, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/python2.6/dist-packages/lazr/restfulclient/_browser.py", line 130, in _request
    redirections, cachekey)
  File "/usr/lib/python2.6/dist-packages/httplib2/__init__.py", line 901, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/lib/python2.6/dist-packages/httplib2/__init__.py", line 871, in _conn_request
    response = conn.getresponse()
  File "/usr/lib/python2.6/httplib.py", line 988, in getresponse
    method=self._method)
  File "/usr/lib/python2.6/httplib.py", line 330, in __init__
    self.fp = sock.makefile('rb', 0)
AttributeError: 'NoneType' object has no attribute 'makefile'

This only happens on private bugs, public bugs work fine.

Tags:

Related branches

lp:~adeuring/launchpad/bug-620458-private-bugattachments-api-access

Merged into lp:launchpad/db-devel at revision 9742

Deryck Hodge (community): Approve (code) on 2010-09-02

lp:~adeuring/launchpad/librarian-filealiases

Merged into lp:launchpad at revision 12041

Graham Binns (community): Approve (code) on 2010-12-07

lp:~adeuring/launchpad/webservice-access-to-private-bug-attamchments

Merged into lp:launchpad at revision 12212

Leonard Richardson (community): Approve on 2011-01-06

Graham Binns (community): Approve (code) on 2011-01-04

Revision history for this message

Francis J. Lacoste (flacoste) wrote on 2010-08-19:

The problem is that HostedFile only redirects to the librarian, but this file is available through the restricted librarian so cannot be fetched direcly by the clients (for now, the fix for bug 395960 will likely change that).

HostedFile should stream the file directly when it is a restricted file.

affects:

launchpadlib → launchpad-foundations

Revision history for this message

Francis J. Lacoste (flacoste) wrote on 2010-08-19:

We should expedite the fix as it is a regression and is causing a huge backlog of unprocessed apport bugs.

Changed in launchpad-foundations:
importance:	Undecided → High

Gary Poster (gary) on 2010-08-19

Changed in launchpad-foundations:
status:	New → Triaged
affects:	launchpad-foundations → launchpadlib
affects:	launchpad → launchpad-foundations
Changed in launchpad-foundations:
importance:	Undecided → High
status:	New → Triaged
Changed in launchpadlib:
importance:	High → Low

Francis J. Lacoste (flacoste) on 2010-08-19

Changed in launchpad-foundations:
importance:	High → Low
affects:	launchpadlib → malone
affects:	launchpad-foundations → ilaunchpad-shell

Francis J. Lacoste (flacoste) on 2010-08-19

affects:

ilaunchpad-shell → launchpadlib

Revision history for this message

James Westby (james-w) wrote on 2010-08-19: Re: [Bug 620458] [NEW] cannot access attachments of private bugs any more

On Thu, 19 Aug 2010 14:11:28 -0000, Martin Pitt <email address hidden> wrote:
> Public bug reported:
>
> It seems that this regressed in
> Launchpad itself, but I'm also filing it against python-launchpadlib for
> a potential improvement of the error message, since it is very unclear.

That's a httplib2 bug.

http://code.google.com/p/httplib2/issues/detail?id=62

Thanks,

James

Abel Deuring (adeuring) on 2010-08-19

Changed in malone:
assignee:	nobody → Abel Deuring (adeuring)

Revision history for this message

Martin Pitt (pitti) wrote on 2010-08-19:

Ah, thanks James. I close the upstream httplib2 task then since it's fixed upstream.

affects:	launchpadlib → httplib2
Changed in httplib2:
status:	Triaged → Fix Released
importance:	Low → Unknown
status:	Fix Released → Unknown

Revision history for this message

Francis J. Lacoste (flacoste) wrote on 2010-08-19:

The easiest way to patch this is to override the behavior of the LibraryBackedByteStorage alias_url property for BugAttachment.

To do so, we should create a new AttachmentField extending Bytes which provide an IAttachementField interface and register a subclass of LibraryBackedByteStorage as its IByteStorage implementation.

(See lib/canonical/launchpad/zcml/webservice.zcml for the existing registration)

The alias_url property should return a URL to the attachment on the web service host for private attachment.
(We cannot point at the normal web site, since the launchpadlib client won't know how to authenticate their.) Public attachment should use the normal librarian URL.

We should declare that StreamOrRedirectFileAliasView provides IHTTPResource so that the view can work on the web service host.

Revision history for this message

Sebastien Bacher (seb128) wrote on 2010-08-23:

is there any chance that can be workarounded in some way or will be worked this week?
without the retracers the crash bugs on ubuntu don't reach the bug triagers

Revision history for this message

Abel Deuring (adeuring) wrote on 2010-08-23:

I hope that I can fix the bug this week.

Revision history for this message

Deryck Hodge (deryck) wrote on 2010-08-23:

Just to be clear for seb and others, this is the next thing in Abel's queue. He will be taking up the work this afternoon, and will work on this until the bug is fixed.

Cheers,
deryck

Revision history for this message

Sebastien Bacher (seb128) wrote on 2010-08-23:

Thanks Abel and Deryck for your work on the issue!

Revision history for this message

Abel Deuring (adeuring) wrote on 2010-08-30:

#10

lifeless cautioned on IRC that serving the files via StreamOrRedirectLibraryFileAliasView may easily lead to timeouts. He suggested to give the retracer machines direct access to the restricted librarian; that should fix the narrow" issue. I've asked our admins in RT 41109 to do this.

Once lifeless's branch to allow direct access to restricted Librarian files via a query parameter has landed, we can also fix the issue for "ordinary" webservice clients, without risking timeouts or hammering the app servers.

Deryck Hodge (deryck) on 2010-09-01

Changed in malone:
importance:	Low → High

Deryck Hodge (deryck) on 2010-09-03

Changed in malone:
status:	Triaged → In Progress

Revision history for this message

Launchpad QA Bot (lpqabot) wrote on 2010-09-04: Bug fixed by a commit

#11

Fixed in stable r11491 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/11491>.

Changed in malone:
milestone:	none → 10.09
tags:	added: qa-needstesting
Changed in malone:
status:	In Progress → Fix Committed

Revision history for this message

Markus Korn (thekorn) wrote on 2010-09-05:

#12

I can confirm it's fixed:

In [7]: from launchpadlib.uris import LPNET_SERVICE_ROOT
In [8]: from launchpadlib.launchpad import Launchpad
In [9]: lp = Launchpad.login_with("testing", LPNET_SERVICE_ROOT, "/tmp/lp", version="devel")
[...]
In [10]: b = lp.bugs[622597]
In [11]: b.attachments[0].data
Out[11]: <lazr.restfulclient.resource.HostedFile object at 0x2928a90>
In [12]: f = b.attachments[0].data.open()
In [13]: f
Out[13]: <lazr.restfulclient.resource.HostedFileBuffer instance at 0x2921a28>

Revision history for this message

Robert Collins (lifeless) wrote on 2010-09-05: Re: [Bug 620458] Re: cannot access attachments of private bugs any more

#13

On Sun, Sep 5, 2010 at 9:27 PM, Markus Korn <email address hidden> wrote:
> I can confirm it's fixed:
>
> In [7]: from launchpadlib.uris import LPNET_SERVICE_ROOT
> In [8]: from launchpadlib.launchpad import Launchpad
> In [9]: lp = Launchpad.login_with("testing", LPNET_SERVICE_ROOT, "/tmp/lp", version="devel")
> [...]
> In [10]: b = lp.bugs[622597]
> In [11]: b.attachments[0].data
> Out[11]: <lazr.restfulclient.resource.HostedFile object at 0x2928a90>
> In [12]: f = b.attachments[0].data.open()
> In [13]: f
> Out[13]: <lazr.restfulclient.resource.HostedFileBuffer instance at 0x2921a28>

Where are you running that? this isn't meant to be fixed except on a
very specific machine in the canonical datacentre.

Revision history for this message

Launchpad QA Bot (lpqabot) wrote on 2010-09-06: Bug fixed by a commit

#14

Fixed in stable r11506 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/11506>.

Revision history for this message

Deryck Hodge (deryck) wrote on 2010-09-07:

#15

We have confirmed the permissions issues are fixed, so I'll mark this qa-ok. There are remaining issues like timeouts with uploads that abel and seb128 are actively working on.

tags:

added: qa-ok
removed: qa-needstesting

Curtis Hovey (sinzui) on 2010-09-09

Changed in malone:
status:	Fix Committed → Fix Released

Revision history for this message

Martin Pitt (pitti) wrote on 2010-09-13:

#16

This is still happening in the Apport test suite, for a bug filed on staging. Test case:

$ lp-shell staging
Connected to LP service "https://api.staging.launchpad.net/" with API version "1.0":
Note: LP can be accessed through the "lp" object.
>>> b=lp.bugs[634699]
>>> f = b.attachments[0].data.open()
Traceback (most recent call last):
[...]
AttributeError: 'NoneType' object has no attribute 'makefile'

Bug 634699 is only accessible to me for now, but I don't think it's specific to me. Just try filing a private bug, and use that one.

Revision history for this message

Martin Pitt (pitti) wrote on 2010-09-13:

#17

Reopening, since this is still happening.

Changed in malone:
status:	Fix Released → Confirmed

Revision history for this message

Deryck Hodge (deryck) wrote on 2010-09-13:

#18

Martin is right. This isn't fixed yet. We only allowed access for the retracers until we can get a permanent fix in place. Abel will be working with Robert to finish off the new private librarian work, which should fix this for everyone.

Changed in malone:
status:	Confirmed → Triaged
milestone:	10.09 → none

Peter Petrakis (peter-petrakis) on 2010-12-08

Changed in oem-priority:
importance:	Undecided → High

Revision history for this message

Launchpad QA Bot (lpqabot) wrote on 2010-12-10:

#19

Fixed in stable r12041 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/12041>.

tags:	added: qa-needstesting removed: qa-ok
Changed in malone:
status:	Triaged → Fix Committed

Revision history for this message

Abel Deuring (adeuring) wrote on 2010-12-10:

#20

r12041 unfortunately does _not_ fix this bug: Webservice clients simply "see" the public URL for attachments of private bugs, but they should see a URL containing the access token

Changed in malone:
status:	Fix Committed → In Progress

Jonathan Lange (jml) on 2010-12-10

tags:

added: ubuntu-platform

Abel Deuring (adeuring) on 2010-12-10

tags:

added: qa-bad
removed: qa-needstesting ubuntu-platform

Deryck Hodge (deryck) on 2010-12-10

tags:

added: ubuntu-platform

Revision history for this message

Deryck Hodge (deryck) wrote on 2010-12-16:

#21

Marking qa-ok since r12041 has been reverted. Abel forgot to use the --rollback flag. Also, I'm sorry to say I'm still dumb about what to do to patch up the tagging when this happens. If we need a bad commit tag manually, or not?

tags:

added: qa-ok
removed: qa-bad

Diogo Matsubara (matsubara) on 2010-12-16

tags:

added: qa-bad
removed: qa-ok

Revision history for this message

Diogo Matsubara (matsubara) wrote on 2010-12-16:

#22

Set to qa-bad again to block deployment until we sort out the confusion done by the revert. Due to the missing --rollback option the https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html is reporting the wrong revision to be deployed.

Revision history for this message

Francis J. Lacoste (flacoste) wrote on 2010-12-17:

#23

Marked as qa-ok to unblock the qa-tagger report, but 12041 shouldn't be rolled out without 12080.

tags:	added: qa-ok removed: qa-bad
tags:	added: bad-commit-12041

Revision history for this message

Launchpad QA Bot (lpqabot) wrote on 2011-01-05:

#24

Fixed in stable r12165 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/12165>.

Changed in launchpad:
milestone:	none → 11.01
tags:	added: qa-needstesting removed: qa-ok
Changed in launchpad:
status:	In Progress → Fix Committed

Abel Deuring (adeuring) on 2011-01-06

tags:

added: qa-ok
removed: qa-needstesting

Curtis Hovey (sinzui) on 2011-01-11

Changed in launchpad:
status:	Fix Committed → Fix Released

Revision history for this message

Steve Magoun (smagoun) wrote on 2011-04-25:

#25

Removing from oem-priority, this isn't an appropriate oem-priority escalation. The bug in LP appears to be fixed, anyway.

affects:

oem-priority → null

William Grant (wgrant) on 2011-04-27

Changed in null:
status:	New → Invalid

Vikram Venkatesh (vikramvenkatesh85) on 2011-07-13

Changed in launchpad:
assignee:	Abel Deuring (adeuring) → Vikram Venkatesh (vikramvenkatesh85)
assignee:	Vikram Venkatesh (vikramvenkatesh85) → nobody

Robert Collins (lifeless) on 2011-07-13

Changed in launchpad:
assignee:	nobody → Abel Deuring (adeuring)

Curtis Hovey (sinzui) on 2011-11-11

no longer affects:

null

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

auto-code.google.com-18 #62 Edit

Bug watches keep track of this bug in other bug trackers.