proxying user supplied libarian files via the launchpad appserver domain has security and performance issues

Idea: OpenID authentication might come to the rescue here. librarian.launchpad.net could be an OpenID consumer, and every file served out of there needing authentication authenticated with a different trust root.

@curtis, could you expand on your assessment of low here? I would naively expect a security issue to be high/critical.

see also https://bugs.edge.launchpad.net/launchpad-foundations/+bug/242482

My prioritisation is something of a provocation. The bug was reported by a launchpad engineer, and I expect an engineer to be able to judge priority. Given that the issue was ignore for 6 months, it is clear no Launchpad engineer was concerned about this. I set it low to be honest about how we are treating this.

I don't think setting priority purely based on how much attention a bug has got is a good model, otherwise why bother having a priority at all - why not just auto-generate it based on that (hmm, bug heat?). Priority is supposed to be what we want it to be, rather than reinforcing that's it's okay to ignore something just because it's not been done already.

I agree it is not the best way. But I am certain this bug was not high. This bugs we believe were more important were worked instead. Launchpad has more high bugs that engineers will address in the next 6 months. These bugs are lying about their priority. All parties subscribed to a bug are obligated to re-evaluate a bugs priority as the system and the way launchpad is used changes. A bug that is low will not be low if the need for it changes. (And high bugs do become low as needs change.)

Our rules for critial priortisation are lose of user data, security issues users are experiencing, privacy issues users are experiencing. I have found and fixed a dozen bugs, some reported by Launchpad engineers, that would have been fixed months sooner if someone had taken the time to read the bug, judge the priorty, and start a discussion about why the priority was selected.

So in summary. The foundations team demonstrated this bug was not a priority, and no subscriber challenged it until recently. Something in the system, or how Launchpad is used, or we have changed and we think this bug needs re-evaluation. This is the correct and expected behaviour. I welcome someone to give this bug a high priorty and treat is as such.

Given Curtis' description of how to triage (only use High and Low importance, where High means "we'll do it in the next three months" and Low means "we'll see"), which has been accepted as the Launchpad approach and which does make sense to me given the absence of priority queues for our bugs, his analysis of the situation is apt.

That doesn't mean I like it.

This bug comment is not the right forum to discuss my opinions on the situation.

As far as the bug is concerned, no, I don't expect that Foundations will tackle it in the next three months unless it gains momentum above other efforts that we are driving and that we have been requested to pursue.

As I understand it from Stuart, this can be addressed easily if Robert's proposal to use tokens between the Librarian and the application server (and thus no longer have the application server proxy the Librarian) works.

I think the confusion here is about importance, not the value, but the field itself. Important is severity to the user + engineer's will to address + certainty of success. The composite value is used communicate when the bug will be fixed. It is clear by the private state that this is severe, but the description hints at a lot of uncertainty, which I attribute to why the bug was not address sooner.

This new information shows certainty has changed, but that it is not enough to influence will. I think Gary is staying that while we are know how to fix it, the fix requires more time than he can allocate in the near future because more important issues.

Wow; I didn't mean to provoke such deep discussion :) - I really was just interested in the reasoning that went into this particular change.

Anyhow, there are a cluster of bugs around private librarian behaviour - it is slow, it causes timeouts, its going to be hard to fix, and changing the behaviour will make it fast, reduce timeouts and be easy to do - I'm nearly done in fact, though I don't have the mental agility to do more code today.

Now in QA on qastaging.

Hm, this doesn't appear to work for me at all. For example, fetching this:

https://qastaging.launchpad.net/~ubuntu-security/+archive/ppa/+files/openjdk-6_6b18-1.8.2-0ubuntu1%7E9.10.1_source.changes

causes a 302 to:

https://i57525770.restricted.qastaging.launchpadlibrarian.net/57525770/openjdk-6_6b18-1.8.2-0ubuntu1~9.10.1_source.changes?token=xxxxxxxxxxxxxxxxxxb746002bd0b5e8

(edited to include the "x"s) which gives me a 404 both through the API and the web UI.

And this:

https://qastaging.launchpad.net/~ubuntu-security/+archive/ppa/+files/openjdk-6_6b20-1.9-0ubuntu1_6b20-1.9.1-0ubuntu1.diff.gz

302s to:

https://i57525944.restricted.qastaging.launchpadlibrarian.net/57525944/openjdk-6_6b20-1.9-0ubuntu1_6b20-1.9.1-0ubuntu1.diff.gz?token=xxxxxxxx775360f0228f45ac3b7764eb

which results in "processing failed".

@Kees thanks for poking. Librarian content isn't sync'd to the staging
environment- you need to generate new content within that environment.

Ah, well, stuff I manually add to a bug seems to work. I assume there isn't a builder attached to qastaging, so I won't test package builds yet. Thanks!

On Tue, Nov 2, 2010 at 7:58 PM, Kees Cook <email address hidden> wrote:
> Ah, well, stuff I manually add to a bug seems to work. I assume there
> isn't a builder attached to qastaging, so I won't test package builds
> yet. Thanks!

There isn't yet, though we're going to have one soonish. We can check
package builds when we enable it on prod given how fast it is to
disable again.

Thanks for looking!

Was it faster for you?

I won't be able to meaningfully measure this until I can pull giant build logs and debs out of soyuz using it, but for simple bug attachments, it's fine. But then, I could never reproduce the 503s on simple attachments. :P

This is now live; I need to do two further things:
 - back out the 'use the internal name' stuff from the LP APIs
 - do a config change dance to use session_librarianN rather than session from the librarian.

RT filed for the config; and the use the internal name aspect is bug 629804

Just to avoid any doubt here, this isn't meant to actually fix bug 620458, right? This is still perfectly reproducible after the latest LP rollout from today.

Martin, you are right, bug 620458 is still open. There is a branch ready for merging (lp:~adeuring/launchpad/librarian-filealiases) which will again allow webserive clients to access data of private bugs, but it made unfortunately not into our devel branch berfore the roll out.

...but I'll land it as soon as PQM is open again