Branch rewrite map should switch to master db when lag gets too high

Bug #435783 reported by Martin Pool
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Triaged
Low
Unassigned

Bug Description

I don't know what's going on here but it's pretty strange

https://code.edge.launchpad.net/~bzr/bzr-website/trunk/ - works
http://bazaar.launchpad.net/~bzr/bzr-website/trunk - shows up ok in loggerhead
bzr info log+http://bazaar.launchpad.net/~bzr/bzr-website/trunk/ - fails with "not a branch" with this trace
bzr info log+sftp://<email address hidden>/~bzr/bzr-website/trunk/ -- works
bzr info log+sftp://<email address hidden>/~bzr/bzr-website/trunk/ -- also works

Thu 2009-09-24 19:34:32 +1000
0.085 bzr arguments: [u'info', u'log+http://bazaar.launchpad.net/~bzr/bzr-website/trunk/']
0.110 looking for plugins in /home/mbp/.bazaar/plugins
0.281 looking for plugins in /home/mbp/bzr/trunk/bzrlib/plugins
0.288 looking for plugins in /usr/lib/python2.6/dist-packages/bzrlib/plugins
0.288 Plugin name qbzr already loaded
0.288 Plugin name launchpad already loaded
0.288 Plugin name netrc_credential_store already loaded
0.292 encoding stdout as sys.stdout encoding 'UTF-8'
0.332 bzr-svn: using Subversion 1.6.5 ()
0.373 get http://bazaar.launchpad.net/~bzr/bzr-website/trunk/.bzr/branch-format
1.271 --> No such file: 'http://bazaar.launchpad.net/~bzr/bzr-website/trunk/.bzr/branch-format'
1.271 0.897s
1.274 get http://bazaar.launchpad.net/~bzr/bzr-website/.bzr/branch-format
1.728 --> No such file: 'http://bazaar.launchpad.net/~bzr/bzr-website/.bzr/branch-format'
1.729 0.455s
1.731 get http://bazaar.launchpad.net/~bzr/.bzr/branch-format
2.127 --> No such file: 'http://bazaar.launchpad.net/~bzr/.bzr/branch-format'
2.128 0.397s
2.130 get http://bazaar.launchpad.net/.bzr/branch-format
2.505 --> No such file: 'http://bazaar.launchpad.net/.bzr/branch-format'
2.505 0.375s
2.517 Traceback (most recent call last):
  File "/home/mbp/bzr/trunk/bzrlib/commands.py", line 842, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/home/mbp/bzr/trunk/bzrlib/commands.py", line 1037, in run_bzr
    ret = run(*run_argv)
  File "/home/mbp/bzr/trunk/bzrlib/commands.py", line 654, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/home/mbp/bzr/trunk/bzrlib/commands.py", line 1052, in ignore_pipe
    result = func(*args, **kwargs)
  File "/home/mbp/bzr/trunk/bzrlib/builtins.py", line 1468, in run
    show_bzrdir_info(bzrdir.BzrDir.open_containing(location)[0],
  File "/home/mbp/bzr/trunk/bzrlib/bzrdir.py", line 899, in open_containing
    return BzrDir.open_containing_from_transport(transport)
  File "/home/mbp/bzr/trunk/bzrlib/bzrdir.py", line 931, in open_containing_from_transport
    raise errors.NotBranchError(path=url)
NotBranchError: Not a branch: "log+http://bazaar.launchpad.net/~bzr/bzr-website/trunk/".

Tags: lp-code
Revision history for this message
Martin Pool (mbp) wrote :

I verified this from two different machines. http access to ~bzr/bzr/trunk works ok.

Revision history for this message
Tim Penhey (thumper) wrote : Re: [Bug 435783] [NEW] branch appears in loggerhead but can't be read over http
Download full text (3.5 KiB)

On Thu, 24 Sep 2009 21:37:41 Martin Pool wrote:
> Public bug reported:
>
> I don't know what's going on here but it's pretty strange
>
> https://code.edge.launchpad.net/~bzr/bzr-website/trunk/ - works
> http://bazaar.launchpad.net/~bzr/bzr-website/trunk - shows up ok in
> loggerhead bzr info
> log+http://bazaar.launchpad.net/~bzr/bzr-website/trunk/ - fails with "not
> a branch" with this trace bzr info
> log+sftp://<email address hidden>/~bzr/bzr-website/trunk/ -- works bzr
> info log+sftp://<email address hidden>/~bzr/bzr-website/trunk/ -- also
> works
>
> Thu 2009-09-24 19:34:32 +1000
> 0.085 bzr arguments: [u'info',
> u'log+http://bazaar.launchpad.net/~bzr/bzr-website/trunk/'] 0.110 looking
> for plugins in /home/mbp/.bazaar/plugins
> 0.281 looking for plugins in /home/mbp/bzr/trunk/bzrlib/plugins
> 0.288 looking for plugins in
> /usr/lib/python2.6/dist-packages/bzrlib/plugins 0.288 Plugin name qbzr
> already loaded
> 0.288 Plugin name launchpad already loaded
> 0.288 Plugin name netrc_credential_store already loaded
> 0.292 encoding stdout as sys.stdout encoding 'UTF-8'
> 0.332 bzr-svn: using Subversion 1.6.5 ()
> 0.373 get
> http://bazaar.launchpad.net/~bzr/bzr-website/trunk/.bzr/branch-format
> 1.271 --> No such file:
> 'http://bazaar.launchpad.net/~bzr/bzr-website/trunk/.bzr/branch-format'
> 1.271 0.897s
> 1.274 get http://bazaar.launchpad.net/~bzr/bzr-website/.bzr/branch-format
> 1.728 --> No such file:
> 'http://bazaar.launchpad.net/~bzr/bzr-website/.bzr/branch-format' 1.729
> 0.455s
> 1.731 get http://bazaar.launchpad.net/~bzr/.bzr/branch-format
> 2.127 --> No such file:
> 'http://bazaar.launchpad.net/~bzr/.bzr/branch-format' 2.128 0.397s
> 2.130 get http://bazaar.launchpad.net/.bzr/branch-format
> 2.505 --> No such file: 'http://bazaar.launchpad.net/.bzr/branch-format'
> 2.505 0.375s
> 2.517 Traceback (most recent call last):
> File "/home/mbp/bzr/trunk/bzrlib/commands.py", line 842, in
> exception_to_return_code return the_callable(*args, **kwargs)
> File "/home/mbp/bzr/trunk/bzrlib/commands.py", line 1037, in run_bzr
> ret = run(*run_argv)
> File "/home/mbp/bzr/trunk/bzrlib/commands.py", line 654, in
> run_argv_aliases return self.run(**all_cmd_args)
> File "/home/mbp/bzr/trunk/bzrlib/commands.py", line 1052, in ignore_pipe
> result = func(*args, **kwargs)
> File "/home/mbp/bzr/trunk/bzrlib/builtins.py", line 1468, in run
> show_bzrdir_info(bzrdir.BzrDir.open_containing(location)[0],
> File "/home/mbp/bzr/trunk/bzrlib/bzrdir.py", line 899, in open_containing
> return BzrDir.open_containing_from_transport(transport)
> File "/home/mbp/bzr/trunk/bzrlib/bzrdir.py", line 931, in
> open_containing_from_transport raise errors.NotBranchError(path=url)
> NotBranchError: Not a branch:
> "log+http://bazaar.launchpad.net/~bzr/bzr-website/trunk/".
>
> ** Affects: launchpad-code
> Importance: Undecided
> Status: New
>

Weird, works for me.

tim@slacko:~/src/lp/job-fail$ GET http://bazaar.launchpad.net/~bzr/bzr-
website/trunk/.bzr/branch-format
Bazaar-NG meta directory, format 1
tim@slacko:~/src/lp/job-fail$ bzr info
...

Read more...

Revision history for this message
Martin Pool (mbp) wrote : Re: [Bug 435783] [NEW] branch appears in loggerhead but can't be read over http

2009/9/24 Tim Penhey <email address hidden>:
> Weird, works for me.
>
> tim@slacko:~/src/lp/job-fail$ GET http://bazaar.launchpad.net/~bzr/bzr-
> website/trunk/.bzr/branch-format
> Bazaar-NG meta directory, format 1
> tim@slacko:~/src/lp/job-fail$ bzr info
> log+http://bazaar.launchpad.net/~bzr/bzr-website/trunk/
> Repository branch (format: 1.14-rich-root or 1.9-rich-root)
> Location:
>  shared repository: log+http://bazaar.launchpad.net/~bzr/bzr-website/trunk/
>  repository branch: log+http://bazaar.launchpad.net/~bzr/bzr-website/trunk/
>
> Related branches:
>  parent branch: log+http://people.canonical.com/~ianc/dancingmonkeys/

It works for me too now. Could it be there's a cache somewhere along
the line that can get out of sync?

One place I tested from was in the Canonical datacentre.

--
Martin <http://launchpad.net/~mbp/>

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote : Re: branch appears in loggerhead but can't be read over http

There is a cache, but we don't negatively cache at all....

Ah, this was caused by a spike in replication lag around the time you were testing this out -- the rewrite gadget connects to a slave database. Application servers switch over to the main db server when replication lag gets too high, I guess the rewrite map should do that too.

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Oh, and btw, the reason it shows up in loggerhead is basically a fluke -- we can't generate a 404 from the rewrite script, so we pass requests for non-existent urls on to loggerhead to do that.

Paul Hummer (rockstar)
summary: - branch appears in loggerhead but can't be read over http
+ Branch rewrite map should switch to master db when lag gets too high
Changed in launchpad-code:
status: New → Triaged
importance: Undecided → Medium
Curtis Hovey (sinzui)
Changed in launchpad:
importance: Medium → Low
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.