Bazaar

getting a stacked branch over the smart protocol fails with "Could not install revisions"

Bug #261315 reported by Michael Hudson-Doyle
2
Affects Status Importance Assigned to Milestone
Bazaar
Fix Released
Critical
Martin Pool
Bazaar 1.7

Bug Description

This shell transcript and ~/.bzr.log excerpt demonstrates a way to reproduce:

mwh@grond:branching-stacked-branches$ rm -rf ~/tmp/branching-stacked-branches
mwh@grond:branching-stacked-branches$ mkdir -p ~/tmp/branching-stacked-branches
mwh@grond:branching-stacked-branches$ cd ~/tmp/branching-stacked-branches
mwh@grond:branching-stacked-branches$ bzr init base --1.6
mwh@grond:branching-stacked-branches$ cd base
mwh@grond:base$ bzr ci -m . --unchanged
Committing to: /home/mwh/tmp/branching-stacked-branches/base/

You need a passphrase to unlock the secret key for
user: "Michael Hudson <email address hidden>"
1024-bit DSA key, ID 6EC0EE48, created 2007-09-28

Committed revision 1.
mwh@grond:base$ cd ..
mwh@grond:branching-stacked-branches$ bzr push -d base bzr+ssh://localhost/$(pwd)/stacked --stacked-on base
Created new stacked branch referring to file:///home/mwh/tmp/branching-stacked-branches/base.
mwh@grond:branching-stacked-branches$ bzr get -Dhpss bzr+ssh://localhost/$(pwd)/stacked got
bzr: ERROR: Could not install revisions:
<email address hidden>
mwh@grond:branching-stacked-branches$ cat /home/mwh/.bzr.log

this is a debug log for diagnosing/reporting problems in bzr
you can delete or truncate this file, or include sections in
bug reports to https://bugs.launchpad.net/bzr/+filebug

0.059 encoding stdout as sys.stdout encoding 'UTF-8'
0.059 bzr arguments: [u'init', u'base', u'--1.6']
0.060 looking for plugins in /home/mwh/.bazaar/plugins
0.096 bzr-svn: using Subversion 1.4.6 ()
0.191 looking for plugins in /usr/lib/python2.5/site-packages/bzrlib/plugins
0.192 Plugin name __init__ already loaded
0.192 Plugin name __init__ already loaded
0.192 Plugin name gtk already loaded
0.192 Plugin name pqm already loaded
0.199 encoding stdout as sys.stdout encoding 'UTF-8'
0.218 opening SVN RA connection to 'file:///home/mwh/tmp/branching-stacked-branches/base'
0.220 Unable to open <bzrlib.transport.local.LocalTransport url=file:///home/mwh/tmp/branching-stacked-branches/base/> with Subversion: Unable to open an ra_local session to URL
0.224 opening SVN RA connection to 'file:///home/mwh/tmp/branching-stacked-branches'
0.226 Unable to open <bzrlib.transport.local.LocalTransport url=file:///home/mwh/tmp/branching-stacked-branches/> with Subversion: Unable to open an ra_local session to URL
0.226 opening SVN RA connection to 'file:///home/mwh/tmp'
0.227 Unable to open <bzrlib.transport.local.LocalTransport url=file:///home/mwh/tmp/> with Subversion: Unable to open an ra_local session to URL
0.228 opening SVN RA connection to 'file:///home/mwh'
0.229 Unable to open <bzrlib.transport.local.LocalTransport url=file:///home/mwh/> with Subversion: Unable to open an ra_local session to URL
0.230 opening SVN RA connection to 'file:///home'
0.231 Unable to open <bzrlib.transport.local.LocalTransport url=file:///home/> with Subversion: Unable to open an ra_local session to URL
0.231 opening SVN RA connection to 'file:'
0.232 Unable to open <bzrlib.transport.local.LocalTransport url=file:///> with Subversion: Unable to open an ra_local session to URL
0.233 creating repository in file:///home/mwh/tmp/branching-stacked-branches/base/.bzr/.
0.236 creating branch <bzrlib.branch.BzrBranchFormat7 object at 0x1132590> in file:///home/mwh/tmp/branching-stacked-branches/base/.bzr/
0.258 trying to create missing lock '/home/mwh/tmp/branching-stacked-branches/base/.bzr/checkout/dirstate'
0.258 opening working tree '/home/mwh/tmp/branching-stacked-branches/base'
0.264 return code 0

0.054 encoding stdout as sys.stdout encoding 'UTF-8'
0.055 bzr arguments: [u'ci', u'-m', u'.', u'--unchanged']
0.055 looking for plugins in /home/mwh/.bazaar/plugins
0.089 bzr-svn: using Subversion 1.4.6 ()
0.181 looking for plugins in /usr/lib/python2.5/site-packages/bzrlib/plugins
0.181 Plugin name __init__ already loaded
0.182 Plugin name __init__ already loaded
0.182 Plugin name gtk already loaded
0.182 Plugin name pqm already loaded
0.188 encoding stdout as sys.stdout encoding 'UTF-8'
0.203 opening working tree '/home/mwh/tmp/branching-stacked-branches/base'
0.214 preparing to commit
[31500] 2008-08-26 09:23:22.671 INFO: Committing to: /home/mwh/tmp/branching-stacked-branches/base/
0.225 Selecting files for commit with filter []
0.242 ** Env var TTY empty, cannot set GPG_TTY. Is TTY exported?
[31500] 2008-08-26 09:23:22.738 INFO: Committed revision 1.
0.295 return code 0

0.054 encoding stdout as sys.stdout encoding 'UTF-8'
0.054 bzr arguments: [u'push', u'-d', u'base', u'bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked', u'--stacked-on', u'base']
0.055 looking for plugins in /home/mwh/.bazaar/plugins
0.090 bzr-svn: using Subversion 1.4.6 ()
0.185 looking for plugins in /usr/lib/python2.5/site-packages/bzrlib/plugins
0.185 Plugin name __init__ already loaded
0.185 Plugin name __init__ already loaded
0.186 Plugin name gtk already loaded
0.186 Plugin name pqm already loaded
0.192 encoding stdout as sys.stdout encoding 'UTF-8'
0.241 ssh implementation is OpenSSH

0.071 encoding stdout as bzrlib.user_encoding 'UTF-8'
0.071 bzr arguments: [u'serve', u'--inet', u'--directory=/', u'--allow-writes']
0.072 looking for plugins in /home/mwh/.bazaar/plugins
0.105 bzr-svn: using Subversion 1.4.6 ()
0.195 looking for plugins in /usr/lib/python2.5/site-packages/bzrlib/plugins
0.195 Plugin name __init__ already loaded
0.195 Plugin name __init__ already loaded
0.195 Plugin name gtk already loaded
0.196 Plugin name pqm already loaded
0.201 encoding stdout as bzrlib.user_encoding 'UTF-8'
0.588 creating repository in bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/.
0.611 creating branch <bzrlib.branch.BzrBranchFormat7 object at 0x7fc903278790> in bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/
0.679 fetch up to rev {<email address hidden>}
[31509] 2008-08-26 09:23:23.516 INFO: Created new stacked branch referring to file:///home/mwh/tmp/branching-stacked-branches/base.
0.737 return code 0
0.392 return code 0

0.065 encoding stdout as sys.stdout encoding 'UTF-8'
0.066 bzr arguments: [u'get', u'-Dhpss', u'bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked', u'got']
0.067 looking for plugins in /home/mwh/.bazaar/plugins
0.105 bzr-svn: using Subversion 1.4.6 ()
0.198 looking for plugins in /usr/lib/python2.5/site-packages/bzrlib/plugins
0.199 Plugin name __init__ already loaded
0.199 Plugin name __init__ already loaded
0.199 Plugin name gtk already loaded
0.199 Plugin name pqm already loaded
0.205 encoding stdout as sys.stdout encoding 'UTF-8'
0.215 hpss: Built a new medium: SmartSSHClientMedium
0.218 hpss call: 'BzrDir.open', '/home/mwh/tmp/branching-stacked-branches/stacked/'
0.218 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.247 ssh implementation is OpenSSH

0.062 encoding stdout as bzrlib.user_encoding 'UTF-8'
0.062 bzr arguments: [u'serve', u'--inet', u'--directory=/', u'--allow-writes']
0.062 looking for plugins in /home/mwh/.bazaar/plugins
0.096 bzr-svn: using Subversion 1.4.6 ()
0.189 looking for plugins in /usr/lib/python2.5/site-packages/bzrlib/plugins
0.190 Plugin name __init__ already loaded
0.190 Plugin name __init__ already loaded
0.190 Plugin name gtk already loaded
0.190 Plugin name pqm already loaded
0.196 encoding stdout as bzrlib.user_encoding 'UTF-8'
0.666 result: ('yes',)
0.666 hpss call: 'BzrDir.open_branch', '/home/mwh/tmp/branching-stacked-branches/stacked/'
0.666 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.669 result: ('ok', '')
0.669 hpss call: 'BzrDir.find_repositoryV2', '/home/mwh/tmp/branching-stacked-branches/stacked/'
0.669 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.671 result: ('ok', '', 'no', 'no', 'yes')
0.671 hpss call: 'Branch.last_revision_info', '/home/mwh/tmp/branching-stacked-branches/stacked/'
0.671 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.685 result: ('ok', '1', '<email address hidden>')
0.687 hpss call: 'get', '//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/branch-format'
0.687 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.689 result: ('ok',)
0.689 35 body bytes read
0.689 hpss call: 'get', '//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/branch/format'
0.689 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.690 result: ('ok',)
0.690 39 body bytes read
0.691 hpss call: 'stat', '//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/branch'
0.691 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.691 result: ('stat', '4096', '040755')
0.692 hpss call: 'get', '//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/repository/format'
0.692 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.693 result: ('ok',)
0.693 43 body bytes read
0.693 hpss call: 'stat', '//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/repository'
0.693 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.694 result: ('stat', '4096', '040755')
0.698 hpss call: 'get', '//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/branch/branch.conf'
0.698 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.700 result: ('ok',)
0.700 75 body bytes read
0.707 hpss call: 'get', '//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/checkout/format'
0.707 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.709 result: ('NoSuchFile', '/home/mwh/tmp/branching-stacked-branches/stacked/.bzr/checkout/format')
0.713 hpss call: 'BzrDir.open_branch', '/home/mwh/tmp/branching-stacked-branches/stacked/'
0.713 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.714 result: ('ok', '')
0.714 hpss call: 'BzrDir.find_repositoryV2', '/home/mwh/tmp/branching-stacked-branches/stacked/'
0.714 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.716 result: ('ok', '', 'no', 'no', 'yes')
0.728 opening SVN RA connection to 'file:///home/mwh/tmp/branching-stacked-branches'
0.731 Unable to open <bzrlib.transport.local.LocalTransport url=file:///home/mwh/tmp/branching-stacked-branches/> with Subversion: Unable to open an ra_local session to URL
0.732 opening SVN RA connection to 'file:///home/mwh/tmp'
0.733 Unable to open <bzrlib.transport.local.LocalTransport url=file:///home/mwh/tmp/> with Subversion: Unable to open an ra_local session to URL
0.733 opening SVN RA connection to 'file:///home/mwh'
0.735 Unable to open <bzrlib.transport.local.LocalTransport url=file:///home/mwh/> with Subversion: Unable to open an ra_local session to URL
0.735 opening SVN RA connection to 'file:///home'
0.737 Unable to open <bzrlib.transport.local.LocalTransport url=file:///home/> with Subversion: Unable to open an ra_local session to URL
0.737 opening SVN RA connection to 'file:'
0.739 Unable to open <bzrlib.transport.local.LocalTransport url=file:///> with Subversion: Unable to open an ra_local session to URL
0.739 creating repository in file:///home/mwh/tmp/branching-stacked-branches/got/.bzr/.
0.745 hpss call: 'get', '//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/repository/format'
0.745 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.746 result: ('ok',)
0.747 43 body bytes read
0.747 hpss call: 'stat', '//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/repository'
0.747 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.748 result: ('stat', '4096', '040755')
0.750 hpss call: 'get', '//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/repository/pack-names'
0.751 (to bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/)
0.753 result: ('ok',)
0.753 60 body bytes read
0.754 Using fetch logic to copy between KnitPackRepository('bzr+ssh://localhost//home/mwh/tmp/branching-stacked-branches/stacked/.bzr/repository/')(<RepositoryFormatKnitPack5>) and KnitPackRepository('file:///home/mwh/tmp/branching-stacked-branches/got/.bzr/repository/')(<RepositoryFormatKnitPack5>)
0.762 Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 857, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 797, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 499, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 866, in run
    hardlink=hardlink, stacked=stacked)
  File "/usr/lib/python2.5/site-packages/bzrlib/bzrdir.py", line 1078, in sprout
    result_repo.fetch(source_repository, revision_id=revision_id)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 975, in fetch
    find_ghosts=find_ghosts)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 3053, in fetch
    find_ghosts=find_ghosts)
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/repository.py", line 2797, in fetch
    raise errors.InstallFailed([revision_id])
InstallFailed: Could not install revisions:
<email address hidden>

0.763 return code 3
0.306 return code 0

I don't really know where the problem is at all -- somehow the server isn't looking in the stacked on repository for revisions?

Related branches

lp:~mbp/bzr/261315-stacked-hpss (Merged)
Aaron Bentley (abentley)
Changed in bzr:
importance: Undecided → Critical
status: New → Triaged
Martin Pool (mbp)
Changed in bzr:
assignee: nobody → mbp
Revision history for this message
Martin Pool (mbp) wrote :

I can reproduce it using mwh's script.

The InterPackRepo's source is a KnitPackRepository which (unsurprisingly but incorrectly) has no fallback repositories configured; this would explain why we can't find the revision. We're using InterPackRepo because the current InterRemoteToOther always gets the real repository and fetches through that.

I suspect the problem is that the repository is somewhere being opened directly, rather than through the branch object, which is responsible for configuring the fallbacks into the repository when it is opened.

Going up the stack, I see the RemoteRepository also does not have fallback repositories configured, which is probably how it should be if this is to be hidden on the server side.

Changed in bzr:
status: Triaged → In Progress
Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 261315] Re: getting a stacked branch over the smart protocol fails with "Could not install revisions"

My intent for this case was that the client would fetch
 - some data from the remote server
 - some data from the basis

itself - explicitly.

I thought this was better because it avoids having the mess of
pass-credentials-to-untrusted-server, FTP site-site copy style of
approach.

-Rob

Revision history for this message
Martin Pool (mbp) wrote : Re: [Bug 261315] Re: getting a stacked branch over the smart protocol fails with "Could not install revisions"

@robertc, thanks for that, that is what I thought on consideration.
So the problem seems to lie in initially getting a Repository with no
fallbacks configured. I'll keep investigating tomorrow.

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Ah, so does that mean that the fact that the smart server tries to open the basis branch too is a bug?

Revision history for this message
Martin Pool (mbp) wrote :

@mwh, I think it does.

I proposed this documentation of how it's meant to work <http://bundlebuggy.aaronbentley.com/project/bzr/request/%3C20080904053630.600CA3901B7%40grace.sourcefrog.net%3E> and Robert agrees with it.

Revision history for this message
Martin Pool (mbp) wrote :

At the time we open the RemoteBranch it needs to get the fallback repository from the server (if any), and open that too. That isn't explicitly done at present, but it seems to me it might be happening implicitly inside the vfs-based branch and repository.

The real repository does not have the right fallback, but the _real_branch of a remotebranch does contain it, though it has a file URL rather than an ssh url, which is a bit strange. This is because the branch.conf contains the stacked_on_location as a full url, not a relative path. If that's always being created it might be a separate bug.

I suspect the _real_repository is being created without the _real_branch ever getting a chance to poke fallback repositories into it.

Revision history for this message
Martin Pool (mbp) wrote :

We also need to decide if the fallback repositories should be seen at the RemoteRepository level, or only inside the _real_repository, or both. It seems to me like the first is better; the last might cause confusion with the repository being searched twice.

I'm kind of surprised there is not already a test failing for this: we should have a per-repository test for stacking, and the scenarios ought to be configuring that to be accessed across a remote repository.

We don't seem to actually be calling RemoteBranch._ensure_real, so my first attempt at configuring the fallbacks using the _real_branch isn't working.

Hooking this in at the creation of the RemoteBranch does seem to work so now I'm going to see why it's not tested, or how to test it.

Revision history for this message
Martin Pool (mbp) wrote :

fix posted for review

http://bundlebuggy.aaronbentley.com/project/bzr/request/%3Ce01316480809050416r72b3f857h4a664a2cc4e157b0%40mail.gmail.com%3E

Changed in bzr:
status: In Progress → Fix Committed
Revision history for this message
Martin Pool (mbp) wrote :

Ok, so the patch posted does fix this particular bug, but it causes several test failures in test_remote, because it's changing the rpcs that are issued when opening a branch. I'll update it monday or on the weekend.

Changed in bzr:
status: Fix Committed → In Progress
Revision history for this message
Martin Pool (mbp) wrote :

https://code.launchpad.net/~mbp/bzr/261315-stacked-hpss has a partial fix, which adds a new rpc Branch.get_stacked_on_url and uses it when opening the branch.

However, this will fail pretty miserably on an old server because it can't cope if that rpc is not supported.

Revision history for this message
Martin Pool (mbp) wrote :

ok, i think the real fix is now in both 1.7 and trunk

Changed in bzr:
status: In Progress → Fix Released
Revision history for this message
Jonathan Lange (jml) wrote :

I don't see it in trunk (http://bazaar-vcs.org/bzr/bzr.dev/ right?)

John A Meinel (jameinel)
Changed in bzr:
milestone: none → 1.7
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.