Plan and UI for upgrading multiple stacked branches

Some draft content which highlights some issues.

Upgrading to a rich root or 2.0 format
++++++++++++++++++++++++++++++++++++++

Overall process
===============

* Test that your code migrates
* Schedule a migration time and tell all users
* Users migrate in advance of trunk
* Backup
* Migrate
* Rollback if necessary

Test migration
==============

* Run 'bzr check'. If there are errors try
  * 'bzr reconcile' and 'bzr check' to see if they are known
    issues bzr can fix
  * Otherwise file a bug at https://bugs.launchpad.net/bzr
* Take a copy of trunk now that it is clean. If you have a shared
repository you need to branch into a new shared repository or standalone
branch before doing the upgrade.
* Run 'bzr upgrade --<format>' to upgrade to that format.
* Run 'bzr check'. If there are errors, file a bug.

Schedule a time
===============

All stacking branches *must* upgrade before your trunk branch (or
whatever branch they stacked on). So its very important to give users
plenty of time to upgrade. If you have a tight knit community or are not
using stacked branches you can likely do this in 24 hours or so. If the
community is more widespread leave more time. If you are using launchpad
you will be using stacked branches. Launchpad cannot yet upgrade these
for you, and once upgraded a stacked branch cannot be used until the
trunk upgrades - another key reason to advertise this process widely.

Users migrate
=============

Users should follow the 'do a test migration' instructions for each
branch they have.

If branches fail, file bugs and the user should alert the coordinator
that the migration can't go ahead.

If you have stacked branches, you currently cannot migrate.

A backup should be taken of repositories/branches before backing up.

Backup
======

Take a full backup of your trunk and project branches.

Migrate
=======

* Follow the migration checklist again (new commits need checking too).
Instead of copying before upgrading though, just upgrade in-place.

Rollback
========

* Check things are working well for all users. If not, rollback before
people start committing in the new format: once they start you cannot
roll back.

 Valid signature

Errata, upgrading stacked branches post hoc is now possible. Bug https://bugs.edge.launchpad.net/launchpad-code/+bug/385815

Comments from thumper on one process option ...

(11:19:30) thumper: one person gets trunk
(11:19:32) thumper: reconciles
(11:19:34) thumper: converts
(11:19:44) thumper: locally
(11:19:50) thumper: unsets dev focus branch
(11:19:57) thumper: pushes new format 2 branch to LP
(11:20:05) thumper: sets dev focus to new branch
(11:20:07) thumper: done
(11:20:13) thumper: users
(11:20:19) thumper: resubsrcibe to new trunk
(11:20:49) thumper: also grab new trunk into a new area
(11:20:57) thumper: branch old local branches into new repo

The proposed Upgrade Guide can be found here: http://people.ubuntu.com/~ianc/doc/en/upgrade-guide/

We may want some more UI work here, e.g. I have a branch in development for making upgrade of a shared repository easier. A nicer way of resubscribing users from the old trunk to the new one would be good too. (Tim has suggested an lplib script instead of an LP UI tweak, given the "one-off" nature of this.)

I think LP support is needed because the trunk will likely have *many*
attached bugs etc that need to be moved over to the new trunk.

Documentation has landed but it describes making new trunks and branches - this leads to various forms of confusion, excess disk space usage and so forth. Bzr supports in-place upgrades, we should recommend that.

As long as stacking requires formats to be kept in sync across the base and stacked branch, upgrades will be semi-ugly.

Is it really as simple as 'bzr upgrade URL' for a project trunk that has other branches stacked on it? Will those other branches continue to work? If not, then they must be upgraded as well, right? Until they are, won't they be broken?

If so, then how does one find out what those impacted branches are? LP will need a feature to tell me those branches and who owns them. Owners are potentially anyone on LP, yes? IIUIC, the documented process solves this by simply leaving existing branches alone. So teams can upgrade trunk and just what they want to, when they want to. Isn't that better than putting teams into a situation where things they can't control will become broken?

I appreciate that there are concerns re linked bugs by introducing a fresh trunk. OTOH, listing hundreds of bugs linked to trunk as LP does now is, well, questionable at best from a UI perspective. If this proves a real problem, then LP could gain a feature for moving linked bugs from one branch to another, say.

On Wed, 2009-08-12 at 05:58 +0000, Ian Clatworthy wrote:
> As long as stacking requires formats to be kept in sync across the base
> and stacked branch, upgrades will be semi-ugly.
>
> Is it really as simple as 'bzr upgrade URL' for a project trunk that has
> other branches stacked on it? Will those other branches continue to
> work? If not, then they must be upgraded as well, right? Until they are,
> won't they be broken?

They will be broken while the branch is upgraded.

> If so, then how does one find out what those impacted branches are? LP
> will need a feature to tell me those branches and who owns them. Owners
> are potentially anyone on LP, yes? IIUIC, the documented process solves
> this by simply leaving existing branches alone. So teams can upgrade
> trunk and just what they want to, when they want to. Isn't that better
> than putting teams into a situation where things they can't control will
> become broken?

folk with existing branches will have to update them anyway; and if
users on lp pivot the branch names around (so that they end up with
'trunk' again) the breakage of downstream branches will happen *anyway*.

> I appreciate that there are concerns re linked bugs by introducing a
> fresh trunk. OTOH, listing hundreds of bugs linked to trunk as LP does
> now is, well, questionable at best from a UI perspective. If this proves
> a real problem, then LP could gain a feature for moving linked bugs from
> one branch to another, say.

There is a bug on LP for an upgrade UI for branches anyway - it makes a
lot of sense for upgrading a project to 2a to be a button push in the
launchpad UI.

The problems as I see with the current documented process are that:
 - old branches won't get upgraded and will thus stay around forever, in
an incompatible format
 - bugs / merges / translations will still keep refering to old format
branches
 - the upgrade will cause a significant jump in disk usage (because the
old format trunk is left around).

AIUI there are some operational concerns about disk capacity on
launchpad anyway - having usage jump by 30% when everyone upgrades would
be a significant problem. Upgrading in place won't do that, because the
old branch data will be replaced (as further operations take place,
assuming they use your clean-up-aftwards flag). Even if they don't use
said flag, we could remove backup.bzr automatically (after health
checking the branch or something).

-Rob

This remains high, but after discussion with igc I'm untargeting it from 2.0. The upgrade guide has some nice documentation.

The Upgrade Guide has been improved by Barry since the initial draft. I don't believe further work is required on it at this time. If additional changes are required for Launchpad or Bazaar, we can open new, more explicit bugs accordingly.