How to check out EPICS Base with Bazaar

Bazaar can support several different approaches; the one you should use depends on who you are and what you expect to do with the code-base. It usually allows you to change your mind later though, so don't worry if you pick the wrong model to start with, you can modify your configuration as you need later on (although I'm not going to give instructions on how to do that here, read the Bazaar documentation).

You must have Bazaar version 2.x installed; older versions do not support the same repository format. I am currently using version 2.0.3, but any release in the 2.x series should be fine.

Bazaar has extensive built-in help text, just type 'bzr help <command>' for more information on <command> or 'bzr help topics' for a list of other help topics. The complete documentation for the tool can be found online here.

If you have problems connecting to launchpad and your network requires you to use an HTTP proxy server, make sure that you have both the http_proxy and https_proxy environment variables set appropriately.

If you are behind a firewall and the 'lp:' shorthand does not work, try the following transport specification:

  bzr checkout bzr+ssh://<lp-user-name>@bazaar.launchpad.net/~epics-core/epics-base/3.14/

Non-developers

These options apply to EPICS users who do not expect to make changes to the source code themselves, but who want to be able to access the latest official development versions of the code or to look at how it has changed between different past revisions.

Check out a working tree (CVS model, remote history)

Use this option if you just want to a copy of the latest version of Base from the repository (for example to build and test) but don't expect to modify the code at all, or to look at its history much. This approach is very similar to using CVS since you only download a copy of the particular revision you're interested in, but it require network connectivity for all activities that access the repository in any way. An EPICS Base 3.14 working tree uses about 12MB of disk space at the time of writing.

The following command will create a directory tree called base-3.14 containing the latest code that was checked in to the 3.14 branch of Base:

  bzr checkout --lightweight lp:epics-base/3.14 base-3.14

The lp: path above specifies a branch to check out from the launchpad server. The branch at lp:epics-base/3.14 is a branch containing all the development work for the R3.14 releases of Base. There are similar branches for the older series, although we're not expecting anyone to check any of these out; they're provided for historical reasons. The development on the branch called trunk will eventually become the 3.15 series.

If you want to check out a specific tagged version, in this case we're requesting the R3.14.11 release, use this command instead:

  bzr checkout --lightweight -r tag:R3.14.11 lp:epics-base/3.14 base-3.14

To update a lightweight working tree at a later date with the latest changes from the branch, use this command inside the tree:

  bzr update

You can use the command 'bzr tags' from inside the working tree to list all the tags known on this branch.

Unlike CVS, 'bzr update' cannot apply changes that move your tree to a different historical revision; for that you use the command below, in this case to move to the R3.14.10 release tag:

  bzr revert -r tag:R3.14.10

If you made any local changes to the files in the working tree your changes will be retained through a 'bzr update' or 'bzr revert' operation. Any local changes that overlap with changes from the repository will create a conflict and have to be resolved by hand, just as happens with CVS.

If you find yourself using Bazaar commands inside the tree more than once or twice, you should probably check out a branch instead, which makes a local copy of the complete history of the branch and removes the need for network activity to access that history.

Check out a branch (local history)

This option is similar to the previous one but also gives you a local copy of the complete history of the branch. This speeds up all activities that need to examine the repository history, but since the local branch is still bound to the central repository at launchpad.net you can't commit any changes to it without write permission for the repository branch. A 3.14 branch needs about 28MB of disk space at the time of writing.

The following command will create a directory tree called base-3.14 containing the latest code and all of the history that was checked in to the 3.14 branch of Base:

  bzr checkout lp:epics-base/3.14 base-3.14

The lp: path above specifies a branch to check out from the launchpad server. The branch at lp:epics-base/3.14 is a branch containing all the development work for the R3.14 releases of Base. There are similar branches for the older series, although we're not expecting anyone to check any of these out; they're provided for historical reasons. The development on the branch called trunk will eventually become the 3.15 series.

If you want to check out a particular tagged version, use this command instead:

  bzr checkout -r tag:R3.14.11 lp:~epics-core/epics-base/3.14 base-3.14.11

Inside the working tree, Bazaar supports the expected commands 'bzr log' and 'bzr diff' to examine file history. Use 'bzr help revisionspec' for information on specifying particular revisions to examine.

To update your working tree with the latest changes from the parent branch at some later date, use this command inside the tree:

  bzr update

Note that if you're working in a regular branch and not a checkout, the command to use is

  bzr pull

and if you have committed changes to your branch that are not in the parent

  bzr merge

Note that 'bzr pull' or 'bzr update' cannot move your tree to a different historical revision like in CVS, for that you use this command:

  bzr revert -r tag:R3.14.10

You can use 'bzr tags' when inside the working tree to list all the tags known on this branch.

If you made any local changes to the files in the working tree your changes will be retained through a 'bzr update' or 'bzr revert' operation. Any local changes that overlap with changes from the repository will create a conflict and have to be resolved by hand, just as happens with CVS.

If you want to keep some permanent local changes, consider unbinding your tree from the central repository branch, which converts it to a regular branch. This will allow you to commit local changes into your local repository. The 'bzr unbind' command does this, and can be undone again with 'bzr bind'.

Developers

Developers usually want to be able to commit changes to a local repository while they're working on a new feature or bug-fix. Once those changes are complete they will publish and propose them for review; once approved a member of the epics-core team will merge the new code into the appropriate central repository and it will appear in the next release of that development series.

Repository Initialization

Unless you are only going to work on one modification to epics-base, you should create a local repository, which will be used as the backing-store to hold the history of all your development work on EPICS Base, and will be shared by all your bazaar branches. Doing this saves network bandwidth, as it avoids having to download the past history of changes to epics-base from Launchpad more than once. This command will create an epics-base directory in your home directory, inside of which you will check out your branches (you can use a different name or location, but it should be a parent or grand-parent directory of where you'll create your task branches):

  bzr init-repo ~/epics-base

Creating a Task Branch

Bazaar encourages developers to create a separate branch for each separate feature or bug fix being developed, which would normally be created as a sibling to the mirror branch we checked out above. While working on the fix you are free to commit changes whenever you like, which are recorded in your local repository:

  cd ~/epics-base
  bzr branch lp:epics-base fix-123
  cd fix-123
  # Hack...
  bzr ci -m 'Hacked the frooz-bodger to stop it wibbling'

If a particular commit fixes a bug that is registered on Launchpad, you can have Bazaar record that fact at the time like this:

  bzr ci --fixes=lp:123

When you are sure that your branch fully implements the new feature or bug-fix that you've been working on and you want the core developers to take a look at it, you should publish it on launchpad and propose it for merging (see #Publishing a Branch below).

Updates and Merging

While you're working on your task branch, other developers' changes may be being merged into the central branch. In most cases there will be little conflict between their work and yours, thus you should not normally merge changes from the central branch into yours before your branch has been proposed for merger and reviewed. Merging changes from the central branch before the review will add all those changes to your merge proposal, which makes the reviewers' task harder. Minor conflicts are to be expected in the release notes, but these will be fixed by the core developer who merges your branch after it has been approved.

Pulling

If you have not committed any changes to your fix-123 branch yet, you can pull all the central changes into your branch in a single command:

  cd fix-123
  bzr pull
  # Should report any conflicts

If you're not sure whether you've made any commits or not, you can ask Bazaar:

  cd fix-123
  bzr missing
  # Reports commit differences between trees

Merging Updates

If a change does get merged to the central branch which significantly affects your work (and you have already made at least one commit to your branch), you should bring this to the attention of the core developers and ask them how to proceed. If they tell you to merge in from the central branch, make sure you have no uncommitted changes (by either committing or shelving them), then:

  cd fix-123
  bzr merge
  # Fix any conflicts, then tell Bazaar like this:
  bzr resolve file
  # Finally commit the merge
  bzr ci -m 'Merged with latest 3.14 changes'

If Bazaar reports any conflicts from the 'bzr merge' command, you must resolve them with 'bzr resolve' before attempting to commit the result. You will find a pristine copy of your version in file.THIS, the other version in file.OTHER and the common ancestor in file.BASE, and collectively these should help you to see what's happening. The command 'bzr remerge' might also be useful in some circumstances, it allows you to redo the merge using some other merge algorithms. The .THIS, .OTHER and .BASE files will be deleted when you run the 'bzr resolve' command.

If you decide that a merge was a mistake and want to undo it, the 'bzr revert' command will undo all uncommitted changes and restore your tree to the way it was after the last commit.

Rebasing a Private Branch

If it is suggested that you rebase your private branch rather than merging the changes from the central branch into it, you will need the rebase Bazaar plugin installed (also now called the rewrite plugin). Use 'bzr plugins' to find out if it is already present or not. If not, you can find it at Rewrite Plugin.

FIXME: Usage instructions for 'bzr rebase' go here...

Shelving Local Changes

If you have some local changes that you don't want to commit (say some CONFIG_SITE changes needed to build the code at your site) but you're about to use a command such as 'bzr merge' that doesn't work if there uncommitted changes, you can use 'bzr shelve configure/CONFIG_SITE' to temporarily remove them, and 'bzr unshelve' later to bring them back. The shelve command will show you each unsaved change in the named file(s) in turn and ask whether you want to shelve it or not (the '--all' option skips that part). Read the 'bzr help shelve' output for more details.

Publishing a Branch

Anyone can publish branches of the epics-base project on Launchpad, where other people can find and use them. You must have registered for an account at Launchpad.net first and uploaded your ssh public-key. Then:

  bzr launchpad-login userid
  cd ~/epics-base/fix-123
  bzr push lp:~user-id/epics-base/fix-123

Core Developers

In addition to developing your own features and fixes, if you are a member of the epics-core team you will be able to merge changes (either your own or someone else's) into the official branches in the central repository. That task is simplified if you keep a checkout (mirror) of the relevant central branch, which you can also use to directly commit tiny changes to the repository.

The model described here corresponds to the "decentralized with shared mainline" workflow given in the Bazaar User Guide.

Initial Checkout

First initialize a local repository in a parent directory which will be used as the backing-store to hold the history of all your development work on EPICS Base. Make sure you are logged into Launchpad, then checkout a mirror branch of Base inside your repository directory:

  bzr init-repo ~/epics-base
  bzr launchpad-login userid
  cd ~/epics-base
  bzr checkout lp:epics-base/3.14 mirror-3.14

This mirror branch should not be used directly for developing new code or major bug fixing, but can be used for making "obviously correct" changes to things like comments or string literals. Major work should take place in a different tree usually called a feature or task branch.

The mirror branch allows you to merge tested sets of changes from your feature branches into the central repository, or to try out merging other branches while performing code reviews. Remember that performing a commit in the mirror branch is only possible if you are an epics-core member, and will immediately push the change to the official branch on the Launchpad website.

Creating a Task Branch

Bazaar encourages developers to create a separate branch for each separate feature or bug fix being developed, which would normally be created as a sibling to the mirror branch we checked out above. While working on the fix you are free to commit changes whenever you like, which are recorded in your local repository:

  cd ~/epics-base
  bzr branch mirror-3.14 fix-123
  cd fix-123
  # Hack...
  bzr ci -m 'Hacked the frooz-bodger to stop it wibbling'

If a particular commit fixes a bug that is registered on Launchpad, you should have Bazaar record that fact at the time like this:

  bzr ci --fixes=lp:123

When you are sure that your branch fully implements the new feature or bug-fix that you've been working on and you want the other core developers to review it, you should publish it on launchpad and propose it for merging (see #Publishing a Branch below).

Updating the Mirror

Update your mirror branch frequently to fetch changes from the central repository. Since you aren't using this checkout area for development there should never be any conflicts when you do this:

  cd ~/epics-base
  bzr update mirror-3.14

Pulling Changes

If you have not committed any changes to one or more of your feature branches, you can pull the changes from the update into those branches very easily, and Bazaar will merge them with your uncommitted edits. After updating the mirror-3.14 tree as above, you just do this:

  cd fix-123
  bzr pull
  # Should report any conflicts

If you're not sure whether you've made any commits or not, you can ask Bazaar:

  cd fix-123
  bzr missing
  # Reports commit differences between trees

Updates and Merging

While you're working on your task branch, other developers' changes may be being merged into the central branch. In most cases there will be little conflict between their work and yours, thus you should not normally merge changes from the central branch into yours before your branch has been proposed for merger and reviewed. Merging changes from the central branch before the review will add all those changes to your merge proposal, which makes the reviewers' task harder. Minor conflicts are to be expected in the release notes, but they should only be fixed when you merge in your branch after it has been approved.

Merging vs. Rebasing

FIXME: Pointers to the debate about using rebase, describe when it's appropriate...

Merging Updates

If you decide to merge in the central branch changes into your work, the steps are:

  cd ~/epics-base/feature
  bzr merge
  # Fix any conflicts, then tell Bazaar like this:
  bzr resolve file
  # Finally commit the merge
  bzr ci -m 'Merged changes from 3.14 branch'

Note that you should either shelve (see below) or check in any uncommitted local modifications into the task branch before doing the above merge step, if you try to use 'bzr merge --force' instead you risk mixing up your changes with someone else's, which can be confusing and tricky to resolve.

If Bazaar reports any conflicts from the 'bzr merge' command, you must resolve them with 'bzr resolve' before attempting to commit the result. You will find a pristine copy of your version in file.THIS, the other version in file.OTHER and the common ancestor in file.BASE, and collectively these should help you to see what's happening. The command 'bzr remerge' might also be useful in some circumstances, it allows you to redo the merge using some other merge algorithms. The .THIS, .OTHER and .BASE files will be deleted when you run the 'bzr resolve' command.

If you decide that the merge was a mistake and want to undo it, the 'bzr revert' command will undo all uncommitted changes and restore your tree to the way it was after the last commit.

Rebasing a Private Branch

If it is suggested that you rebase your private branch rather than merging the changes from the central branch into it, you will need the rebase Bazaar plugin installed (also now called the rewrite plugin). Use 'bzr plugins' to find out if it is already present or not. If not, you can find it at Rewrite Plugin.

FIXME: Usage instructions for 'bzr rebase' go here...

Merging Task or Feature Branches

The ability to commit changes to the central repository is restricted to members of the epics-core team at Launchpad.net, but Bazaar and Launchpad will still allow you to publish branches containing your own development even if you do not belong to that team; a core developer may subsequently merge and commit the changes from your branch into the official one.

Note that you can undo a merge anytime before the final commit by using the 'bzr revert' command.

Merging a Local Branch

For epics-core members, a feature branch of your own that has been reviewed and approved is merged and committed to the central branch as follows:

  cd ~/epics-base/mirror-3.14
  bzr update
  bzr missing ../fix-123
  # Check that you're about to merge the right changes
  bzr merge ../fix-123
  # Fix any conflicts, then tell Bazaar like this:
  bzr resolve file
  # Finally commit the merge
  bzr ci -m 'Fixed bug #123'

After you have merged and committed your changes from a feature branch into the central branch(es) at launchpad, you can delete the branch directory, you don't have to do anything special with Bazaar.

Merging a Launchpad Branch

If you're merging someone else's branch which is published on Launchpad, you can use the lp:~user/epics-base/branch-name URL for their branch directly, it is not necessary to download or checkout a copy of a branch to merge it:

  cd ~/epics-base/mirror-3.14
  bzr update
  bzr missing lp:~mdavidsaver/epics-base/rec-init
  # Check that you're about to merge the right changes
  bzr merge lp:~mdavidsaver/epics-base/rec-init
  # Fix any conflicts, then tell Bazaar like this:
  bzr resolve file
  # Finally commit the merge
  bzr ci -m "Merged Michael Davidsaver's 'rec-init' branch."

Cherry-picking

This is not recommended, discuss on core-talk before you use cherry-picking...

To apply a bug-fix change to more than one series branch, say to both 3.14 and the main trunk, Bazaar's merge command allows you to cherry-pick commits from a different branch. If it's just one check-in this is very easy, especially if it's the last commit you made to that source branch. In that case, you would use these commands to merge those changes to the trunk:

  cd ~/epics-base/mirror-trunk
  bzr update
  bzr merge -c -1 ../fix-123
  # Fix any conflicts and tell bzr like this:
  bzr resolve file
  # Finally commit the merge
  bzr ci -m 'Fixed bug #123'

The '-c -1' option to 'bzr merge' means "cherry-pick just the last commit", '-c -2' would be the penultimate commit and so on. You can use 'bzr log' in the feature branch to find a specific absolute revision number on that branch and use that instead of the negative numbers. If there were several commits you can give 'bzr merge' a range using the option '-r 44..47' instead, but note that '-r 44' means "merge all changes up to and including revision 44." If you get it wrong, you can always use 'bzr revert' to undo.

Shelving Local Changes

If you have some local changes that you don't want to commit, say some CONFIG_SITE changes needed to build the code at your site, you can use 'bzr shelve configure/CONFIG_SITE' to temporarily hide them before committing or merging, and 'bzr unshelve' later to bring them back. The shelve command will show you each of the changes in the named file(s) in turn and ask whether you want to shelve it or not. Read the 'bzr help shelve' output for more details.

Publishing a Branch

Anyone can publish branches of the epics-base project on Launchpad, where other people can find and use them. You must have registered for an account at Launchpad.net first and uploaded your ssh public-key. Then:

  bzr launchpad-login userid
  cd ~/epics-base/fix-123
  bzr push lp:~user-id/epics-base/fix-123

Members of the epics-core team can create shared branches that other team members can also commit to by using the group name as the user-id in the Bazaar target path:

  bzr push lp:~epics-core/epics-base/feature-xyz