How to check out EPICS Base with Bazaar
Bazaar can support several different approaches; the one you should use depends on who you are and what you expect to do with the code-base. It usually allows you to change your mind later though, so don't worry if you pick the wrong model to start with, you can modify your configuration as you need later on (although I'm not going to give instructions on how to do that here, read the Bazaar documentation).
You must have a recent copy of Bazaar version 2.x installed, preferably 2.5 or 2.6 although earlier versions will still work.
Bazaar has extensive built-in help text, just type 'bzr help <command>' for more information on <command>, 'bzr help commands' for a list of all the commands your installation knows or 'bzr help topics' for a list of other help topics. The complete documentation for all version of the tool can be found online here.
Each separate project branch stored in a Bazaar repository at Launchpad has a URL which can take one of several forms. The short-hand form for the EPICS Base project is just 'lp:epics-base' which refers to the default (development, currently 3.15) branch. You can also look at the 3.14 branch using 'lp:epics-base/3.14' or explicitly at the 3.15 branch using 'lp:epics-base/3.15'.
If you have a Launchpad account but don't want to use it (or it gives you errors) you can make read-only checkouts of Base by giving an http: or https: URL such as 'http://code.launchpad.net/epics-base/3.14'
If you have problems connecting to launchpad over http: or https: URLs and your network requires you to use an HTTP proxy server, make sure that you have both the http_proxy and https_proxy environment variables set appropriately.
You will need to register for an account at Launchpad.net and upload your ssh public-key to be able to push code to a public repository, and you must be using the bzr+ssh: URL scheme. After registering you only have to tell Bazaar what your Launchpad userid is once, like this:
bzr launchpad-login userid
Your firewall will need to let you access the server bazaar.launchpad.net over ssh (TCP port 22) for access to repositories over a bzr+ssh: URL. If you are behind a firewall and have a Launchpad account but the 'lp:' URLs still don't work, try doing the scheme translation to bzr+ssh: manually, using a URL like this instead:
bzr checkout bzr+ssh://bazaar.launchpad.net/~epics-core/epics-base/3.14/
Non-developers
These options apply to EPICS users who do not expect to make changes to the source code themselves, but who want to be able to access the latest official development versions of the code or to look at how it has changed between different past revisions.
Check out a working tree (CVS model, remote history)
Use this option if you just want to a copy of the latest version of Base from the repository (for example to build and test) but don't expect to modify the code at all, or to look at its history much. This approach is very similar to using CVS since you only download a copy of the particular revision you're interested in, but it require network connectivity for all activities that access the repository in any way. An EPICS Base 3.14 working tree uses about 12MB of disk space at the time of writing.
The following command will create a directory tree called base-3.14 containing the latest code that was checked in to the 3.14 branch of Base:
bzr checkout --lightweight lp:epics-base/3.14 base-3.14
The lp: path above specifies a branch to check out from the launchpad server. The branch at lp:epics-base/3.14 is a branch containing all the development work for the R3.14 releases of Base. There are similar branches for the older series, although we're not expecting anyone to check any of these out; they're provided for historical reasons. The development on the branch called trunk will eventually become the 3.15 series.
If you want to check out a specific tagged version, in this case we're requesting the R3.14.11 release, use this command instead:
bzr checkout --lightweight -r R3.14.11 lp:epics-base/3.14 base-3.14
To update a lightweight working tree at a later date with the latest changes from the branch, use this command inside the tree:
bzr update
You can use the command 'bzr tags' from inside the working tree to list all the tags known on this branch.
Unlike CVS, 'bzr update' cannot apply changes that move your tree to a different historical revision; for that you use the command below, in this case to move to the R3.14.10 release:
bzr revert -r R3.14.10
If you made any local changes to the files in the working tree your changes will be retained through a 'bzr update' or 'bzr revert' operation. Any local changes that overlap with changes from the repository will create a conflict and have to be resolved by hand, just as happens with CVS.
If you find yourself using Bazaar commands inside the tree more than once or twice, you should probably check out a branch instead, which makes a local copy of the complete history of the branch and removes the need for network activity to access that history.
Check out a branch (local history)
This option is similar to the previous one but also gives you a local copy of the complete history of the branch. This speeds up all activities that need to examine the repository history, but since the local branch is still bound to the central repository at launchpad.net you can't commit any changes to it without write permission for the repository branch. A 3.14 branch needs about 28MB of disk space at the time of writing.
The following command will create a directory tree called base-3.14 containing the latest code and all of the history that was checked in to the 3.14 branch of Base:
bzr checkout lp:epics-base/3.14 base-3.14
The lp: path above specifies a branch to check out from the launchpad server. The branch at lp:epics-base/3.14 is a branch containing all the development work for the R3.14 releases of Base. There are similar branches for the older series, although we're not expecting anyone to check any of these out; they're provided for historical reasons. The development on the branch called trunk will eventually become the 3.15 series.
If you want to check out a particular tagged version, use this command instead:
bzr checkout -r R3.14.11 lp:~epics-core/epics-base/3.14 base-3.14.11
Inside the working tree, Bazaar supports the expected commands 'bzr log' and 'bzr diff' to examine file history. Use 'bzr help revisionspec' for information on specifying particular revisions to examine.
To update your working tree with the latest changes from the parent branch at some later date, use this command inside the tree:
bzr update
Note that if you're working in a regular branch and not a checkout, the command to use is
bzr pull
and if you have committed changes to your branch that are not in the parent
bzr merge
Note that 'bzr pull' or 'bzr update' cannot move your tree to a different historical revision like in CVS, for that you use this command:
bzr revert -r R3.14.10
You can use 'bzr tags' when inside the working tree to list all the tags known on this branch.
If you made any local changes to the files in the working tree your changes will be retained through a 'bzr update' or 'bzr revert' operation. Any local changes that overlap with changes from the repository will create a conflict and have to be resolved by hand, just as happens with CVS.
If you want to keep some permanent local changes, consider unbinding your tree from the central repository branch, which converts it to a regular branch. This will allow you to commit local changes into your local repository. The 'bzr unbind' command does this, and can be undone again with 'bzr bind'.
Developers
Developers usually want to be able to commit changes to a local repository while they're working on a new feature or bug-fix. Once those changes are complete they will publish and propose them for review; once approved a member of the epics-core team will merge the new code into the appropriate central repository and it will appear in the next release of that development series.
Repository Initialization
Unless you are only going to work on one modification to epics-base, you should create a local repository, which will be used as the backing-store to hold the history of all your development work on EPICS Base, and will be shared by all your bazaar branches. Doing this saves network bandwidth, as it avoids having to download the past history of changes to epics-base from Launchpad more than once. This command will create an epics-base directory in your home directory, inside of which you will check out your branches (you can use a different name or location, but it should be a parent or grand-parent directory of where you'll create your task branches):
bzr init-repo ~/epics-base
Creating a Task Branch
Bazaar encourages developers to create a separate branch for each separate feature or bug fix being developed, which would normally be created as a sibling to the mirror branch we checked out above. While working on the fix you are free to commit changes whenever you like, which are recorded in your local repository:
cd ~/epics-base bzr branch lp:epics-base fix-123 cd fix-123 # Hack... bzr ci -m 'Hacked the frooz-bodger to stop it wibbling'
If a particular commit fixes a bug that is registered on Launchpad, you can have Bazaar record that fact at the time like this:
bzr ci --fixes=lp:123456
When you are sure that your branch fully implements the new feature or bug-fix that you've been working on and you want the core developers to take a look at it, you should publish it on launchpad and propose it for merging (see #Publishing a Branch below).
Updates and Merging
While you're working on your task branch, other developers' changes may be being merged into the central branch. In most cases there will be little conflict between their work and yours, thus you should not normally merge changes from the central branch into yours before your branch has been proposed for merger. Merging changes from the central branch before requesting the review adds those changes to your merge proposal, which makes the reviewers' task harder. Minor conflicts are to be expected in the release notes, but these will be fixed by the core developer who merges your branch after it has been approved.
Pulling Updates
If you have not committed any changes to your fix-123 branch yet, you can pull all the central changes into your branch in a single command:
cd fix-123 bzr pull # Should report any conflicts
If you're not sure whether you've made any commits or not, you can ask Bazaar:
cd fix-123 bzr missing # Reports commit differences between trees
Merging Updates
If a change does get merged to the central branch which significantly affects your work (and you have already made at least one commit to your branch), you should bring this to the attention of the core developers and ask them how to proceed. If they tell you to merge in from the central branch, make sure you have no uncommitted changes (by either committing or shelving them), then:
cd fix-123 bzr merge # Fix any conflicts, then tell Bazaar like this: bzr resolve file # Finally commit the merge bzr ci -m 'Merged with latest 3.14 changes'
If Bazaar reports any conflicts from the 'bzr merge' command, you must resolve them with 'bzr resolve' before attempting to commit the result. You will find a pristine copy of your version in file.THIS, the other version in file.OTHER and the common ancestor in file.BASE, and collectively these should help you to see what's happening. The command 'bzr remerge' might also be useful in some circumstances, it allows you to redo the merge using some other merge algorithms. The .THIS, .OTHER and .BASE files will be deleted when you run the 'bzr resolve' command.
If you decide that a merge was a mistake and want to undo it, the 'bzr revert' command will undo all uncommitted changes and restore your tree to the way it was after the last commit.
Rebasing a Private Branch
If it is suggested that you rebase your private branch rather than merging the changes from the central branch into it, you will need the rebase Bazaar plugin installed (also now called the rewrite plugin). Use 'bzr plugins' to find out if it is already present or not. If not, you can find it at Rewrite Plugin.
The 'bzr rebase' command is used like this:
cd fix-123 bzr rebase # Fix any conflicts, then tell Bazaar like this: bzr resolve file # Now continue the rebase: bzr rebase-continue # Repeat resolve and rebase-continue commands until done
Never rebase a branch that you have published and which other people may be using as the basis for other development work. Rebasing rewrites the history of your branch, and can result in major spaghetti problems unless you know what you're doing. If in any doubt, make a copy of your branch first ('bzr branch fix-123 clone-123') and do the rebase operation on the copy while leaving the original untouched. Test the rebased branch to make sure your new functionality still works before continuing development or publishing the result.
Shelving Local Changes
If you have some local changes that you don't want to commit (say some CONFIG_SITE changes needed to build the code at your site) but you're about to use a command such as 'bzr merge' that doesn't work if there are uncommitted changes, you can use 'bzr shelve configure/CONFIG_SITE' to temporarily remove them, and 'bzr unshelve' later to bring them back. The shelve command will show you each unsaved change in the named file(s) in turn and ask whether you want to shelve it or not (the '--all' option skips that part). Read the 'bzr help shelve' output for more details.
Publishing a Branch
Anyone can publish branches of the epics-base project on Launchpad, where other people can find and use them:
cd ~/epics-base/fix-123 bzr push lp:~user-id/epics-base/fix-123
Permission Problems
If you get errors like this when trying to access the Launchpad repositories
Permission denied (publickey).
the problem is that your current SSH public key is not the same as the one you registered with Launchpad. Log into Launchpad and go to your personal page, which is linked from the top-right corner of every Launchpad page.
Look under the heading SSH Keys for your currently registered key(s) and compare the contents of your ~/.ssh/id_rsa.pub file with the file your browser shows when you click on the link(s) under the SSH Keys heading. If they are different you will need to upload your new key, click the yellow-circled pencil next to the heading to edit your current keys.
Core Developers
In addition to developing your own features and fixes, if you are a member of the epics-core team you will be able to merge changes (either your own or someone else's) into the official branches in the central repository. That task is simplified if you keep a checkout (mirror) of the relevant central branch, which you can also use to directly commit tiny changes to the repository.
The model described here corresponds to the "decentralized with shared mainline" workflow given in the Bazaar User Guide.
Initial Checkout
First initialize a local repository in a parent directory which will be used as the backing-store to hold the history of all your development work on EPICS Base. Make sure you are logged into Launchpad, then checkout a mirror branch of Base inside your repository directory:
bzr init-repo ~/epics-base bzr launchpad-login userid cd ~/epics-base bzr checkout lp:epics-base/3.14 mirror-3.14
This mirror branch should not be used directly for developing new code or major bug fixing, but can be used for making "obviously correct" changes to things like comments or string literals. Major work should take place in a different tree usually called a feature or task branch.
The mirror branch allows you to merge tested sets of changes from your feature branches into the central repository, or to try out merging other branches while performing code reviews. Remember that performing a commit in the mirror branch is only possible if you are an epics-core member, and will immediately push the change to the official branch on the Launchpad website.
Creating a Task Branch
Bazaar encourages developers to create a separate branch for each separate feature or bug fix being developed, which would normally be created as a sibling to the mirror branch we checked out above. While working on the fix you are free to commit changes whenever you like, which are recorded in your local repository:
cd ~/epics-base bzr branch mirror-3.14 fix-123 cd fix-123 # Hack... bzr ci -m 'Hacked the frooz-bodger to stop it wibbling'
If a particular commit fixes a bug that is registered on Launchpad, you should have Bazaar record that fact at the time like this:
bzr ci --fixes=lp:123
When you are sure that your branch fully implements the new feature or bug-fix that you've been working on and you want the other core developers to review it, you should publish it on launchpad and propose it for merging (see #Publishing a Branch below).
Updating the Mirror
Update your mirror branch frequently to fetch changes from the central repository. Since you aren't using this checkout area for development there should never be any conflicts when you do this:
cd ~/epics-base bzr update mirror-3.14
Pulling Updates
If you have not committed any changes to one or more of your feature branches, you can pull the changes from the update into those branches very easily, and Bazaar will merge them with your uncommitted edits. After updating the mirror-3.14 tree as above, you just do this:
cd fix-123 bzr pull # Should report any conflicts
If you're not sure whether you've made any commits or not, you can ask Bazaar:
cd fix-123 bzr missing # Reports commit differences between trees
Updates and Merging
While you're working on your task branch, other developers' changes may be being merged into the central branch. In most cases there will be little conflict between their work and yours, thus you should not normally merge changes from the central branch into yours before your branch has been proposed for merger. Merging changes from the central branch before requesting the review adds those changes to your merge proposal, which makes the reviewers' task harder. Minor conflicts are to be expected in the release notes, but they should only be fixed when you merge in your branch after it has been approved.
Merging vs. Rebasing
Rebasing a branch is a controversial issue in some communities, there are proponents and nay-sayers on whether it should be used or not. Here are a couple of the discussions:
* A truce in the merge vs. rebase war? * Alternatives to rebasing in Bazaar
Ralph and Andrew are of the opinion that rebasing is an acceptable technique in private branches.
Merging Updates
If you decide to merge in the central branch changes into your work, the steps are:
cd ~/epics-base/feature bzr merge # Fix any conflicts, then tell Bazaar like this: bzr resolve file # Finally commit the merge bzr ci -m 'Merged changes from 3.14 branch'
Note that you should either shelve (see below) or check in any uncommitted local modifications into the task branch before doing the above merge step, if you try to use 'bzr merge --force' instead you risk mixing up your changes with someone else's, which can be confusing and tricky to resolve.
If Bazaar reports any conflicts from the 'bzr merge' command, you must resolve them with 'bzr resolve' before attempting to commit the result. You will find a pristine copy of your version in file.THIS, the other version in file.OTHER and the common ancestor in file.BASE, and collectively these should help you to see what's happening. The command 'bzr remerge' might also be useful in some circumstances, it allows you to redo the merge using some other merge algorithms. The .THIS, .OTHER and .BASE files will be deleted when you run the 'bzr resolve' command.
If you decide that the merge was a mistake and want to undo it, the 'bzr revert' command will undo all uncommitted changes and restore your tree to the way it was after the last commit.
Rebasing a Private Branch
If it is suggested that you rebase your private branch rather than merging the changes from the central branch into it, you will need the rebase Bazaar plugin installed (also now called the rewrite plugin). Use 'bzr plugins' to find out if it is already present or not. If not, you can find it at Rewrite Plugin.
The 'bzr rebase' command is used like this:
cd fix-123 bzr rebase # Fix any conflicts, then tell Bazaar like this: bzr resolve file # Now continue the rebase: bzr rebase-continue # Repeat resolve and rebase-continue commands until done
Never rebase a branch that you have published and which other people may be using as the basis for other development work. Rebasing rewrites the history of your branch, and can result in major spaghetti problems unless you know what you're doing. If in any doubt, make a copy of your branch first ('bzr branch fix-123 clone-123') and do the rebase operation on the copy while leaving the original untouched. Test the rebased branch to make sure your new functionality still works before continuing development or publishing the result.
Merging Task or Feature Branches
The ability to commit changes to the central repository is restricted to members of the epics-core team at Launchpad.net, but Bazaar and Launchpad will still allow you to publish branches containing your own development even if you do not belong to that team; a core developer may subsequently merge and commit the changes from your branch into the official one.
Note that you can undo a merge anytime before the final commit by using the 'bzr revert' command.
Merging a Local Branch
For epics-core members, a feature branch of your own that has been reviewed and approved is merged and committed to the central branch as follows:
cd ~/epics-base/mirror-3.14 bzr update bzr missing ../fix-123 # Check that you're about to merge the right changes bzr merge ../fix-123 # Fix any conflicts, then tell Bazaar like this: bzr resolve file # Finally commit the merge bzr ci -m 'Fixed bug #123'
After you have merged and committed your changes from a feature branch into the central branch(es) at launchpad, you can delete the branch directory, you don't have to do anything special with Bazaar.
Merging a Launchpad Branch
If you're merging someone else's branch which is published on Launchpad, you can use the lp:~user/epics-base/branch-name URL for their branch directly, it is not necessary to download or checkout a copy of a branch to merge it:
cd ~/epics-base/mirror-3.14 bzr update bzr missing lp:~mdavidsaver/epics-base/rec-init # Check that you're about to merge the right changes bzr merge lp:~mdavidsaver/epics-base/rec-init # Fix any conflicts, then tell Bazaar like this: bzr resolve file # Finally commit the merge bzr ci -m "Merged Michael Davidsaver's 'rec-init' branch."
Cherry-picking
This is not recommended, discuss on core-talk before you use cherry-picking...
To apply a bug-fix change to more than one series branch, say to both 3.14 and the main trunk, Bazaar's merge command allows you to cherry-pick commits from a different branch. If it's just one check-in this is very easy, especially if it's the last commit you made to that source branch. In that case, you would use these commands to merge those changes to the trunk:
cd ~/epics-base/mirror-trunk bzr update bzr merge -c -1 ../fix-123 # Fix any conflicts and tell bzr like this: bzr resolve file # Finally commit the merge bzr ci -m 'Fixed bug #123'
The '-c -1' option to 'bzr merge' means "cherry-pick just the last commit", '-c -2' would be the penultimate commit and so on. You can use 'bzr log' in the feature branch to find a specific absolute revision number on that branch and use that instead of the negative numbers. If there were several commits you can give 'bzr merge' a range using the option '-r 44..47' instead, but note that '-r 44' means "merge all changes up to and including revision 44." If you get it wrong, you can always use 'bzr revert' to undo.
Shelving Local Changes
If you have some local changes that you don't want to commit, say some CONFIG_SITE changes needed to build the code at your site, you can use 'bzr shelve configure/CONFIG_SITE' to temporarily hide them before committing or merging, and 'bzr unshelve' later to bring them back. The shelve command will show you each of the changes in the named file(s) in turn and ask whether you want to shelve it or not. Read the 'bzr help shelve' output for more details.
Publishing a Branch
Anyone can publish branches of the epics-base project on Launchpad, where other people can find and use them. You must have registered for an account at Launchpad.net first and uploaded your ssh public-key. Then:
bzr launchpad-login userid cd ~/epics-base/fix-123 bzr push lp:~user-id/epics-base/fix-123
Members of the epics-core team can create shared branches that other team members can also commit to by using the group name as the user-id in the Bazaar target path:
bzr push lp:~epics-core/epics-base/feature-xyz
