* Branches This is a quick overview of the branches that are being used for debian's zsh package maintenance. This part does not go into much detail on how the workflow works. For details about that, see the `Workflow' part below. ** upstream The upstream sources from . ** debian The debian changes for the debian `zsh' package. ** others These branches are rather optional and not required for basic maintenance for of the `zsh' package. *** master This is the old repository's main branch. Kept for historical reasons. *** debian-beta This could basically do *exactly* the same thing as `debian' does, but just merge more often. Such a branch *could* then be used to keep the packaging information for `zsh-beta'. * Workflow This is a diagram, that outlines the proposed workflow. Basically, the `debian' branch walks alongside the `upstream' branch and upstream changes get merged into `debian' at defined points (for the non-beta zsh package, that should be upstream commit tags). #+BEGIN_EXAMPLE * debian/4.3.11-4 | | * debian/4.3.11-5 | | | | * debian/4.3.12-1 | | | | | | * debian/4.3.12-2 | | | | | | | | * debian/4.3.13-1 | | | | | | | | | | * debian/4.3.13-2 | | | | | | | | | | | | * debian/4.3.13-3 | | | | | | | | | | | | | | a-b-c-d---e-f-g-h-i-------j-k-l-m-n-o-p `debian' / / / A-B-C-D-E-F-G-H-I-J-K-L-M-N-O-P-Q `upstream' | | | | | | | | * zsh-4.3.13 | * zsh-4.3.12 | * zsh-4.3.11 #+END_EXAMPLE ** Working on the package *** Changes Every change to the source (non-debian-dir) should be done by adding quilt patches. Permanent patches (patches that would be kept in 4.3.12-1) should be named specially ("deb_*" comes to mind). To keep order for cases in which a vast number of patches are required, any quilt patch needs to be numbered according to the order in which "quilt series" would list them. The format looks like this: - NNNN_.diff - deb_NNNN_.diff Where `N' is a digit from 0 to 9. Counting starts at `0000'. Debian-specific (i.e. deb_NNNN_.diff) patches should be applied after patches targeted towards upstream since the patches targeted for upstream should not be against debian-specific code. Every patch should have a short annotation above the actual unified diff content, outlining the status of the patch. It is especially important to know whether a change is already applied upstream because if so, the patch in question must be dropped before merging a new upstream release into the `debian' branch. **** Dealing with quilt's .pc directory When quilt works, it keeps track of what it does in a directory by the name `.pc'. This directory will show up in the output of "git status" etc. It should *never* *ever* by committed to the git repository at any point. We cannot add the directory to `.gitignore' because we would change the zsh source directly instead of via `debian/patches'. To deal with the directory, there is another git-facility, that we can use for fun and profit. % echo .pc/ >> .git/info/exclude Now git will ignore quilt's directory for this repository. Unfortunately, this has to be done for each checkout. Luckily, this only has an impact for people who want to work on `pkg-zsh'. Everyone else can savely ignore the directory. **** Example of adding a fix via a quilt patch Let's say, there is an issue `#12345' which can be fixed by patching `Functions/Misc/colors'. Here is how you could add that patch (assuming clean git working directory and the topmost patch is 0002-foo.diff): #+BEGIN_EXAMPLE ## First, add all existing non-debian patches, so the new one is ## added on top. % quilt push 0002-foo.diff ## Add the new patch (say, the topmost patch is 0002-foo.diff). % quilt new 0003-fix-colors.diff ## Tell quilt which files are going to be changed. % quilt add Functions/Misc/colors ## Import the fix (manually by editor or by patching). % vi Functions/Misc/colors ## Refresh the patch % quilt refresh ## Pop all patches again to clean up the upstream source. % quilt pop -a ## Commit the new patch and the changes `series' file to git. % git add debian/patches/0003-fix-colors.diff % git add debian/patches/series % git commit -a -m'Fixing foo in colors function (Closes: #12345)' #+END_EXAMPLE That's all. **** Cherry-picking patches from upstream into quilt When there is an existing patch (e.g. from upstream's git repository), the above can be largely automated if the patch applies cleanly to the current state of the debian branch. The `./debian/patch2quilt' helper script takes care of that task. It's called like this: #+BEGIN_EXAMPLE % ./debian/patch2quilt ../existing.diff 0023-new-quilt.diff #+END_EXAMPLE Here "../existing.diff" is the file containing the existing patch and "0023-new-quilt.diff" is the name of the to-be-added quilt series patch (make sure its nameing is in line with the established conventions). The exact operation of the script is described at the top of the script file. There are a few things to keep in mind: - At the end of successful operation you are dropped into an editor which gives you the opportunity to add annotations at the top of the patch file (like if the patch in question is included upstream already). - Never *ever* run the script when you got uncommitted changes in the worktree, which you don't plan on losing. The worktree will be cleaned and reset first thing in the script. - As an extension of the previous point, don't put the existing patch you're planing to import into the git working tree. It would be wiped away, too. - Patches from upstream will likely include changes to the ChangeLog file. Those changes will probably not apply cleanly, which will break the scripts execution. Just open the existing patch and delete all hunks that do changes in ChangeLog. When the script finishes (after you exit your editor), it will suggest how to commit the newly intoduced patch. Season to taste. **** Keeping the local repository clean Before making changes of any kind, it should be made sure that the local repository you are working on is in a clean state. To clean up the local repository do this: #+BEGIN_EXAMPLE % git clean -xdf % git reset --hard #+END_EXAMPLE That will make sure that any non-tracked files are removed and that any changes in tracked files are reverted. The latter will also make sure that no parts of a quilt patch-queue are still applied. *** Releases When a change justifies the release of a new package version, the debian/changelog file should be updated and the resulting commit should be tagged debian/-n+1. *** Updating debian/changelog This file should *not* be updated manually. The changes should be inserted by running the `git-dch' tool from the package `git-buildpackage' before a new release is about to be made. Changelog entries should be prefixed by a "[hashsum] " string, where `hashsum' is a string that represents the first eight characters of commit the changelog entry was generated from. Also, if multiple authors are involved in a changelog entry-set, each author should only appear once in the series with all her/his changes listed below her/him in chronological order. Here is a command line which will result in the desired changelog format: #+BEGIN_EXAMPLE % git-dch --debian-branch=debian --id-length=8 --multimaint-merge #+END_EXAMPLE If you absolutely *must* make changelog entries by other means, you should make sure that you prefix any resulting commits with "[dch-ignore] ", so those commits can be weeded out easily. There is a helper script "debian/do-dch" which takes care of all formatting options as well as the "[dch-ignored] " weeding. The script should be used unless there is a good reason not to. ** Transitioning to a new upstream version When upstream releases a new version, we should follow these steps: *** Removing non deb_* quilt patches All non deb_* patches should be removed from `debian/patches' directory, unless they fix an issue that was *not* addressed upstream and is therefore missing from upstream's code base. If such a change should prove to be required to be kept with the package permanently (e.g. because upstream refuses to apply the patch), the patch should eventually be renamed to match the "deb_*" nameing convention. *** Merging `upstream' into `debian' After the `debian/patches' directory was cleaned up in the previous step, merging `upstream' into `debian' should generally lead to a working package again. If old patches were still around, that could lead to conflicts when those would be applied during the build process. *** Update the autotools files for the new release This should be done to make sure, the build systems is always kept up to date. For convenience, there is a script that automates this process: #+BEGIN_EXAMPLE % ./debian/at2quilt #+END_EXAMPLE After that, the involved quilt patches are updated, but not yet committed to the git repository to allow for review. **** Updating autotools files whenever input files are changing Sometimes, it will be necessary to backport changes to autotools input files (such as `configure.ac'). As with any other change, such a task should be done by adding a patch to the packages quilt queue. Obviously, if an input file changes, the autotools related patches for the package need to be updated. Again, the `at2quilt' script from the `debian/' subdirectory helps. The only difference to the way it was used before is that you need to tell the script which patches in the patch-queue are relevant for autotools. Say there is a patch `0017-autotools-fix-gnu-hurd.diff' which alters `configure.ac', here is what would need to be done: #+BEGIN_EXAMPLE % ./debian/at2quilt debian/patches/0017-autotools-fix-gnu-hurd.diff #+END_EXAMPLE If there is more than one patch `at2quilt' need to consider, list all of them in the order in which `quilt' would apply them (if in doubt, ask "quilt series"). *** Insert initial changelog for the new upstream release `git-dch' seems to be in trouble with non-linear histories. Therefore we introduced a small helper script that will help `git-dch' to a linear history again. Basically, you after merging the upstream release tag into the debian branch, you'll be left with an history that looks something like this: #+BEGIN_EXAMPLE * at2quilt: Updating autotools patches M Merge commit 'zsh-4.3.13' into debian |`* unposted: released 4.3.13 | * ... | * ... lots of other upstream commits ... | * ... * | Removing upstream patches due to new release * | Last debian/4.3.12-* commit * | ... * | ... lot's of other debian/4.3.12-* commits * | ... M´ Merge commit 'zsh-4.3.12' into debian |`* unposted: released 4.3.12 ... older history #+END_EXAMPLE And what you really want added to debian/changelog is the "atquilt: Updating autotools patches" and the "Removing upstream patches due to new release" commits. You need to figure out the sha1 sums of the commits and then call this: #+BEGIN_EXAMPLE % ./debian/urcl -p=zsh -v=4.3.13-1 b495ba1e f575f568 #+END_EXAMPLE ...where "4.3.13-1" is the version of the upcoming debian package and "b495ba1e" and "f575f568" are the sha1 sums of the wanted commits. At the end the script will drop you into an editor pointed at the changelog file so you can sanity-check the generated output. At this point it would also make sense to add a line like this: #+BEGIN_EXAMPLE * New upstream release #+END_EXAMPLE or something like this if the release fixes outstanding bugs: #+BEGIN_EXAMPLE * New upstream release (Closes: #1234567890) #+END_EXAMPLE When creating a commit with these changelog changes, make sure you prefix the commit message with "[dch-ignore] " so it doesn't come up in later git-dch runs. *** Update debian/gbp.conf The debian/gbp.conf file contains a reference pointing to the upstream branch. Therefore the upstream-branch configuration inside debian/gbp.conf needs to be adjusted for new upstream releases. *** Fix outstanding bug If *any* outstanding bugs are known, they should be fixed before releasing a new package. Obviously, if any of the known bugs are very hard to fix and the issue is not serious in nature, releasing the package with the issue may be more important. Again, all changes to non `debian/*' files should be done via quilt patches. *** Verify that the package builds #+BEGIN_EXAMPLE % git reset --hard % git clean -xdf % QUILT_PATCHES=debian/patches % export QUILT_PATCHES % quilt push -a % ./configure % make all test #+END_EXAMPLE *** Update changelog again for the release The `do-dch' helper script should be used to do this. It wraps git-dch with appropriate options and weeds out any commits that are prefixed with "[dch-ignore] ". All options to the script are turned over to git-dch and at least `--since=...' should be used. At this particular point the sha1 of the previous initial changelog update commit would be a good idea. Also "-R" to tell git-dch to prepare the changelog for an actual commit. So: #+BEGIN_EXAMPLE % ./debian/do-dch --since=1234deadbeef -R #+END_EXAMPLE You'll be dropped into an editor again to double check the generated changelog. *** Tag debian/-1 After fixes for all serious and trivially fixable issues have been added and it has been verified that the package builds and `do-dch' has updated `debian/changelog' and the resulting commit should be tagged as `debian/-1'. ** Generating packages *** gitpkg `gitpkg' is a simple tool to help generating packages from debian packages which are maintained in the git version control system. It works quite well in this workflow. In fact, it works out of the box: #+BEGIN_EXAMPLE % gitpkg debian/4.3.12-5 zsh-4.3.12 #+END_EXAMPLE The first parameter (debian/4.3.12-5) is the debian tag which points at the debian package version you want to build. The second parameter is the tag of the upstream version of the corresponding upstream release (zsh-4.3.12). Per default, `gitpkg' generates it's output in `../deb-packages'. This location is configurable. Below directories for individual packages are created and in those, data for individual package versions are created. For the above example, this would look like this: #+BEGIN_EXAMPLE ../deb-packages/zsh/ ../deb-packages/zsh/zsh-4.3.12/ ../deb-packages/zsh/zsh_4.3.12.orig.tar.gz ../deb-packages/zsh/zsh_4.3.12-5.debian.tar.gz ../deb-packages/zsh/zsh_4.3.12-5.dsc #+END_EXAMPLE You may now change to `../deb-packages/zsh/zsh-4.3.12/' and build binary package using `dpkg-buildpackage', `debuild' or the like. `gitpkg' is available as Debian package or from: *** git-buildpackage Alternatively, `git-buildpackage' also provides ways of building packages from our packaging codebase. And since we are using the `git-dch' tool from this utility suite anyway, the tool should be available already. `git-buildpackage' allows building the package from within the package repository: #+BEGIN_EXAMPLE % git-buildpackage --debian-branch=debian #+END_EXAMPLE Make sure that the local repository is cleaned up after doing this before working on the package again, to avoid accidentially committing anything. See "Cleaning up the local repository" above for details. `git-buildpackage' is available as Debian package or from: ** Git repository setup Getting the basic pkg-zsh git repository is quite easy. If you want a read only clone, use this: #+BEGIN_EXAMPLE % git clone git://git.debian.org/collab-maint/zsh.git pkg-zsh #+END_EXAMPLE If you are reading this, though, you probably want write access. To get a thusly cloned repository, first get an alioth login and upload an ssh-public key. As soon as the key made it to all involved machines, use this: #+BEGIN_EXAMPLE % git clone ssh://@git.debian.org/git/collab-maint/zsh.git pkg-zsh #+END_EXAMPLE Where `' is your alioth login. (Note, that this may be something with a `-guest' suffix, in case you're not a debian developer.) *** Branches Like described earlier, pkg-zsh development involves two branches; `debian' and `upstream'. The former is checked out by default for freshly cloned repositories. To get a local version of the `upstream' branch, use: #+BEGIN_EXAMPLE % git checkout -b upstream origin/upstream #+END_EXAMPLE This is useful to update the remote upstream branch with ongoing development from the zsh project. *** Remotes There is one remote repository with direct interest for pkg-zsh, and that is the zsh project's git repository. Currently, this is only a mirror of the project's cvs repository. But it is updated every ten minutes by one of zsh's developers. (Also note, that there has been a brief discussion about whether git may become the official VCS for git after a bigger future release.) In order to have zsh's ongoing development available from within your pkg-zsh repository, do this: #+BEGIN_EXAMPLE % git remote add zsh.git git://zsh.git.sf.net/gitroot/zsh/zsh -t master % git fetch zsh.git #+END_EXAMPLE *** Merging and pushing upstream changes To get updates back into origin/upstream, do this: #+BEGIN_EXAMPLE ## Get the latest updates. % git fetch zsh.git ## Switch to the local `upstream' branch for integration. % git checkout upstream ## Merge upstream's changes (*). % git merge zsh.git/master ## Push the code into pkg-zsh's central repository. % git push origin ## Make sure the central repository also has all tags. % git push --tags origin (*) This step should *always* result in a fast-forward merge. If it does not, something went terribly wrong. Investigate and fix the situation *before* pushing to origin. #+END_EXAMPLE