diff options
Diffstat (limited to 'debian/pkg-zsh-workflow_outdated.md')
| -rw-r--r-- | debian/pkg-zsh-workflow_outdated.md | 512 |
1 files changed, 512 insertions, 0 deletions
diff --git a/debian/pkg-zsh-workflow_outdated.md b/debian/pkg-zsh-workflow_outdated.md new file mode 100644 index 000000000..00b33559e --- /dev/null +++ b/debian/pkg-zsh-workflow_outdated.md @@ -0,0 +1,512 @@ +Branches +======== + +* **upstream**: The upstream sources from `git://zsh.git.sf.net/gitroot/zsh/zsh`. +* **debian**: The debian changes for the debian `zsh` package. Only + difference to the upstream branch is the `debian` directory. +* master: This is the _old_ repository's main branch. Only kept for + _historical_ reasons. +* Other branches: These branches are rather optional and not required + for basic maintenance for of the `zsh` package. Most likely feature + branches. + + +Workflow +======== + +This diagram outlines the workflow with git branches and +tags. Basically, the `debian` branch walks alongside the `upstream` +branch and upstream changes get merged into `debian` at defined points. +For the zsh package, this _must_ be upstream commit tags -- otherwise +Jenkins +[fails to build the source package](http://jenkins.grml.org/view/Debian/job/zsh-source/219/console). + + * debian/4.3.11-4 + | + | * debian/4.3.11-5 + | | + | | * debian/4.3.12-1 + | | | + | | | * debian/4.3.12-2 + | | | | + | | | | * debian/4.3.13-1 + | | | | | + | | | | | * debian/4.3.13-2 + | | | | | | + | | | | | | * debian/4.3.13-3 + | | | | | | | + | | | | | | | + a-b-c-d---e-f-g-h-i-------j-k-l-m-n-o-p debian + / / / + A-B-C-D-E-F-G-H-I-J-K-L-M-N-O-P-Q upstream + | | | + | | | + | | * zsh-4.3.13 + | * zsh-4.3.12 + | + * zsh-4.3.11 + +Working on the package +---------------------- + +### Patching upstream code + +Every change to the source (outside the `debian` directory) must be +done by adding quilt patches. + +#### Patch naming conventions + +* File names of patches which are not meant for upstream should be + prefixed with `debian-`. +* File names of patches which are cherry-picked from upstream should be prefixed + with `cherry-pick-$shortcommitid` where `$shortcommitid` are the + first 8 characters of the upstream git commit id. + +In case a vast number of patches are required, quilt patches should be +prefixed by ascending numbers according to the order in which "quilt +series" would list them. + +#### Patch format + +Every patch should have +[DEP3 conforming patch headers](http://dep.debian.net/deps/dep3/) +above the actual unified diff content, outlining the status of the +patch. + +Except for the obvious case of a cherry-picked patch from upstream +(which should contain an `Origin: commit $commitid` header), it is +important that the fields `Forwarded` and `Applied-Upstream` are kept +uptodate to know whether a change is already applied upstream. This +helps to drop the right patches before merging a new upstream release +into the `debian` branch. + + +#### Example of adding a fix via a quilt patch + +Let's say, there is an issue _#12345_ which can be fixed by patching +`Functions/Misc/colors`. Here is how you could add that patch +(assuming clean git working directory): + +First, push all existing patches, so the new one is added on top. + + % quilt push -a + +Alternatively push the patches one by one (by calling `quilt push` +multiple times) until you are at the patch queue position where you +want to insert the patch. + +##### Adding a patch from a file or by editing + +Add the new patch (say, the topmost patch is 0002-foo.diff). + + % quilt new fix-colors.diff + +Tell quilt which files are going to be changed. + + % quilt add Functions/Misc/colors + +Import the fix either manually using your favourite editor… + + % $EDITOR Functions/Misc/colors + +… or by patching: + + % patch -p$n < ../patch_from_somewhere_else.diff + +##### Cherry-picking patches from upstream + +When there is an existing patch (e.g. from upstream's git repository), +the above can be largely automated if the patch applies to the current +state of the debian branch. + + % patchname="cherry-pick-$shortcommitid-description-with-dashes" + % git show $commitid | filterdiff -x a/ChangeLog > debian/patches/$patchname + % sed -e '1 s/^commit/Origin: commit/' -i debian/patches/$patchname + % echo $patchname >> debian/patches/series + % $EDITOR debian/patches/$patchname + % git add debian/patches/series debian/patches/$patchname + % git commit -v + +Patches from upstream will likely include changes to the ChangeLog +file. Those changes will probably not apply cleanly, so just open the +created patch file and delete all hunks that do changes in +ChangeLog. The `filterdiff` command above should catch that. + + % $EDITOR debian/patches/$patchname + +Check if the patch applies + + % quilt push + +All the above is mostly automated in `debian/bin/commit2patch`. + +##### Finish import of the patch + +Refresh the patch to get rid of any fuzz or offset: + + % quilt refresh + +Pop all patches again to clean up the upstream source. + + % quilt pop -a + +Edit the patch headers according to +[DEP3](http://dep.debian.net/deps/dep3/). + + % $EDITOR debian/patches/$patchname + +Commit the new patch and the changed `series` file to git. + + % git add debian/patches/fix-colors.diff + % git add debian/patches/series + % git commit -m 'Fixing foo in colors function (Closes: #12345)' + +That's all. + +### Releases + +When a change justifies the release of a new package version, the +debian/changelog file should be updated and the resulting commit +should be tagged `debian/${zsh_version}-${n+1}`. + + +### Updating debian/changelog + +This file should *not* be updated manually. The changes should be +inserted by running the `gbp dch` command from the package +`git-buildpackage` (see gbp-dch(1)) before a new release is about to +be made. + +Changelog entries should be prefixed by a `[$hashsum] ` string, where +`$hashsum` is a string that represents the first eight characters of +commit the changelog entry was generated from. + +Also, if multiple authors are involved in a changelog entry-set, each +author should only appear once in the series with all her/his changes +listed below her/him in chronological order. + +Given that `debian/gbp.conf` is up-to-date, using the `gbp dch` +command will result in the desired changelog format: + + % gbp dch + +If you absolutely *must* make changelog entries by other means, you +should make sure that you prefix any resulting commits with +"[dch-ignore] ", so those commits can be weeded out easily. + +There is a helper script `debian/bin/do-dch` which takes care of all +formatting options as well as the "[dch-ignore] " weeding. The +script should be used unless there is a good reason not to. + + +Transitioning to a new upstream version +--------------------------------------- + +When upstream releases a new version, we should follow these steps: + +### Merging new upstream tag (`zsh-$version`) into our upstream branch + + % git checkout upstream + % git pull origin + % git fetch zsh + % git merge --ff-only zsh-$version + +If that doesn't do a fast-forward merge, a fast-forward merge can be +enforced as follows: + + % git checkout upstream + % git reset --hard zsh-$version + +### Remove all quilt patches which are applied upstream + +All patches applied should be removed from `debian/patches` directory, +unless they fix an issue that was *not* addressed upstream and is +therefore missing from upstream's code base. + +### Merging the branch upstream into the branch debian + +After the `debian/patches` directory was cleaned up in the previous +step, merging `upstream` into `debian` should generally lead to a +working package again. + +If old patches were still around, that could lead to conflicts +when those would be applied during the build process. + +The message for the merge commit should be set to "New upstream +release" to allow `gbp dch` to pick it up correctly later. **TODO**: +Doesn't really work. + +### Insert initial changelog for the new upstream release + +`gbp dch` seems to be in trouble with non-linear histories. Therefore +we introduced a small helper script that will help `gbp dch` to a +linear history again. + +Basically, you after merging the upstream release tag into the debian +branch, you'll be left with an history that looks something like this: + + * Updating autotools patches + M Merge commit 'zsh-4.3.13' into debian + |'* unposted: released 4.3.13 + | * … + | * … lots of other upstream commits … + | * … + * | Removing upstream patches due to new release + * | Last debian/4.3.12-* commit + * | … + * | … lot's of other debian/4.3.12-* commits + * | … + M´ Merge commit 'zsh-4.3.12' into debian + |'* unposted: released 4.3.12 + … older history + +And what you really want added to debian/changelog is the "Updating +autotools patches" and the "Removing upstream patches due to new +release" commits. You need to figure out the sha1 sums of the commits +and then call this: + + % debian/bin/urcl -p=zsh -v=4.3.13-1 b495ba1e f575f568 + +… where `4.3.13-1` is the version of the upcoming debian package and +`b495ba1e` and `f575f568` are the SHA1 hashsums of the wanted commits. + +At the end the script will drop you into an editor pointed at the +changelog file so you can sanity-check the generated output. + +At this point it would also make sense to add a line like this: + + * New upstream release + +or, if someone explicitly requested a package of this upstream +release, with mentioning of the according bug report number: + + * New upstream release (Closes: #1234567) + +If the upstream release fixes additional bugs reported in Debian or +security relevant bugs, the corresponding upstream commits should be +listed indented by two spaces, together with a short description and +the according bug report and/or CVE numbers, e.g. like this: + + * New upstream release + + [abcdefgh] Fixes foo (Closes: #1234567) + + [deadcafe] Adds Completion for bar (Closes: #987654) + + [babeabed] Fixes CVE-2014-9876 + +When creating a commit with these changelog changes, you may want to +prefix the commit message with `[dch-ignore] ` or add `-m "Gbp-Dch: +Ignore"` to the commit command so it doesn't come up in later `gbp dch` +runs. + +### Create the fake orig tar ball (until we can work with upstream's tarball) + +This requires the upstream release to be properly tagged. + + % make -f debian/rules get-orig-source + +### Fix outstanding bugs + +If *any* outstanding bugs are known, they should be fixed before +releasing a new package. Obviously, if any of the known bugs are very +hard to fix and the issue is not serious in nature, releasing the +package with the issue may be more important. + +Again, all changes to non `debian/*` files should be done via quilt +patches. + + +### Verify that the package builds + +There are many ways to do this. Important is: + +* Use a clean and uptodate Debian Sid chroot (e.g. by using `sbuild`, + or `pbuilder` and friends) to make a comprehensive test or for + uploading. + + Axel prefers: `pdebuild --debbuildopts -j6` + +* For a quick sanity check, a simple `dpkg-buildpackage -B` (just + builds the architecture-dependent binary packages or a `debuild -uc + -us` (builds source and binary packages, runs `lintian` afterwards) + may suffice. + +* Use `gbp buildpackage` to automatically make sure, you are on the + correct branch and that the working directory is clean. + + +### Update changelog again for the release + +The `do-dch` helper script should be used to do this. It wraps `gbp dch` +with appropriate options and weeds out any commits that are prefixed +with "[dch-ignore] ". All options to the script are turned over to +`gbp dch` and at least `--since=…` should be used. + +At this particular point the sha1 of the previous initial changelog +update commit would be a good idea. Also "-R" to tell `gbp dch` to +prepare the changelog for an actual commit. So: + + % debian/bin/do-dch --since=1234deadbeef -R + +You'll be dropped into an editor again to double check the generated +changelog. + + +### Tag debian/${new_zsh_version}-1 + +After fixes for all serious and trivially fixable issues have been +added and it has been verified that the package builds and `do-dch` +has updated `debian/changelog` and the resulting commit should be +tagged as `debian/${new_zsh_version}-1`. + + +Generating packages +------------------- + +### git-buildpackage aka gbp + +Alternatively, `git-buildpackage` (short `gbp`) also provides ways of +building packages from our packaging codebase. And since we are using +the `gbp dch` command (formerly the `git-dch` tool from this utility +suite anyway, the tool should be available already. + +`git-buildpackage` allows building the package from within the +package repository: + + % gbp buildpackage + +Make sure that the local repository is cleaned up after doing this +before working on the package again, to avoid accidentially committing +anything. See *Cleaning up the local repository* below for details. + +`git-buildpackage` is available as Debian package or from +https://honk.sigxcpu.org/piki/projects/git-buildpackage/ + + +Git repository setup +-------------------- + +Getting the basic pkg-zsh git repository is quite easy. If you want +a read only clone, use this: + + % gbp clone git://anonscm.debian.org/collab-maint/zsh.git pkg-zsh + +If you are reading this, though, you probably want write access. To +get a thusly cloned repository, first get an alioth login and upload +an ssh-public key. As soon as the key made it to all involved +machines, use this: + + % gbp clone $user@git.debian.org:/git/collab-maint/zsh.git pkg-zsh + +Where `$user` is your Alioth login. (Note, that this may be something +with a `-guest` suffix, in case you're not a Debian Developer.) + +### Branches + +Like described earlier, pkg-zsh development involves two branches; +`debian` and `upstream`. The former is checked out by default for +freshly cloned repositories. + +If you cloned the repository with `gbp clone` as shown above, gbp +already took care of also creating a local `upstream` branch. + +If you didn't, you can get a local version of the `upstream` branch by +calling + + % git checkout -b upstream origin/upstream + +This is useful to update the remote upstream branch with ongoing +development from the zsh project. + +### Remotes + +There is one remote repository with direct interest for pkg-zsh, and +that is the zsh project's git repository. Currently, this is only a +mirror of the project's cvs repository. But it is updated every ten +minutes by one of zsh's developers. (Also note, that there has been a +brief discussion about whether git may become the official VCS for git +after a bigger future release.) + +In order to have zsh's ongoing development available from within +your pkg-zsh repository, do this: + + % git remote add zsh.git git://zsh.git.sf.net/gitroot/zsh/zsh -t master + % git fetch zsh.git + +### Merging and pushing upstream changes + +To get updates back into `origin/upstream`, do this: + +Get the latest updates. + + % git fetch zsh.git + +Switch to the local `upstream` branch for integration. + + % git checkout upstream + +Merge upstream's changes (*). + + % git merge zsh.git/master + +Push the code into pkg-zsh's central repository. + + % git push origin + +Make sure the central repository also has all tags. + + % git push --tags origin + +(*) This step should *always* result in a fast-forward merge. If it +does not, something went terribly wrong. Investigate and fix the +situation *before* pushing to origin. + +### Dealing with quilt's .pc directory + +When quilt works, it keeps track of what it does in a directory by the +name `.pc`. This directory will show up in the output of `git status` +etc. It should *never* *ever* by committed to the git repository at +any point. + +We cannot add the directory to `.gitignore` because we would change +the zsh source directly instead of via `debian/patches`. + +To deal with the directory, there is another git-facility, that we can +use for fun and profit. + + % echo .pc/ >> .git/info/exclude + +Now git will ignore quilt's directory for this repository. +Unfortunately, this has to be done for each checkout. Luckily, this +only has an impact for people who want to work on *pkg-zsh*. Everyone +else can savely ignore the directory. + + +General Git Hints +----------------- + +### Keeping the local repository clean + +Before making changes of any kind, it should be made sure that the +local repository you are working on is in a clean state. To clean up +the local repository do this: + + % git clean -xdf + % git reset --hard + +That will make sure that any non-tracked files are removed and that +any changes in tracked files are reverted. The latter will also make +sure that no parts of a quilt patch-queue are still applied. + + +TODO +==== + +* How and when to tag releases → `gbp buildpackage --git-tag` or even + `gbp buildpackage --git-tag-only` only after the upload has been + uploaded/accepted. +* How and when to push tags. Debian Perl Group's `dpt push` (from the + package `pkg-perl-tools`) comes in handy. +* `git commit -m 'Something unimportant' -m 'Gbp-Dch: Ignore'` +* `export QUILT_PATCHES=debian/patches` should be mentioned under + *Repository setup*, not under *Verify that the package builds*. +* `* New upstream release` changelog entries should have the git + commit id of the upstream tag. |