Exercise 2: dh-make and dpkg-buildpackage

Exercise 2: dh-make and dpkg-buildpackage
	Chapter 2. The Workshop Instructions

Figure 2.3. DebPackaging with dpkg-buildpackage

A sketch of the data flow when creating a package with dpkg-buildpackage

The situation in the section called “ Exercise 1: Manual Packaging (optional) ” is not very realistic. Typically, the packagers of software are not the same people as its authors, and the payload of a package consists of dozens of files instead of just two. For the current exercise, we are going to fetch existing software from the 'net, and we are going to automate some tasks already.

As shown in Figure 2.3, “DebPackaging with dpkg-buildpackage”, the packager now fetches the source of the software-to-be-packaged from upstream (top left corner), then runs dh_make to create control files from templates instead of from scratch. With a near-perfect upstream source, the bulk of the work then consists of editing these still imperfect control files to satisfaction. Instead of creating the intermediate data.tar.gz and control.tar.gz, a single command: dpkg-buildpackage creates the final packages, in this case both a source and a binary package.

Dpkg-buildpackage just calls make with debian/rules as the Makefile, and certain standard targets. The template rules created by dh_make is tailored to work with the Autotools' ./configure; make; make install. So while ruled by debian/rules, Make in its turn by default calls another Make ruled by the upstream Makefile. Read that again. If your particular upstream doesn't provide a Makefile, you can either provide it, or write a rules file of your own.

When running the upstream make install, dpkg-buildpackage sets $DESTDIR set to a subdirectory of debian. This is the reason that in order to use the standard rules, either upstream or you must provide a Makefile that respects $DESTDIR. A glimpse of the rules file will show that it makes extensive use of the debhelper suite, dh_*.

Procedure 2.2. packaging with dh-make and dpkg_buildpackage

Fetching the source
Set up a working directory, then fetch the source, untar it, and cd into the source directory ^[2] :


cd ~/packaging
mkdir hello
cd hello
wget http://ftp.gnu.org/gnu/hello/hello-2.2.tar.gz
tar zxvf hello-2.2.tar.gz
cd hello-2.2

Running dh_make to create control files

In the previous exercise, we created the control files (control, readme.Debian and copyright) from scratch with an editor. This time we use dh_make to give us a head start.

If you haven't yet done so your ~/.bashrc, then now is the time to set a couple of environment variables, then call dh_make:

export DEBEMAIL=j.bokma@rug.nl
export DEBFULLNAME="Jurjen Bokma"
dh_make --copyright gpl  -f ../hello-2.2.tar.gz

      Type of package: single binary, multiple binary, library, kernel module or cdbs? [s/m/l/k/b] s   Maintainer name : Jurjen Bokma Email-Address   : j.bokma@rug.nl Date            : Mon, 07 Sep 2009 19:48:51 +0200 Package Name    : hello Version         : 2.2 License         : gpl Using dpatch    : no Type of Package : Single Hit <enter> to confirm: Done. Please edit the files in the debian/ subdirectory now. hello uses a configure script, so you probably don't have to edit the Makefiles.

	This option not neccessary, but it makes for a more complete `copyright` file.
	This option here to prevent creation of a `.orig` file.
	“Single binary” is to say that we create a binary package, and just a single package from this directory structure. The single package here is the source package. If the software needs to be recompiled for each architecture, even a single source package may yield many binary packages. In that case, you still need to choose “single binary”.
	CDBS, the Common Debian Build System is an effort to reduce the amount of legacy in `rules` files by reducing them to a two-line file that sources from regularly updated Makefile snippets in `/usr/share/cdbs`, possibly amended with some hooks. For clarity, we stick with a complete Makefile in `rules` this time.

Running dh_make has created a file ../hello_2.2.orig.tar.gz (note that it is identical to ../hello-2.2.tar.gz) and a directory./debian.

	Warning
	Don't run dh_make twice in the same directory. It is not idempotent.

Inspecting and modifying the files in ./debian

Seeing what has changed

Cd into ./debian and take a look at what is there:

	Only a few control files have to exist.
	Many proposed control files are just examples.
	The `debian/rules` file is necessary only if we want to use dpkg-buildpackage later on. (We do.)

For an explanation of what each of these files does, see the sections on required and optional controlfiles in the New Maintainer's Guide. Or, for a more in-depth explanation, see the DPM.

	Note
	The astute reader will notice that some of the files that will end up in the `data` tarball inside the .deb archive are kept in the `debian` directory, which also holds (all) files that will go into the `control` tarball. This is true, and it makes it easier to keep the Debian additions separate from the upstream source.

Cleaning up a bit
We start with removing the files that we are not going to need:


    rm -f rm *.ex *.EX README.Debian info docs

The file control

The description of what a control file must look like can be found in the DP section on control files. As you can see, our directory is going to produce two packages: a source package and a binary package. Edit the one dh_make provided you with and fill in the Homepage (if appropriate) and Description fields, like e.g.:

Source: hello
Section: unknown
Priority: extra
Maintainer: Jurjen Bokma <j.bokma@rug.nl>
Build-Depends: debhelper (>= 7), autotools-dev 
Standards-Version: 3.7.3
Homepage: http://www.gnu.org/software/hello/

Package: hello
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: option parsing, localized hello-world program
Unlike the minimal version often seen, GNU Hello processes its argument list to
 modify its behavior, supports greetings in many languages, and so on.
 The primary purpose of GNU Hello is to demonstrate how to write other programs
 that do these things; it serves as a model for GNU coding standards and GNU
 maintainer practices.

	A list of all sections for Lenny is available at http://packages.debian.org/stable/. The actual section names to be filled in here can be obtained by clicking on the hyperlinks in that page.
	The Debian FAQ has a short explanation on package priorities.
	If you are aware of any build-time dependencies not listed here, do add them.
	Most dependencies on libraries will be added automatically, others you should still list.
	The long description is indented by at least a space.
	The long indentation is not wrapped automatically, so mind your line length.

The changelog file

The changelog file must also be edited. Be descriptive but terse:

hello (2.2-1) unstable; urgency=low

  * Initial release (Closes: #9999) 

 -- Jurjen Bokma <j.bokma@rug.nl>  Mon, 07 Sep 2009 19:48:51 +0200

	This specifies the distribution the package is intended for. Some repository maintaining software (i.c. reprepro) will reject the package from a repository that it doesn't match.
	The “9999” is a dummy number, but the initial release needs to close and ITP
	Be careful with whitespace in this line. Lintian is quite sensitive.

Building the package

If we take a look at the files in our debian directory now, there are still a few ones we didn't touch:

ls -trl

      -rw-r--r-- 1  [...]  dirs -rw-r--r-- 1  [...]  compat -rw-r--r-- 1  [...]  copyright -rwxr-xr-x 1  [...]  rules -rw-r--r-- 1  [...]  info -rw-r--r-- 1  [...]  docs -rw-r--r-- 1  [...]  control -rw-r--r-- 1  [...]  changelog

Some we leave alone here to avoid cluttering the packaging flow. But the rules file is important! When we are going to call dpkg-buildpackage, it will use the rules to create the actual packages. The rules file is a Makefile which in turn will call many of the debhelper scripts. It will also call make with the Makefile that came with hello tarbundle, but only because the rules says so. If the particular software you are packaging doesn't come with a Makefile, and you don't want to create one but rather use whatever the tarbundle does come with, then the rules file is the one to adjust.

So now we cd out of the debian directory and back to the unpacked -now modified- tarbundle, and we run dpkg-buildpackage:

cd ..
dpkg-buildpackage -rfakeroot  -k'j.bokma@rug.nl'

      <snip humongous output>

	`Dpkg-buildpackage` needs to turn root now and then. `Fakeroot` is enough to make it succeed.
	Use your own mail address or other key identifier here. Forgot what you told `GPG`? gpg --list-keys

	Note
	Note that if you have set up a `~/.gnupg` with appropriate keys, as described in the section called “ Exercise 0: setting up for the exercises ”, `dpkg-buildpackage` will ask for your passphrase in order to sign the packages.

Analyzing the output

So let's see what we produced. A few files and directories have precipitated in ./debian. Note in particular ./debian/hello, into which the make install of the tarball has had its output directed.

ls -trl ./debian/

      total 44 -rw-r--r-- 1  [...]  19:48 dirs -rw-r--r-- 1  [...]  19:48 compat -rw-r--r-- 1  [...]  19:48 copyright -rwxr-xr-x 1  [...]  19:48 rules -rw-r--r-- 1  [...]  19:48 info -rw-r--r-- 1  [...]  19:48 docs -rw-r--r-- 1  [...]  13:28 control -rw-r--r-- 1  [...]  14:04 changelog drwxr-xr-x 4  [...]  15:50 hello -rw-r--r-- 1  [...]  15:50 hello.substvars -rw-r--r-- 1  [...]  15:50 files -rw-r--r-- 1  [...]  15:50 hello.debhelper.log

And in the parent directory, we have additional packages:

ls -trl ../

      total 876 -rw-r--r--  [...]  408308 2006-11-24 00:49 hello-2.2.tar.gz -rw-r--r--  [...]  408308 2006-11-24 00:49 hello_2.2.orig.tar.gz -rw-r--r--  [...]    2385 2009-09-11 15:50 hello_2.2-1.diff.gz -rw-r--r--  [...]     722 2009-09-11 15:50 hello_2.2-1.dsc drwxr-xr-x  [...]    4096 2009-09-11 15:50 hello-2.2 -rw-r--r--  [...]   57940 2009-09-11 15:50 hello_2.2-1_amd64.deb -rw-r--r--  [...]    1358 2009-09-11 15:50 hello_2.2-1_amd64.changes

	The original tarbundle that we downloaded.
	The file that the `debhelper` tools consider the original. In our case, it's just a copy of the actual original tarbundle.
	This file contains the patches that must be applied to the original bundle in order to get the source that is used for packaging. We didn't use it this time. But if you need to modify the Makefile that comes with the software in order to enable it to install into $(DESTDIR), or if you need to otherwise patch the original source to make it work, then this is where these changes end up.
	The source file header. Together with the `.orig.tar.gz` and the `.diff.gz`, this file forms the `Debian Source Package`.
	The directory which we have prepared. Inside it are the unpacked, patched original tarball, as well as the `debian` subdirectory that guided `dpkg-buildpackage` through creating the source and binary packages.
	By default, `dpkg-buildpackage` creates a binary package for the architecture it is running on. But it can sometimes cross-compile. On a 64-bit system, you can try e.g. fakeroot dpkg-buildpackage -b -ai386 to build i386 packages.
	The `.changes` file documents what changes have been make to the package(s) in the latest run. When the packages are uploaded to a repository, this file is needed to maintain the repository administration.

Let's dissect that name hello_2.2-1_amd64.deb... the package name is “hello”, the upstream version is “2.2”, the revision, or packaging version, is “1”, and the architecture is “amd64”.

Installing the package... ... or should we?
Now the packaging has succeeded, so the package must be good, and we can install it with dpkg -i ../hello_2.2-1_amd64.deb or by putting it in the local repository on our VM. Well... yes. We could install it, and that would work.
Note
If you feel particularly frivolous, go ahead and try.
But we don't, because the package doesn't yet comply with the standard set for .deb packages.
Checking the package for packaging errors


lintian ../hello_2.2-1.dsc

      W: hello source: out-of-date-standards-version 3.7.3 (current is 3.8.0)

lintian ../hello_2.2-1_amd64.deb

      W: hello: package-contains-empty-directory usr/sbin/ W: hello: copyright-lists-upstream-authors-with-dh_make-boilerplate W: hello: copyright-has-url-from-dh_make-boilerplate W: hello: copyright-contains-dh_make-todo-boilerplate W: hello: copyright-without-copyright-notice E: hello: section-is-dh_make-template

Oops. We've got to get rid of those.
Note
Use lintians -i option to find out what these warnings and errors really mean, and fix them.
Installing, uploading
So... you fixed debian/{dirs,copyright,control,changelog} removing boilerplate, and you re-ran dpkg-buildpackage, and now lintian doesn't complain any more. You can upload your package to the VM and install it there like we did in Step 6 of Procedure 2.1, “ The Manual Packaging of HW-0.1 ”.
If you would like to upload the package to a repository, e.g. one maintained with reprepro, then you would probably succeed. This time ate least... But once there is a version of your package in the repository, just repackaging and uploading the new version won't work unless you also indicate in the changelog that there is indeed a new version. So that is what we are going to do in the next section.

^[2] Yes, that's right. We're not picking the latest version.

	Note
	Use `lintian`s `-i` option to find out what these warnings and errors really mean, and fix them.


Exercise 1: Manual Packaging (optional)		Exercise 3: Packaging Update with dpkg-buildpackage

	Note
	If you feel particularly frivolous, go ahead and try.