Chapter 12. Package components - files, directories and contents

Table of Contents

12.1. Makefile
12.2. distinfo
12.3. patches/*
12.3.1. Structure of a single patch file
12.3.2. Creating patch files
12.3.3. Sources where the patch files come from
12.3.4. Patching guidelines
12.3.5. Feedback to the author
12.4. Other mandatory files
12.5. Optional files
12.5.1. Files affecting the binary package
12.5.2. Files affecting the build process
12.5.3. Files affecting nothing at all
12.6. work*
12.7. files/*

Whenever you're preparing a package, there are a number of files involved which are described in the following sections.

12.1. Makefile

Building, installation and creation of a binary package are all controlled by the package's Makefile. The Makefile describes various things about a package, for example from where to get it, how to configure, build, and install it.

A package Makefile contains several sections that describe the package.

In the first section there are the following variables, which should appear exactly in the order given here. The order and grouping of the variables is mostly historical and has no further meaning.

  • DISTNAME is the basename of the distribution file to be downloaded from the package's website.

  • PKGNAME is the name of the package, as used by pkgsrc. You need to provide it if DISTNAME (which is the default) is not a good name for the package in pkgsrc or DISTNAME is not provided (no distribution file is required). Usually it is the pkgsrc directory name together with the version number. It must match the regular expression ^[A-Za-z0-9][A-Za-z0-9-_.+]*$, that is, it starts with a letter or digit, and contains only letters, digits, dashes, underscores, dots and plus signs. New packages should have entirely lower-case names, with the exception of some that must follow pre-existing conventions (e.g. R packages begin with R-). You can use the :tl variable modifier to lower-case the package name.

  • CATEGORIES is a list of categories which the package fits in. You can choose any of the top-level directories of pkgsrc for it.

    Currently the following values are available for CATEGORIES. If more than one is used, they need to be separated by spaces:

    archivers     cross         geography     meta-pkgs     security
    audio         databases     graphics      misc          shells
    benchmarks    devel         ham           multimedia    sysutils
    biology       editors       inputmethod   net           textproc
    cad           emulators     lang          news          time
    chat          finance       mail          parallel      wm
    comms         fonts         math          pkgtools      www
    converters    games         mbone         print         x11
    
  • MASTER_SITES, DYNAMIC_MASTER_SITES, DIST_SUBDIR, EXTRACT_SUFX and DISTFILES are discussed in detail in Section 13.5, “The fetch phase”.

The second section contains information about separately downloaded patches, if any.

  • PATCHFILES: Name(s) of additional files that contain distribution patches. There is no default. pkgsrc will look for them at PATCH_SITES. They will automatically be uncompressed before patching if the names end with .gz or .Z.

  • PATCH_SITES: Primary location(s) for distribution patch files (see PATCHFILES above) if not found locally.

  • PATCH_DIST_STRIP: an argument to patch(1) that sets the pathname strip count to help find the correct files to patch. It defaults to -p0.

The third section contains the following variables.

  • MAINTAINER is the email address of the person who is currently responsible for this package. In theory, the maintainer will be aware of upstream release, keep it updated, and generally resolve problems. Others contemplating significant changes should ask the maintainer for advice or review of a proposed change. This is not necessary for micro updates, bug fixes, pkgsrc-wide changes affecting many packages, and other things expected to be uncontroversial. It goes with this "should ask" that the maintainer should respond to inquiries reasonably promptly (most of the time), and either approve changes or provide reasons why not. Typically, when packaging a new program, set MAINTAINER to yourself. If you don't intend to maintain the package (especially if you feel you have no special insight), set it to . This special value is a code point indicating that there is no maintainer, but the variable is mandatory, absent OWNER.

  • OWNER can be used instead of MAINTAINER in relatively rare circumstances when the package is such that modifying it is riskier than normal. This is a stronger form of being the maintainer, with an expectation that others will ask for almost all changes, except revbumps and accommodating structural changes. Someone who becomes OWNER has a strong obligation to keep the package very well maintained, to answer queries promptly and to be a steward of the package on behalf of users. In this case, MAINTAINER should be omitted.

    Because of the friction that OWNER adds, PMC may downgrade it to MAINTAINER at any time.

  • HOMEPAGE is a URL where users can find more information about the package.

  • COMMENT is a one-line description of the package (should not include the package name).

  • LICENSE indicates the license(s) applicable for the package. See Section 21.1.3, “Handling licenses” for further details.

Other variables that affect the build:

  • WRKSRC: The directory where the interesting distribution files of the package are found. The default is ${WRKDIR}/${DISTNAME}, which works for most packages.

    If a package doesn't create a subdirectory for itself (most GNU software does, for instance), but extracts itself in the current directory, you should set WRKSRC=${WRKDIR}.

    If a package doesn't create a subdirectory with the name of DISTNAME but some different name, set WRKSRC to point to the proper name in ${WRKDIR}, for example WRKSRC=${WRKDIR}/${DISTNAME}/unix. See lang/tcl and x11/tk for other examples.

    The name of the working directory created by pkgsrc is taken from the WRKDIR_BASENAME variable. By default, its value is work. If you want to use the same pkgsrc tree for building different kinds of binary packages, you can change the variable according to your needs. Two other variables handle common cases of setting WRKDIR_BASENAME individually. If OBJHOSTNAME is defined in mk.conf, the first component of the host's name is attached to the directory name. If OBJMACHINE is defined, the platform name is attached, which might look like work.i386 or work.sparc.

Please pay attention to the following gotchas:

  • Add MANCOMPRESSED if man pages are installed in compressed form by the package. For packages using BSD-style makefiles which honor MANZ, there is MANCOMPRESSED_IF_MANZ.

  • Replace /usr/local with ${PREFIX} in all files (see patches, below).

  • If the package installs any info files, see Section 21.6.8, “Packages installing info files”.

12.2. distinfo

The distinfo file contains the message digest, or checksum, of each distfile needed for the package. This ensures that the distfiles retrieved from the Internet have not been corrupted during transfer or altered by a malign force to introduce a security hole. To provide maximum security, all distfiles are protected using two different message digest algorithms (BLAKE2s and SHA512), as well as the file size.

The distinfo file also contains the checksums for all the patches found in the patches directory (see Section 12.3, “patches/*). These checksums ensure that patches are only applied intentionally and that they don't accidentally change, e.g. when merging different changes together. They also make sure that new patches are actually added to CVS and old ones are removed. Too see whether the patches and the distinfo file match, run pkglint after changing the patches.

To regenerate the distinfo file, use the make distinfo command.

Some packages have different sets of distfiles depending on the platform, for example lang/openjdk8. These are kept in the same distinfo file and care should be taken when upgrading such a package to ensure distfile information is not lost.

12.3. patches/*

Some packages don't work out-of-the box on the various platforms that are supported by pkgsrc. These packages need to be patched to make them work. The patch files can be found in the patches/ directory.

In the patch phase, these patches are applied to the files in WRKSRC directory after extracting them, in alphabetic order.

12.3.1. Structure of a single patch file

The patch-* files should be in diff -bu format, and apply without a fuzz to avoid problems. (To force patches to apply with fuzz you can set PATCH_FUZZ_FACTOR=-F2). Furthermore, each patch should contain only changes for a single file, and no file should be patched by more than one patch file. This helps to keep future modifications simple.

Each patch file is structured as follows: In the first line, there is the RCS Id of the patch itself. The second line should be empty for aesthetic reasons. After that, there should be a comment for each change that the patch does. There are a number of standard cases:

  • Patches for commonly known vulnerabilities should mention the vulnerability ID (CAN, CVE).

  • Patches that change source code should mention the platform and other environment (for example, the compiler) that the patch is needed for.

The patch should be commented so that any developer who knows the code of the application can make some use of the patch. Special care should be taken for the upstream developers, since we generally want that they accept our patches, so we have less work in the future.

12.3.2. Creating patch files

One important thing to mention is to pay attention that no RCS IDs get stored in the patch files, as these will cause problems when later checked into the NetBSD CVS tree. Use the pkgdiff command from the pkgtools/pkgdiff package to avoid these problems.

For even more automation, we recommend using mkpatches from the same package to make a whole set of patches. You just have to back up files before you edit them to filename.orig, e.g., with cp -p filename filename.orig or, easier, by using pkgvi again from the same package. If you upgrade a package this way, you can easily compare the new set of patches with the previously existing one with patchdiff. The files in patches are replaced by new files, so carefully check if you want to take all the changes.

When you have finished a package, remember to generate the checksums for the patch files by using the make makepatchsum command, see Section 12.2, “distinfo.

When adding a patch that corrects a problem in the distfile (rather than e.g. enforcing pkgsrc's view of where man pages should go), send the patch as a bug report to the maintainer. This benefits non-pkgsrc users of the package, and usually makes it possible to remove the patch in future version.

The file names of the patch files are usually of the form patch-path_to_file__with__underscores.c. Many packages still use the previous convention patch-[a-z][a-z], but new patches should be of the form containing the filename. mkpatches included in pkgtools/pkgdiff takes care of the name automatically.

When updating pre-existing patch files, if a file uses the old patch-[a-z][a-z] convention, it's best not to change it to the new form, as that will just cause churn that makes it harder to track changes to patching over time. Similarly, if a patch now applies at different line offsets, but still applies cleanly as-is, there's no need to update it, as that also unnecessarily complicates the patch history.

12.3.3. Sources where the patch files come from

If you want to share patches between multiple packages in pkgsrc, e.g. because they use the same distfiles, set PATCHDIR to the path where the patch files can be found, e.g.:

PATCHDIR=       ../../editors/xemacs/patches

Patch files that are distributed by the author or other maintainers can be listed in PATCHFILES.

If it is desired to store any patches that should not be committed into pkgsrc, they can be kept outside the pkgsrc tree in the $LOCALPATCHES directory. The directory tree there is expected to have the same category/package structure as pkgsrc, and patches are expected to be stored inside these dirs (also known as $LOCALPATCHES/$PKGPATH). For example, if you want to keep a private patch for pkgsrc/graphics/png, keep it in $LOCALPATCHES/graphics/png/mypatch. All files in the named directory are expected to be patch files, and they are applied after pkgsrc patches are applied.

12.3.4. Patching guidelines

When fixing a portability issue in the code do not use preprocessor magic to check for the current operating system nor platform. Doing so hurts portability to other platforms because the OS-specific details are not abstracted appropriately.

The general rule to follow is: instead of checking for the operating system the application is being built on, check for the specific features you need. For example, instead of assuming that kqueue is available under NetBSD and using the __NetBSD__ macro to conditionalize kqueue support, add a check that detects kqueue itself — yes, this generally involves patching the configure script. There is absolutely nothing that prevents some OSes from adopting interfaces from other OSes (e.g. Linux implementing kqueue), something that the above checks cannot take into account.

Of course, checking for features generally involves more work on the developer's side, but the resulting changes are cleaner and there are chances they will work on many other platforms. Not to mention that there are higher chances of being later integrated into the mainstream sources. Remember: It doesn't work unless it is right!

Some typical examples:

Table 12.1. Patching examples

Where Incorrect Correct
configure script
case ${target_os} in
netbsd*) have_kvm=yes ;;
*)       have_kvm=no  ;;
esac
AC_CHECK_LIB(kvm, kvm_open, have_kvm=yes, have_kvm=no)
C source file
#if defined(__NetBSD__)
#  include <sys/event.h>
#endif
#if defined(HAVE_SYS_EVENT_H)
#  include <sys/event.h>
#endif
C source file
int
monitor_file(...)
{
#if defined(__NetBSD__)
        int fd = kqueue();
        ...
#else
        ...
#endif
}
int
monitor_file(...)
{
#if defined(HAVE_KQUEUE)
        int fd = kqueue();
        ...
#else
        ...
#endif
}

12.3.5. Feedback to the author

Always, always, always feed back any portability fixes or improvements you do to a package to the mainstream developers. This is the only way to get their attention on portability issues and to ensure that future versions can be built out-of-the box on NetBSD. Furthermore, any user that gets newer distfiles will get the fixes straight from the packaged code.

This generally involves cleaning up the patches (because sometimes the patches that are added to pkgsrc are quick hacks), filing bug reports in the appropriate trackers for the projects and working with the mainstream authors to accept your changes. It is extremely important that you do it so that the packages in pkgsrc are kept simple and thus further changes can be done without much hassle.

When you have done this, please add a URL to the upstream bug report to the patch comment.

Support the idea of free software!

12.4. Other mandatory files

DESCR

A multi-line description of the piece of software. This should include any credits where they are due. Please bear in mind that others do not share your sense of humour (or spelling idiosyncrasies), and that others will read everything that you write here.

PLIST

This file governs the files that are installed on your system: all the binaries, manual pages, etc. There are other directives which may be entered in this file, to control the creation and deletion of directories, and the location of inserted files. See Chapter 19, PLIST issues for more information.

12.5. Optional files

12.5.1. Files affecting the binary package

INSTALL

This shell script is invoked twice by pkg_add(1). First time after package extraction and before files are moved in place, the second time after the files to install are moved in place. This can be used to do any custom procedures not possible with @exec commands in PLIST. See pkg_add(1) and pkg_create(1) for more information. See also Section 20.1, “Files and directories outside the installation prefix”. Please note that you can modify variables in it easily by using FILES_SUBST in the package's Makefile:

FILES_SUBST+=  SOMEVAR="somevalue"

replaces "@SOMEVAR@" with somevalue in the INSTALL. By default, substitution is performed for PREFIX, LOCALBASE, X11BASE, VARBASE, and a few others, type make help topic=FILES_SUBST for a complete list.

DEINSTALL

This script is executed before and after any files are removed. It is this script's responsibility to clean up any additional messy details around the package's installation, since all pkg_delete knows is how to delete the files created in the original distribution. See pkg_delete(1) and pkg_create(1) for more information. The same methods to replace variables can be used as for the INSTALL file.

MESSAGE

This file is displayed after installation of the package. While this was used often in the past, it has two problems: the display will be missed if many packages are intalled at once, and the person installing the package and the one using or configuring it may be different. It should therefore be used only in exceptional circumstances where lasting negative consequences would result from someone not reading it.

MESSAGE should not be used for:

  • exhortations to read the documentation

  • reminders to install rc.d files and set variables

  • anything that should be explained in the installation/configuration documentation that should come with the package

If the documentation provided by upstream needs enhancing, create e.g. files/README.pkgsrc and install it in the package's documentation directory.

Note that MESSAGE is shown for all operating systems and all init systems. If a MESSAGE is necessary, it should be narrowed to only those operating systems and init systems to which it applies.

Note that you can modify variables in it easily by using MESSAGE_SUBST in the package's Makefile:

MESSAGE_SUBST+=  SOMEVAR="somevalue"

replaces "${SOMEVAR}" with somevalue in MESSAGE. By default, substitution is performed for PKGNAME, PKGBASE, PREFIX, LOCALBASE, X11BASE, PKG_SYSCONFDIR, ROOT_GROUP, and ROOT_USER.

You can display a different or additional files by setting the MESSAGE_SRC variable. Its default is MESSAGE, if the file exists.

ALTERNATIVES

This file is used by the alternatives framework. It creates, configures, and destroys generic wrappers used to run programs with similar interfaces. See pkg_alternatives(8) from pkgtools/pkg_alternatives for more information.

Each line of the file contains two filenames, first the wrapper and then the alternative provided by the package. Both paths are relative to PREFIX.

12.5.2. Files affecting the build process

Makefile.common

This file contains arbitrary things that could also go into a Makefile, but its purpose is to be used by more than one package. This file should only be used when the packages that will use the file are known in advance. For other purposes it is often better to write a *.mk file and give it a good name that describes what it does.

buildlink3.mk

This file contains the dependency information for the buildlink3 framework (see Chapter 18, Buildlink methodology).

hacks.mk

This file contains workarounds for compiler bugs and similar things. It is included automatically by the pkgsrc infrastructure, so you don't need an extra .include line for it.

options.mk

This file contains the code for the package-specific options (see Chapter 16, Options handling) that can be selected by the user. If a package has only one or two options, it is equally acceptable to put the code directly into the Makefile.

12.5.3. Files affecting nothing at all

README*

These files do not take place in the creation of a package and thus are purely informative to the package developer.

TODO

This file contains things that need to be done to make the package even better.

12.6. work*

When you type make, the distribution files are unpacked into the directory denoted by WRKDIR. It can be removed by running make clean. Besides the sources, this directory is also used to keep various timestamp files. The directory gets removed completely on clean. The default is ${.CURDIR}/work or ${.CURDIR}/work.${MACHINE_ARCH} if OBJMACHINE is set.

12.7. files/*

If you have any files that you wish to be placed in the package prior to configuration or building, you can place these files here and use a ${CP} command in the post-extract target to achieve this.

If you want to share files in this way with other packages, set the FILESDIR variable to point to the other package's files directory, e.g.:

FILESDIR=       ../../editors/xemacs/files