# Add multi-packages support to pkgsrc
In this diary it will be discussed the Google Summer of Code 2017 experience in
splitting packages for pkgsrc builds (i.e. from a pkgsrc package generate
multiple binary packages, AKA SUBPACKAGES support).

You can read the original proposal of the GSoC at the following URL:

 <http://netbsd.org/~leot/misc/gsoc2017/gsoc-multipkg.pdf>


## Rebasing `debugpkg' branch against the current pkgsrc-current
After one year of changes (and some forced-commit!) rebasing `debugpkg' branch
to current pkgsrc-current wasn't trivial. Fortunately, thanks to <Riastradh> we
were able to rebase it with the following incantations:

 % git format-patch --stdout 0bb46e283c3..debugpkg > ../debugpkg.patch
 % git checkout -b debugpkg1-base a50518ba905c164ccbef9847b4b98caf0c4a73e1
 % git checkout -b debugpkg1 debugpkg1-base
 % git am < ../debugpkg.patch
 % git branch debugpkg2-base upstream/trunk
 % git checkout -b debugpkg2 debugpkg1
 % git rebase debugpkg2-base


## More _CONTENTS_TARGETS spkg-ification
Several _CONTENTS_TARGETS (in mk/pkgformat/pkg/metadata.mk) were spkg-ified and
where needed a per-spkg package variable was added:

 - DESCR.<spkg>
 - COMMENT.<spkg>
 - PKG_PRESERVE.<spkg>
 - PKGWILDCARD.<spkg>

Most of the `pkg' pkgformat is per-spkg, notable exceptions to that are:

 - _INSTALL_FILE
 - _DEINSTALL_FILE

(regarding _{,DE}INSTALL_FILE this also means that at the moment the entire
pkginstall framework isn't spkg-ready).

Also _BUILD_VERSION_FILE wasn't spkg-ifed because probably it doesn't make a lot
of sense to have it per-spkg.


## `deinstall' and `replace' target support for SUBPACKAGES
Similarly to `install' phase that install every spkg, `deinstall' and `replace'
targets were updated to respectively deinstall all spkgs and replace all spkgs.


## BUILD_DEPENDS, DEPENDS, TOOL_DEPENDS, ABI_DEPENDS, BUILD_ABI_DEPENDS subpackages-ification
BUILD_DEPENDS, DEPENDS and TOOL_DEPENDS are initialized in mk/bsd.pkg.mk.
ABI_DEPENDS and BUILD_ABI_DEPENDS are respectively appended to DEPENDS and
BUILD_DEPENDS, always in mk/bsd.pkg.mk (they're mostly used by the buildlink3
framework).

BUILD_DEPENDS.<spkg>, DEPENDS.<spkg> and TOOL_DEPENDS.<spkg> were added
similarly in order to be satisfied per-<spkg>. In other words for a pkgsrc
package every subpackages will have BUILD_DEPENDS, DEPENDS and TOOL_DEPENDS
as usual plus its {BUILD_,,TOOL_}DEPENDS.<spkg> counterparts.
Similarly ABI_DEPENDS.<spkg> and BUILD_ABI_DEPENDS.<spkg> are appended to
DEPENDS.<spkg> and BUILD_DEPENDS.<spkg>, respectively.

_ALL_DEPENDS list in mk/bsd.utils.mk was updated in order to also
include every per-spkg dependencies.

BOOTSTRAP_DEPENDS (defined in mk/depends/bsd.depends-vars.mk) will probably
*not* need to be per-spkg. Actually it's used only for the "fetcher" in
mk/fetch/bsd.fetch-vars.mk and for pkg_install in
mk/pkgformat/pkg/pkgformat-vars.mk.

mk/pkgformat/pkg/depends.mk was mostly completely adjusted to be per-spkg. A
notable exception to it is BOOTSTRAP_DEPENDS (and _LIST_DEPENDS_CMD.bootstrap),
for the same reasons written in the previous paragraph.

_DEPENDS_FILE, _RDEPENDS_FILE, _RRDEPENDS_FILE are per-spkg and
TOOL_DEPENDS.<spkg>, BUILD_DEPENDS.<spkg> and DEPENDS.<spkg> are now
recorded via the _{,R,RR}DEPENDS_FILE.<spkg>.


## CONFLICTS subpackages-ification
Similarly to DEPENDS, both CONFLICTS and CONFLICTS.<spkg> are supported.
CONFLICTS works as usual and is honored by every spkgs while CONFLICTS.<spkg> is
honored by just a single <spkg>.


## buildlink3 subpackages-ification
From a MAINTAINER POV, when including bl3 files optional BUILDLINK_TREE special
`spkg' markers can be appended to the BUILDLINK_TREE to restrict propagation of
*DEPENDS only to certain subpackages, i.e.:

 [...]
 BUILDLINK_TREE+=	spkg:<spkg_1,...,spkg_n>
 .include "../../category/foo/buildlink3.mk"
 BUILDLINK_TREE+=	-spkg:<spkg_1,...,spkg_n>
 [...]

...will limit `foo' *DEPENDS only to spkg_1, ..., spkg_n SUBPACKAGES of the
current package.

From a pkgsrc developer POV, mk/buildlink3/bsd.buildlink3.mk was adjusted for
SUBPACKAGES and several internal variables were adjusted.
BUILDLINK_TREE can contains optional `spkg:' markers to restrict *DEPENDS
propagation only to certain subpackages.

_BLNK_PACKAGES was adjusted to always contains spkgs meta-information as
follows:

 <spkg_1,...,spkg_n>:<pkg>

If there aren't any SUBPACKAGES or there aren't any `spkg:' markers in
BUILDLINK_TREE, `_all' is used instead:

 _all:<pkg>

_BLNK_DEPENDS was adjusted to have same semantic of _BLNK_PACKAGES.

Please note that spkg meta-information are used only for *DEPENDS propagation.
All other places of buildlink3 framework ignores that.


## pkginstall subpackages-ification
All the pkginstall framework was spkg-ified and inline documentation was updated
to also reflect SUBPACKAGES changes. For more information please directly give a
look to them.

Exceptions to that are (components not spkg-ified):

 - OCAML_FINDLIB_REGISTER
 - PKG_SHELL
 - LDCONFIG_ADD_CMD and LDCONFIG_REMOVE_CMD
 - FONTS_DIRS.<type>

(all of them are documented about that with an `XXXleot' comment, probably for
most of them having SUBPACKAGES support isn't very useful).

Some variables defined in pkginstall framework but used also outside worth
mentioning:

 - FILES_SUBST.<spkg>: defined as FILES_SUBST (that is shared by all spkgs) plus
   any possible changes relevant only for a particular <spkg>.