pkgsrc on Solaris

Sep 12, 2009
tags: pkgsrc, solaris, zones

For many years, building and installing third-party software on Solaris has been a huge pain. For people who do not use pkgsrc, that is.

Originating from the NetBSD project, pkgsrc is a source-based cross-platform package manager. If you’ve used FreeBSD ports, then it is very similar as it derives from the same codebase, so the basic premise is that you:

$ cd /usr/pkgsrc/www/apache22
$ make package

and pkgsrc will download the source for Apache 2.2, compile it and all dependancies, install it on the local system and create a package which can be installed on other similar systems.

However, we’ve taken ports further and applied the NetBSD philosophy of portability, meaning that it not only works on NetBSD, but across all *BSD as well as Linux, OSX, HP/UX, AIX, IRIX, QNX, Windows (via Interix), and of course Solaris.

So while apt-get might be awesome, it only really works on Linux. FreeBSD might have way more ports than us, but only runs on FreeBSD and OSX. pkgsrc provides a consistent interface across all the platforms listed above, and in some cases provides a superior package manager than the system provides.

Here’s how I use pkgsrc on Solaris, in this specific case Solaris 10/x86. Paths are specific to my setup, you can of course change them.

Create a chroot/zone environment

I use the zones feature of Solaris 10 to ensure that all packages are built in a sandbox. This has a number of benefits:

  • The running system is unaffected by the builds, in that they are not writing to the same file system. This is good when you have misbehaving packages.
  • You can separate the build and install phases, so that you can verify all the packages have been built and are correct before starting any install/upgrade procedure
  • It’s easier to catch package mistakes, e.g. unpackaged files.
  • It avoids pollution from the host environment which may produce bad packages

For creating a zone, I wrote the following create-zone script:

#!/bin/sh

DOMAIN="adsl.perkin.org.uk"
MASQDOMAIN="perkin.org.uk"
PHYSIF="e1000g0"
RPOOL="gromit"

if [ $# -eq 1 ]; then
    name=$1; shift
    ipaddr=`getent hosts ${name} | awk '{print $1}'`
    if [ -z "${ipaddr}" ]; then
        echo "ERROR: Could not determine IP address of $name"
        echo "Either add to hosts database or provide on command line"
        echo
        echo "usage: $0 <name> [ <ipaddr> ]"
        exit 2
    fi
elif [ $# -eq 2 ]; then
    name=$1; shift
    ipaddr=$1; shift
else
    echo "usage: $0 <name> [ <ipaddr> ]"
    exit 2
fi

zfs list ${RPOOL}/zones/$name >/dev/null 2>&1
if [ $? -eq 0 ]; then
    echo "${RPOOL}/zones/$name already exists, not continuing"
    exit 1
fi

# Inherited directories are read-only
cat >/tmp/zonecfg.$$ << EOF
    create
    set zonepath=/zones/$name
    set autoboot=true
    add net
        set address=$ipaddr/24
        set physical=${PHYSIF}
    end
    verify
    commit
    exit
EOF

zonecfg -z $name -f /tmp/zonecfg.$$
rm /tmp/zonecfg.$$

zoneadm -z $name install

#
# Automatically configure new zone with most of the settings from the current
# global, grabbing the encrypted root password directly.  Note that the
# service_profile keyword doesn't seem to be used here, so we configure the
# limited_net profile manually and optionally install a custom site.xml which
# is parsed and activated during first boot.
#
cat << EOF >/zones/$name/root/etc/sysidcfg
network_interface=PRIMARY
{
    hostname=$name.${DOMAIN}
}
name_service=DNS
{
    domain_name=${DOMAIN}
    name_server=`awk '/^nameserver/ { print $NF; exit }' /etc/resolv.conf`
}
nfs4_domain=dynamic
root_password=`awk -F: '/^root/ {print $2}' /etc/shadow`
security_policy=NONE
service_profile=limited_net
system_locale=C
terminal=xterm
timezone=Europe/London
EOF

#
# Use the limited_net profile and install custom site.xml if provided.
#
rm /zones/${name}/root/var/svc/profile/generic.xml
ln -s generic_limited_net.xml /zones/${name}/root/var/svc/profile/generic.xml
if [ -f /install/zones/${name}.xml ]; then
    cp /install/zones/${name}.xml /zones/${name}/root/var/svc/profile/site.xml
fi

#
# Disable nscd host cache
#
ex /zones/$name/root/etc/nscd.conf >/dev/null 2>&1 <<EOF
/enable-cache/s/^#//
wq
EOF

#
# Configure sendmail to masquerade and route via smarthost.
#
ex /zones/$name/root/etc/mail/cf/cf/submit.mc >/dev/null 2>&1 <<EOF
/^dnl/
a
FEATURE(\`masquerade_envelope')dnl
MASQUERADE_AS(\`${MASQDOMAIN}')dnl
dnl
.
/^FEATURE.*msp/s/.127.0.0.1./mail.perkin.org.uk/
wq!
EOF
(
  cd /zones/$name/root/etc/mail/cf/cf
  /usr/ccs/bin/make submit.cf >/dev/null 2>&1
)
cp /zones/$name/root/etc/mail/cf/cf/submit.cf \
   /zones/$name/root/etc/mail/submit.cf

#
# Use SHA512 crypted passwords, and change root's home directory.
#
ex /zones/$name/root/etc/security/policy.conf >/dev/null 2>&1 <<EOF
/^CRYPT_DEFAULT/s/__unix__/6/
wq
EOF
mkdir -m 0700 /zones/$name/root/root
ex /zones/$name/root/etc/passwd >/dev/null 2>&1 <<EOF
/^root/s,:/:,:/root:,
/^root/s,:/sbin/sh,:/bin/bash,
wq
EOF

cp /root/.bash_profile /zones/$name/root/root/.bash_profile

#
# Allow root login over SSH (rest of network takes care of external access
# so this isn't a problem) and enforce SSH key authentication.
#
ex /zones/$name/root/etc/ssh/sshd_config >/dev/null 2>&1 <<EOF
/^PermitRootLogin/s/no$/yes/
/^PasswordAuthentication/s/yes$/no/
wq
EOF
mkdir -m 0700 /zones/$name/root/root/.ssh
cp /home/jperkin/.ssh/id_rsa.pub /zones/$name/root/root/.ssh/authorized_keys

#
# Automount shared directories from global zone using direct mounts
#
ex /zones/$name/root/etc/auto_master >/dev/null 2>&1 <<EOF
a
/-        auto_direct
.
wq
EOF
ex /zones/$name/root/etc/auto_direct >/dev/null 2>&1 <<EOF
a
/content    gromit-lan:/content
/install    gromit-lan:/install
.
wq
EOF

#
# All done!  Fire it up...
#
zoneadm -z $name boot

And the corresponding delete-zone script is:

#!/bin/sh

RPOOL="gromit"

if [ $# -ne 1 ]; then
    echo "usage: $0 <zone>"
    exit 2
fi

zoneadm -z $1 halt
zonecfg -z $1 delete -F
zfs destroy ${RPOOL}/zones/$1

If you want to use them then there are some variables to set at the top, and you may want to scan through them for additional bits to change, for example create-zone copies my ssh public key which will most likely be wrong for your setup :-)

One additional piece of configuration for create-zone is an optional SMF xml file. I use this file to disable inetd inside the zone for additional security, like so:

<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='profile' name='extract'>
  <service name='network/inetd' type='service' version='0'>
    <instance name='default' enabled='false'/>
  </service>
</service_bundle>

The file should be named <yourzonename>.xml. Mine is named vm-generic.xml and I then create symlinks to it for each VM I want with that default configuration.

Fetch pkgsrc

pkgsrc is developed very rapidly. Tracking nearly 9,000 pieces of third-party software means there are always many updates. Thankfully, we provide quarterly branches for people who want more stability, and I recommend using the latest quarterly release. At time of writing, this is known as pkgsrc-2009Q2. Within the next month or so we will release pkgsrc-2009Q3, and you can figure out the names of future releases yourself.

The easiest way to get pkgsrc is using cvs. I keep stuff like this under /content as opposed to the default of /usr, you can use whatever you wish but will need to change all my example scripts to match where you put it.

$ cd /content
$ BRANCH="pkgsrc-2009Q2"
$ cvs -d anoncvs@anoncvs.netbsd.org:/cvsroot co -r${BRANCH} -d${BRANCH} -P pkgsrc

Alternatively, you can fetch either bzip2 or gzip archives of the current branch. I recommend the cvs method as, with the branch being updated for security fixes and other important changes, you can easily track it using

$ cd /content/pkgsrc-2009Q2
$ cvs update

pkgsrc configuration

pkgsrc is configured using a mk.conf file, this is mine:

#
# Local settings
#
OS_HOST!= uname -n

#
# Per-host settings
#
#    vm0 builds local packages
#
.if !empty(OS_HOST:Mvm0.*) || !empty(OS_HOST:Mgromit.*)
PKGDIRSUFFIX=
PKG_DEFAULT_OPTIONS=    inet6 perl sasl ssl ncursesw
PKG_OPTIONS.mutt+=      mutt-hcache mutt-smtp
PKG_UID.cyrus=          2000
#PKG_GID.cyrus=         mail
PKG_SHELL.cyrus=        /usr/bin/false
PKG_UID.www=            2001
PKG_GID.www=            2001
PYTHON_VERSION_DEFAULT= 24
X11_TYPE=               modular
#
#    vm1 modular X11 bulk builds
#
.elif !empty(OS_HOST:Mvm1.*)
X11_TYPE=       modular
PKGDIRSUFFIX=   -${X11_TYPE}
#
#    vm2 native X11 bulk builds
#
.elif !empty(OS_HOST:Mvm2.*)
X11_TYPE=       native
PKGDIRSUFFIX=   -${X11_TYPE}
#
#    vm3 pkgsrc dev builds
#
.elif !empty(OS_HOST:Mvm3.*)
X11_TYPE=       modular
PKGDIRSUFFIX=   -${X11_TYPE}
.endif

.if exists(../../CVS/Tag)
PKG_BRANCH!=    ${TOOLS_PLATFORM.awk} -F- '{print $$2}' < ../../CVS/Tag
.else
PKG_BRANCH=     HEAD
.endif

ALLOW_VULNERABLE_PACKAGES=  YES

BULKFILESDIR=           ${WRKOBJDIR}
DISTDIR=                /install/pkgsrc/distfiles/
FAILOVER_FETCH=         YES
INSTALL_UNSTRIPPED=     YES
LINTPKGSRC_DB=          ${_PKGSRCDIR}/.lintpkgsrc.db
MASTER_SITE_OVERRIDE=   ftp://ftp.netbsd.org/pub/NetBSD/packages/distfiles/
PACKAGES=               /install/pkgsrc/packages/${PKG_BRANCH}${PKGDIRSUFFIX}
PKGCHK_CONF=            /install/pkgsrc/misc/pkgchk.conf
SKIP_LICENSE_CHECK=     YES
WRKOBJDIR=              /tmp/pkgsrc

#
# Parse pkgchk.conf and supply list of packages for the bulk build framework.
#
.if defined(SPECIFIC_PKGS)
PKGLIST!= ${TOOLS_PLATFORM.awk} '$$1 !~ /^(\#|$$)/ {print $$1}' ${PKGCHK_CONF}
.  for _pkg_ in ${PKGLIST}
HOST_SPECIFIC_PKGS+= ${_pkg_}
.  endfor
.endif

This is the primary configuration file for pkgsrc. Again, you may need to tailor this to your environment, and may find it useful to read the pkgsrc guide to understand what it all means.

As I do a lot of pkgsrc development I have a number of virtual machines up and running doing various bits and pieces. Obviously I don’t want to copy that mk.conf around, so I also have a small fragment file which is appended to each virtual machine’s mk.conf (using the --mk-fragment argument to bootstrap) and includes the global copy:

.include "/install/pkgsrc/misc/mk.conf"

The bulk build setup in pkgsrc requires its own configuration, and for this you will need to edit a file inside pkgsrc. There is an example file provided, so what I usually do is symlink this to the real copy then I can easily keep it up-to-date via cvs.

$ cd /content/pkgsrc-2009Q2/mk/bulk
$ ln -s build.conf-example build.conf
$ vi build.conf

Again, you can find my personal build.conf here.

Finally, there is a configuration file for pkg_chk which is a package inside pkgsrc which makes managing upgrades easier (ideally it should be a part of the main pkgsrc tools but that’s for another day). pkgchk.conf is a list of package directories, relative to the pkgsrc top level, which are to be built and installed for this setup. If you have a large installation then pkg_chk has extra features to make it possible to share pkgchk.conf across a number of machines and configure packages on a per-host, per-OS etc basis.

Thus, a sample pkgchk.conf:

chat/irssi
chat/irssi-icb
converters/xlhtml
databases/lbdb
databases/rrdtool
devel/autoconf
devel/automake
devel/bzr
devel/bzrtools
devel/diffutils
devel/libtool-base
devel/scmcvs
editors/vim
mail/mutt-devel
misc/screen
net/p5-Net-SNMP
net/p5-Net-SNMP-Interfaces
net/rsync
net/samba
net/tnftp
pkgtools/digest
pkgtools/pkg_chk
pkgtools/rc.subr
print/mp
security/cy2-login
security/cy2-plain
security/sudo
sysutils/coreutils
textproc/antiword
textproc/grep
textproc/urlview
www/apache22
www/w3m

It is highly likely you will want to change the pkgchk.conf file from what I use :-)

Build scripts

Once everything is set up, I have two scripts to build then update my packages, intuitively called build-packages:

#!/bin/sh

PATH="/opt/pkg/sbin:/opt/pkg/bin:/sbin:/usr/sbin:/usr/bin:/usr/ccs/bin"
export PATH

BRANCH="2009Q2"
export BRANCH

PDIR=/install/pkgsrc

if [ ! -f ${PDIR}/bootstrap-${BRANCH}.tar ]; then
    cd /content/pkgsrc-${BRANCH}/bootstrap
    ./bootstrap --workdir=/tmp/pkgsrc --prefix=/opt/pkg \
                --varbase=/var/opt/pkg --sysconfdir=/etc/opt/pkg \
                --mk-fragment=${PDIR}/misc/mk-include.conf \
                --binary-kit=${PDIR}/bootstrap-${BRANCH}.tar
    rm -rf /tmp/pkgsrc
fi

if [ ! -f /opt/pkg/bin/bmake ]; then
    (cd /; tar -xf ${PDIR}/bootstrap-${BRANCH}.tar)
fi

cd /content/pkgsrc-${BRANCH}

/bin/ksh mk/bulk/build -s  2>&1 | tee -a ${PDIR}/logs/build-${BRANCH}.log
/bin/ksh mk/bulk/upload -n 2>&1 | tee -a ${PDIR}/logs/upload-${BRANCH}.log

and update-packages:

#!/bin/sh

PATH="/opt/pkg/sbin:/opt/pkg/bin:/sbin:/usr/sbin:/usr/bin:/usr/ccs/bin"
export PATH

BRANCH=2009Q2
export BRANCH

PKGSRCDIR=/content/pkgsrc-${BRANCH}
export PKGSRCDIR

if [ ! -f /opt/pkg/bin/bmake ]; then
    (cd /; tar -xf /install/pkgsrc/bootstrap-${BRANCH}.tar)
fi

# Run this anyway in case there are updates to pkg_chk
env PKG_PATH=/install/pkgsrc/packages/${BRANCH}/All pkg_add -u pkg_chk

# Dry run first
pkg_chk -aurbn

printf "Do you wish to install? [y/N]: "
read ans
case "${ans}" in
[Yy])
    pkg_chk -aurb 2>&1 | tee -a /install/pkgsrc/logs/pkg_chk-${BRANCH}.log
    ;;
esac

These are pretty simple as all the hard work has all been done. build-packages is ran inside the zone, then update-packages on the main host. These scripts hardcode the name of the branch currently used, so you will need to update this when moving to newer releases.

Quick recap

Ok, so here is the stuff I have for my setup and where I keep them:

PathDescription
/content/pkgsrc-2009Q2Checked out pkgsrc tree, "2009Q2" branch
/content/scripts/create-zoneCreates Solaris zone
/content/scripts/delete-zoneUninstalls and deletes zone
/content/scripts/build-packagesBulk build packages inside the zone
/content/scripts/update-packagesUpdates installed packages
/install/pkgsrc/misc/mk.confMain pkgsrc configuration file
/install/pkgsrc/misc/mk-include.confFragment file included in each zone's mk.conf, sources the global mk.conf
/install/pkgsrc/misc/pkgchk.confpkg_chk configuration file
/install/zones/vm-generic.xmlShared SMF configuration file, symlinked to from e.g. "vm0.xml"

And these are the paths where stuff will be created:

PathDescription
/install/pkgsrc/distfilesSource tarballs of packages
/install/pkgsrc/packages/2009Q2Resulting binary packages
/tmp/pkgsrcTemporary build area for packages
/content/vwww/www.adsl.perkin.org.uk/pkgstatBulk build results directory

It’s definitely harder than it should be to get this all setup, but the good news is that once it’s done there’s very little maintenance.

Kicking it all off

Once everything is setup:

$ /content/scripts/create-zone vm0
$ ssh vm0 /content/scripts/build-packages
$ /content/scripts/update-packages
$ /content/scripts/delete-zone vm0

This should do the lot. Once build-packages has finished you should, if you configured your email address in build.conf, get an email with the bulk build results which looks similar to this:

http://mail-index.netbsd.org/pkgsrc-bulk/2009/08/23/msg006883.html

A fuller report is available if you configure a web server to serve the pkgstat directory created by the bulk build, and this can help debug problems (again see the above URL for an example).

You will need to add /opt/pkg/sbin:/opt/pkg/bin to $PATH. Configuration files are in /etc/opt/pkg, and log files and metadata are kept in /var/opt/pkg.

This stuff should be obsolete

While this all works well for me, it’s pretty lame for users who just want to install packages and have stuff work. I’m working on providing regular updates of binary packages, including a SVR4 package of the bootstrap kit, so that in theory all a user needs to do is

$ pkgadd TNFpkgsrc.pkg
$ pkg_add apache22
# Upgrade all installed packages to latest releases
$ pkg_chk -aurb

I’m almost there, just needs some tidying up and regular builds. Please feel free to help out!

Share this post on Twitter, HackerNews, Facebook or Google+

All Posts