portupgrade

In the previous article, we took a look at the built-in utilities that can be used to manage the FreeBSD ports collection. In this article, I'd like to continue in that vein. Let's take a look at portupgrade, a feature-rich port designed to help you get the most out of the ports collection.

Installing portupgrade

Like any other port, you install portupgrade by cding into its directory and issuing a make command:

% cd /usr/ports/sysutils/portupgrade
% make install clean

The build will install over a dozen useful utilities. Let's take a trick from the last article and see which manpages, and hence, which utilities, were installed:

% pkg_info -xL portupgrade | grep man
/usr/local/man/man1/pkg_deinstall.1.gz
/usr/local/man/man1/pkg_fetch.1.gz
/usr/local/man/man1/pkg_glob.1.gz
/usr/local/man/man1/pkg_sort.1.gz
/usr/local/man/man1/pkgdb.1.gz
/usr/local/man/man1/portcvsweb.1.gz
/usr/local/man/man1/portsclean.1.gz
/usr/local/man/man1/portsdb.1.gz
/usr/local/man/man1/portupgrade.1.gz
/usr/local/man/man1/portversion.1.gz
/usr/local/man/man5/pkgtools.conf.5.gz
/usr/local/man//man1/pkg_which.1.gz
/usr/local/man//man1/portinstall.1.gz
/usr/local/man//man1/ports_glob.1.gz

It is time well spent to skim through those manpages. You'll get an idea of the power and the flexibility of the ports collection and will uncover tips and tricks you've never thought of. However, don't be dismayed if you feel a bit overwhelmed by the amount of available information. In the next few articles, I'll walk you through some concrete examples to get you started in using these utilities effectively.

First, let's review the ports structure, where it's installed on your system, and how you can keep it up to date.

When you installed FreeBSD and chose to install the ports collection, /usr/ports and its files and subdirectories were created for you. If you ls /usr/ports, you'll see that it contains subdirectories that logically divide the ports collection. For example, there are subdirectories for mail, www, and databases. Each subdirectory contains subdirectories for applicable applications, so www has subdirectories for mozilla and lynx. Each of those subdirectories contains the information needed in order to install that particular application. For example:

$ ls -F /usr/ports/www/mozilla
./	Makefile	distinfo	pkg-descr	pkg-message
../	README.html	files/		pkg-descr.gtk2

We were actually gathering information from the Makefile, pkg-descr, and pkg-message files with some of the utilities and switches we covered in the last article.

The ports collection is constantly being updated. New ports are added regularly, usually on a daily basis. If you're the curious type and like to see a layout of which ports were added when, you'll find FreshPorts an invaluable resource.

It's great to have such a dynamic ports collection, but it does mean that your ports tree, the /usr/ports directory structure, can quickly become out of date. To keep in sync with the changes and ensure that you always have the ability to build any available port, use cvsup.

Syncing Your Ports Tree

I've mentioned cvsup often. If you're not using cvsup yet, you really do want to read through Using CVSup before giving it a go. The handbook is definitely the best place to get the proper understanding of the entire cvsup process. Once you're ready:

% cd /usr/ports/net/cvsup-without-gui
% make install clean

cvsup can be used to keep both your operating system and your ports collection up to date. If you're only interested in keeping your ports tree in sync, a file similar to this will do it:

% more /root/cvs-supfile
*default host=cvsup.ca.freebsd.org
*default base=/usr/local/etc/cvsup
*default prefix=/usr
*default release=cvs delete use-rel-suffix compress
ports-all tag=.

The file can be invoked with this command as the superuser:

% cvsup -g -L 2 ~/cvs-supfile

If that file or that command doesn't make sense to you, read the section of the handbook I referred to above. Also note that the =. in the ports-all line is important, so double-check that your file has it.

That cvsup command will download all of the latest bits of the ports collection and add them to your ports tree. This is the type of command that benefits from being run on a daily basis, so you might wish to add it as a cron job.

The cvsup process also updates a file called /usr/ports/INDEX. This file contains a list of all of the ports in your ports tree. To see how up to date your ports tree is, use this command:

$ ls -l /usr/ports/INDEX
-rw-r--r--  1 root  wheel  3678738 May 17 17:04 INDEX

That particular machine was installed on May 17 and I haven't kept its ports tree up to date. If I compare that system to my main machine, which is cvsuped daily:

$ ls -l /usr/ports/INDEX
-rw-r--r--  1 root  wheel  3912366 Aug 17 08:50 INDEX

you'll see that the size of the file, and thus the number of port entries, has grown considerably in that three-month period.

Updating the Ports Database

Okay, we're finally ready for portupgrade and its suite of utilities. After each cvsup, run this command:

% portsdb -Uu

The first time you use this command, a database called INDEX.db will be created in /usr/ports. This database will be updated every time you repeat that portsdb command after a fresh cvsup. If you use the file utility, you'll see that you won't be able to access the contents of INDEX.db, since it is not an ASCII text file:

$ file /usr/ports/INDEX.db
/usr/ports/INDEX.db: Berkeley DB 1.85/1.86 (Btree, version 3, native byte-order)

However, several of the portupgrade utilities will use this database. The Btree refers to a type of database algorithm that is designed to quickly search through a large amount of data. This is ideal for the ports collection — we'll find that some of the portupgrade utilities are faster and more efficient than the built-in utilities we saw in the last article, because of that Btree.

You'll have to be patient, as portsdb takes a while to run. Once it is finished, you're ready to use portversion to see if any of your installed ports need upgrading. Remember this command?

% portversion -l "<"

If you receive any output, your next step is to upgrade those out-of-date ports. Not surprisingly, we'll use portupgrade, which is also called portinstall.

Upgrading the Ports

In its simplest form, portupgrade -a will upgrade all (-a) of your out-of-date ports. However, over time you may end up with ports that refuse to upgrade. This seeming anomaly is not a limitation of portupgrade, but rather points to the reality of dependencies.

A port has two types of dependencies. The first is called a build dependency and refers to any other ports that need to be installed before this port will successfully install. The second type of dependency refers to other ports that depend upon the particular port you wish to upgrade. You may remember from the last article that pkg_delete wouldn't let you uninstall an application if other applications depended upon it.

If you just upgrade an application but don't check to see if its dependencies also need upgrading, you'll eventually end up with applications that refuse to upgrade. To prevent this from happening, use the two recursive switches with portupgrade, like so:

% portupgrade -arR

The -R will check the build dependencies and the -r will check the applications that depend upon the port being upgraded. This will prevent your system from having outdated dependencies and software incompatibilities.

Occasionally, when you use portupgrade or one of its utilities, you'll see a message asking you to run pkgdb -F. As you may have guessed from the name, this utility updates a package database. That database is found at /var/db/pkg/pkgdb.db. Again, this database uses a Btree to optimize search time.

If you're ever asked to run pkgdb -F, do it. However, don't interrupt this command, or you'll end up with an inconsistent database. If you're ever in that unfortunate situation, this command will fix the inconsistencies:

% pkgdb -fu

That's a pretty easy switch combo to remember, as similar thoughts will probably be running through your head at the time.

Running pkgdb with -F will interactively fix the database. This means that pkgdb will pause to ask you what you would like it to do before doing it. Michael Lucas wrote an article explaining how to respond. However, if you find it intimidating coming up with the correct responses to pkgdb's enquiries, use -fu instead, and it will quietly do what it thinks is best. If you're really paranoid, an alternative is -Fa, which tells pkgdb to only fix the discrepancies that can be fixed securely.

This all sounds scarier than it really is. It's very rare that you'll ever be asked to run pkgdb -F. pkgdb is usually used for other purposes, which is why it's also called pkg_which.

Using pkg_which

pkg_which (or pkgdb) can be used to find out to which application a file belongs. Here's a simple example showing the difference between the built-in which command and pkg_which:

$ which pkgdb
/usr/local/sbin/pkgdb

which is used to find the path to an application. pkg_which will tell me to which port the specified application belongs:

$ pkg_which pkgdb
portupgrade-20030723

This command is equivalent to that last pkg_which command:

$ pkgdb pkgdb
portupgrade-20030723

Here's another example. Let's say you're poking about /usr/local, the directory structure containing the files used by installed applications. You find a whole bunch of files and don't have a clue where they came from or to which application they belong. Here's a job for pkg_which. Take a look at this example snippet from my system:

$ ls /usr/local/bin | grep yaf
tryaffix*
yaf-cdda*
yaf-mpgplay*
yaf-splay*
yaf-tplay*
yaf-vorbis*
yaf-yuv*

If you're like me, those files hold absolutely no meaning to you. Let's see to which applications they belong:

$ pkg_which *yaf*
ispell-3.2.06_3
kdemultimedia-3.1.3
kdemultimedia-3.1.3
kdemultimedia-3.1.3
kdemultimedia-3.1.3
kdemultimedia-3.1.3
kdemultimedia-3.1.3

pkg_which contains a few useful switches. One is the -o, or origin, switch. Say you can't remember where in the ports tree kdemultimedia-3.1.3 came from. Try this:

$ pkg_which -o kdemultimedia-3.1.3
multimedia/kdemultimedia3

The output indicates that the name of the subdirectory from which that application was built is /usr/ports/multimedia/kdemultimedia3.

The -v switch can be useful, as well. If I wish to know which kde applications I currently have installed:

$ pkg_which -v kde*
kde-config: kdelibs-3.1.3
kdebugdialog: kdebase-3.1.3
kded: kdelibs-3.1.3
kdeeject: kdebase-3.1.3
kdeinit: kdelibs-3.1.3
kdeinit_shutdown: kdelibs-3.1.3
kdeinit_wrapper: kdelibs-3.1.3
kdepasswd: kdeutils-3.1.3
kdeprintfax: kdebase-3.1.3
kdesktop: kdebase-3.1.3
kdesktop_lock: kdebase-3.1.3
kdessh: kdeutils-3.1.3
kdesu: kdebase-3.1.3
kdesu_stub: kdelibs-3.1.3
kdesud: kdebase-3.1.3
kdevdlg2ui: kdevelop-2.1.5
kdevelop: kdevelop-2.1.5

That output is pretty informative. Each line shows the name of an application, followed by a :, followed by the name of the port that installed that application. By the way, it looks like this machine is running KDE version 3.1.3.

Before going further, let's summarize the steps you can take to keep your installed software up to date:

1. Run cvsup to sync the ports tree.
2. Run portsdb to update INDEX.db.
3. Use portversion to determine which applications need upgrading.
4. Use portupgrade to upgrade those applications.

Additional portupgrade Switches

We've already seen that portupgrade -arR will properly upgrade all out-of-date applications. However, portupgrade comes with several switches to let you pick and choose which applications to upgrade and how to do it.

One option that is useful if you don't have a constant Internet connection is -F. Typically, when you upgrade, portupgrade will go out on the Internet as it needs a file, then spend the time building that file. If you're doing a large upgrade such as KDE, it may need to go on the Internet every so often over a period of hours.

This command will go out on the Internet and retrieve all of the files needed to upgrade your ports, but won't install anything:

$ portupgrade -aFrR

Once you've downloaded the necessary files, you can disconnect from the Internet and use portupgrade -arR as usual.

Another available switch is -n. This switch simply tells you what portupgrade would do without actually doing it. This is very useful if you are the nervous or the paranoid type and want to know ahead of time what is going to happen to your installed software.

Here's an example output:

$ portupgrade -anrR
--->  Session started at: Sun, 17 Aug 2003 22:06:00 -0400
<a page of output snipped>
--->  Reporting the results (+:done / -:ignored / *:skipped / !:failed)
- lang/ruby16 (ruby-1.6.8.2003.04.19)
- net/cvsup-without-gui (cvsup-without-gui-16.1h)
+ lang/ruby16-shim-ruby18 (ruby-shim-ruby18-1.8.0.p2.2003.04.19)
+ databases/ruby-bdb1 (ruby-bdb1-0.1.9)
- sysutils/portupgrade (portupgrade-20030723)
- www/lynx (lynx-2.8.4.1d)
--->  Session ended at: Sun, 17 Aug 2003 22:06:02 -0400 (consumed 00:00:02)

Let's take a look at that output. portupgrade systematically went through every installed port on the system, then put its results into a report. Each line in the report shows the port's origin, the currently installed version, and a symbol indicating whether or not it needs to be upgraded. This particular report shows that two ports need upgrading. They are the ones on line that begin with a +. If I compare this report to portversion, I'll see similar results, written in a different fashion:

$ portversion -l "<"
ruby-bdb1
ruby-shim-ruby18

If you're cautious about what gets upgraded on your system, you might also enjoy the -i, or interactive, switch. Add it to your portupgrade switches and portupgrade will pause for your input before upgrading an application or any of its dependencies. A pause will look like this:

--->  Upgrading 'ruby-bdb1-0.1.9' to 'ruby-bdb1-0.2.1'
	(databases/ruby/bdb1)
OK? [yes]

Notice the default response of yes in the square brackets. That means if you press enter, you are saying yes. If you decide you'd rather not upgrade this port, type in the word no instead.

The last switch I'd like to cover is the l, or log, switch. This switch is invaluable if you're ever in the situation where a port refuses to install and you need to send the output of the failed install to someone else. Here, I'll upgrade one specified port and send the output to a file called logfile:

$ portupgrade -rR ruby-shim-ruby18 -l logfile

You probably don't want to use the l switch with the a switch, especially if you have a lot of ports that need upgrading. No one is going to want to wade through a log file that large!

In the next article, I want to discuss uninstalling ports and cleaning out files that are no longer in use by any ports. We should also have time to take a look at customizing pkgtools.conf.

Dru Lavigne is a network and systems administrator, IT instructor, author and international speaker. She has over a decade of experience administering and teaching Netware, Microsoft, Cisco, Checkpoint, SCO, Solaris, Linux, and BSD systems. A prolific author, she pens the popular FreeBSD Basics column for O'Reilly and is author of BSD Hacks and The Best of FreeBSD Basics.

Read more FreeBSD Basics columns.

Return to the BSD DevCenter.

Copyright © 2007 O'Reilly Media, Inc.