OpenWRT and opkg

I own several Linksys WRT54G wireless access points. They're about a decade past their prime, but they're well constructed and reliable, and I find them very interesting. Most famously, they are the router for which OpenWRT was created, and I have OpenWRT installed on all of them.

Most WAPs have limited memory, and many of the WRT54G models have only 4MB of Flash and 16MB of RAM. This is barely enough to fit the base OS, and adding more packages is a bit dicey. Nevertheless, OpenWRT has a package management system with a package manager called opkg.

Basic opkg usage: opkg list-installed tells you what packages are installed. opkg update must be run any time you want to install something: opkg loses its list of available packages over a reboot. opkg install python installs a package and its dependencies - in this case, it would probably break most WRT54Gs, because they don't have enough room for python and its deps.

I've ended up writing several scripts to assist with package management (I've done similar stuff for dpkg/apt-get/aptitude on Debian). The first is just an alias, think of it as opkg grep:

alias opkgg="opkg list-installed | grep -i "

Typing "opkgg ash" will tell you what version of ash is installed.

I found that I often wanted to know how much space a particular package was using on the system. opkg can only tell you the size of the compressed package it installed, not the actual disk usage now, so I wrote opkgsize:

#!/bin/ash

# 2013-07-07 by Giles
# Given an installed opkg package name, use the file list opkg keeps
# to figure out how much space is used.

if [ $# -ne 1 ]
then
    echo "$(basename ${0}) <packagename>"
    # Add more detail
    exit 1
fi

package="${1}"
packDir="/usr/lib/opkg/info/"
ovPackDir="/overlay/${packDir}"
fileList="${packDir}/${package}.list"

if ! [ -f "${fileList}" ]
then
    echo "${package} doesn't appear to be installed."
    exit 2
fi

writeStatus="(RO)"
if [ -f "${ovPackDir}/${package}.list" ]
then
    writeStatus="(RW)"
fi

byteSum=0
for file in $(cat "${packDir}/${package}.list")
do
    # Check that it's a regular file ... as soft links are seen as regular
    # files, they can be excluded with -h, but I'm including them as they
    # add to the "weight" of the package
    if [ -f "${file}" ] # && ! [ -h "${file}" ]
    then
        byteSum=$((${byteSum} + $(ls -l "${file}" | awk '{ print $5 }')))
    fi
done

echo "${package} ${writeStatus}: $((${byteSum}/1024)) kb"

But something I often want to know is the sizes of all the packages I've installed. This applies to packages the user installed that aren't part of the original OS install. This requires opkgsize above, and is called opkgsizes:

#!/bin/ash
#
# 2013-07-07 by Giles
# Determine the sizes of all installed packages in /overlay/
# Requires: opkgsize (also by me)

packDir="/usr/lib/opkg/info/"
ovPackDir="/overlay/${packDir}"

for file in $(ls -1 ${ovPackDir}/*.list)
do
    opkgsize "$(basename "${file%.list}")"ยทยท
done

Similarly, before I install a package I'd like to know how likely it is to overrun the available storage. As mentioned, opkg only knows about the compressed size of the package its about to download, so this doesn't provide a complete answer (I call this opkgdeps):

#!/bin/ash

# Filename: opkgdeps
# Author:   Giles Orr
# Created:  2014-12-25
# Modified: 2015-01-04
# Purpose:  Given an opkg package name, determine its dependent packages
#           and their (compressed) sizes.

# "python" is a good test package with deps "libpthread, zlib,
# libffi, python-mini", and libpthread has dep "librt".

if [ $# -ne 1 ]
then
    # HELP:
    echo "$(basename ${0}) <packagename>"
    echo "  Show the dependencies of <packagename> and their sizes"
    exit 1
fi

# Fair warning:
echo "This script is VERY SLOW on a WRT54G*, expect 5-40 seconds."

package="${1}"

getdeps () {
    # grab the "Depends:" line of "opkg info", remove the leading word,
    # and delete commas giving a space-separated list:
    echo "$(opkg info ${1} | grep "^Depends:" | cut -d " " -f 2- | tr -d ",")"
}

deps=""
newdeps="$(getdeps "${package}")"

while [ "${deps}" != "${newdeps}" ]
do
    # Given a list of packages, go through them and determine
    # their dependencies.  Add those to the existing list of
    # dependencies.  Rinse and repeat until the list stops
    # changing.
    deps="${newdeps}"
    deplist="${newdeps}"
    for dep in ${newdeps}
    do
        add="$(getdeps "${dep}")"
        deplist=$(echo ${deplist} ${add})
    done
    # Turn it into a newline separated list, sort, uniq, back to
    # space-separated list:
    newdeps=$(echo ${deplist} | tr " " "\n" | sort | uniq | tr "\n" " ")
done

if [ "${newdeps}x" == "x" ]
then
    # We have no dependencies:
    echo "${package} has no dependencies."
    size=$(opkg info ${package} | grep "^Size:" | cut -d " " -f 2)
    echo "${package} compressed size: $((${size}/1024)) kb"
else
    someAlreadyInstalled=1 # false
    # We have at least one dependency:
    echo "The dependencies for ${package} are:"
    echo -n "    "
    for dep in ${newdeps} ${package}
    do
        if [ $(opkg list-installed | grep -q "^${dep} " ; echo $? ) -eq 0 ]
        then
            # package is already installed:
            echo -n "${dep}* "
            someAlreadyInstalled=0 # true
        else
            echo -n "${dep} "
        fi
    done
    echo
    if [ ${someAlreadyInstalled} -eq 0 ]
    then
        echo "    * already installed"
    fi

    # Get the compressed sizes for all packages and sum them up:
    byteSumAll=0
    byteSumNeeded=0
    for dep in ${newdeps} ${package}
    do
        size=$(opkg info ${dep} | grep "^Size:" | cut -d " " -f 2)
        # Check if Size is defined ("libc" doesn't list a size! 2015-01-04):
        if [ "${size}x" == "x" ]
        then
            echo "'${dep}' reports no size, ignoring."
        else
            byteSumAll=$((${byteSumAll} + ${size}))
            if [ $(opkg list-installed | grep -q "^${dep} " ; echo $? ) -eq 1 ]
            then
                # package is not installed:
                byteSumNeeded=$((${byteSumNeeded} + ${size}))
            fi
        fi
    done
    echo "${package} compressed size with deps: $((${byteSumAll}/1024)) kb"
    if [ ${someAlreadyInstalled} -eq 0 ]
    then
        echo "-> ignoring already installed packages: $((${byteSumNeeded}/1024)) kb"
    fi
fi

As noted in the script itself, this is extremely slow as it's running on a slow processor with no memory and retrieving a lot of information.

Finally, for those times you install something and opkg is too dumb to know it's going to run out of space and bork the router ... I can't claim to have written this one, but I'm very, very happy to have it available (called opkgclean):

#!/bin/sh
# 2014-12-26, from: http://wiki.openwrt.org/doc/techref/opkg#out_of_space
#takes one argument/parameter: the name of the package which didn't install
# correctly and should be removed along with its dependencies
#do opkg update first
#example: ./opkgremovepartlyinstalledpackage.sh pulseaudio-daemon

if [ $# -ne 1 ]
then
    # HELP:
    echo "$(basename ${0}) <packagename>"
    echo "  Clean up after install of <packagename> failed on an"
    echo "  out-of-space error.  (Tested at least once.)"
    exit 1
fi

#get list of all packages that would be installed along with package x
opkg update
PACKAGES=`opkg --force-space --noaction install $1 | grep "http:" | cut -f 2 -d ' ' | sed 's/.$//'`
for i in $PACKAGES
do
    LIST=`wget -qO- $i | tar -Oxz ./data.tar.gz | tar -tz | sort -r | sed 's/^./\/overlay/'`
    for f in $LIST
    do
        if [ -f $f ]
        then
            echo "Removing file $f"
            rm -f $f
        fi
        if [ -d $f ]
        then
            echo "Try to remove directory $f (will only work on empty directories)"
            rmdir $f
        fi
    done
done
echo "You may need to reboot for the free space to become visible"