Package development

Gluon packages are OpenWrt packages and follow the same rules described at https://openwrt.org/docs/guide-developer/packages.

Development workflow

When you are developing packages, it often happens that you iteratively want to deploy and verify the state your development. There are two ways to verify your changes:

  1. One way is to rebuild the complete firmware, flash it, configure it and verify your development then. This usually takes at least a few minutes to get your changes working so you can test them. Especially if you iterate a lot, this becomes tedious.

  2. Another way is to rebuild only the package you are currently working on and to deploy this package to your test system. Here not even a reboot is required. This makes iterating relatively fast. Your test system could be real hardware or even a qemu in most cases.

Gluon provides scripts to enhance workflow 2). Here is an example illustrating the workflow using these scripts:

# start a local qemu instance
contrib/run_qemu.sh output/images/factory/[...]-x86-64.img

# apply changes to the desired package
vi package/gluon-ebtables/files/etc/init.d/gluon-ebtables

# rebuild and push the package to the qemu instance
contrib/push_pkg.sh package/gluon-ebtables/

# test your changes
...

# do more changes
...

# rebuild and push the package to the qemu instance
contrib/push_pkg.sh package/gluon-ebtables/

# test your changes
...

(and so on...)

# see help of the script for more information
contrib/push_pkg.sh -h
...

Features of push_pkg.sh:

  • Works with compiled and non-compiled packages.

    • This means it can be used in the development of C-code, Lua-Code and mostly any other code.

  • Works with native OpenWrt and Gluon packages.

  • Pushes to remote machines or local qemu instances.

  • Pushes multiple packages in one call if desired.

  • Performs site.conf checks.

Implementation details of push_pkg.sh:

  • First, the script builds an opkg package using the OpenWrt build system.

  • This package is pushed to a target machine using scp:

    • By default the target machine is a locally running x86 qemu started using run_qemu.sh.

    • The target machine can also be remote machine. (See the cli switch -r)

    • Remote machines are not limited to a specific architecture. All architectures supported by gluon can be used as remote machines.

  • Finally opkg is used to install/update the packages in the target machine.

    • While doing this, it will not override /etc/config with package defaults by default. (See the cli switch -P).

    • While doing this, opkg calls the check_site.lua from the package as post_install script to validate the site.conf. This means that the site.conf of the target machine is used for this validation.

Note that:

  • push_pkg.sh does neither build nor push dependencies of the packages automatically. If you want to update dependencies, you must explicitly specify them to be pushed.

  • If you add new packages, you must run make update config GLUON_TARGET=....

  • You can change the gluon target of the target machine via make config GLUON_TARGET=....

  • If you want to update the site.conf of the target machine, use push_pkg.sh package/gluon-site/.

  • Sometimes when things break, you can heal them by compiling a package with its dependencies: cd openwrt; make package/gluon-ebtables/clean; make package/gluon-ebtables/compile; cd ...

  • You can exit qemu by pressing CTRL + a and c afterwards.

Gluon package makefiles

As many packages share the same or a similar structure, Gluon provides a package/gluon.mk that can be included for common definitions. This file replaces OpenWrt’s $(INCLUDE_DIR)/package.mk; it is usually included as include ../gluon.mk from Gluon core packages, or as include $(TOPDIR)/../package/gluon.mk from feeds.

Provided macros

  • GluonBuildI18N (arguments: <source directory>)

    Converts the .po files for all enabled languages from the given source directory to the binary .lmo format and stores them in $(PKG_BUILD_DIR)/i18n.

  • GluonInstallI18N

    Install .lmo files from $(PKG_BUILD_DIR)/i18n to /lib/gluon/web/i18n in the package install directory.

  • GluonSrcDiet (arguments: <source directory>, <destination directory>)

    Copies a directory tree, processing all files in it using LuaSrcDiet. The directory tree should only contain Lua files.

  • GluonCheckSite (arguments: <source file>)

    Intended to be used in a package postinst script. It will use the passed Lua snippet to verify package-specific site configuration.

  • BuildPackageGluon (replaces BuildPackage)

    Extends the Package/<name> definition with common defaults, sets the package install script to the common Gluon/Build/Install, and automatically creates a postinst script using GluonCheckSite if a check_site.lua is found in the package directory.

Default build steps

These defaults greatly reduce the boilerplate in each package, but they can also be confusing because of the many implicit behaviors depending on files in the package directory. If any part of Gluon/Build/Compile or Gluon/Build/Install does not work as intended for a package, the compile and install steps can always be replaced or extended.

Build/Compile is set to Gluon/Build/Compile by default, which will

  • run OpenWrt standard default commands (Build/Compile/Default) if a src/Makefile or src/CMakeLists.txt is found

  • run GluonSrcDiet on all files in the luasrc directory

  • run GluonBuildI18N if a i18n directory is found

Package/<name> defaults to Gluon/Build/Install for packages defined using BuildPackageGluon, which will

  • copy all files from $(PKG_INSTALL_DIR) into the package if $(PKG_INSTALL) is 1

  • copy all files from files into the package

  • copy all Lua files built from luasrc into the package

  • installs $(PKG_BUILD_DIR)/respondd.so to /usr/lib/respondd/$(PKG_NAME).so if src/respondd.c exists

  • installs compiled i18n .lmo files

Feature flags

Feature flags provide a convenient way to define package selections without making it necessary to list each package explicitly. The list of features to enable for a Gluon build is determined by the evaluated image-customization.lua file in the root-directory of the Site repository.

The main feature flag definition file is package/features, but each package feed can provide additional definitions in a file called features at the root of the feed repository.

Each flag $flag will include the package the name gluon-$flag by default. The feature definition file can modify the package selection by adding or removing packages when certain combinations of flags are set.

Feature definitions use Lua syntax. Two basic functions are defined:

  • feature(name, pkgs): Defines a new feature. feature() expects a feature (flag) name and a list of packages to add or remove when the feature is enabled.

    • Defining a feature using feature replaces the default definition of just including gluon-$flag.

    • A package is removed when the package name is prefixed with a - (after the opening quotation mark).

  • when(expr, pkgs): Adds or removes packages when a given logical expression of feature flags is satisfied.

    • expr is a logical expression composed of feature flag names (each prefixed with an underscore before the opening quotation mark), logical operators (and, or, not) and parentheses.

    • Referencing a feature flag in expr has no effect on the default handling of the flag. When no feature() entry for a flag exists, it will still add gluon-$flag by default.

    • pkgs is handled as for feature().

Example:

feature('web-wizard', {
  'gluon-config-mode-hostname',
  'gluon-config-mode-geo-location',
  'gluon-config-mode-contact-info',
  'gluon-config-mode-outdoor',
})

when(_'web-wizard' and (_'mesh-vpn-fastd' or _'mesh-vpn-tunneldigger'), {
  'gluon-config-mode-mesh-vpn',
})

feature('no-radvd', {
  '-gluon-radvd',
})

This will

  • disable the inclusion of the (non-existent) packages gluon-web-wizard and gluon-no-radvd when their corresponding feature flags are evaluated as selected in the image-customization.lua file

  • enable four additional config mode packages when the web-wizard feature is enabled

  • enable gluon-config-mode-mesh-vpn when both web-wizard and one of mesh-vpn-fastd and mesh-vpn-tunneldigger are enabled

  • disable the gluon-radvd package when gluon-no-radvd is enabled