Using configfs for dynamic device tree overlays with BalenaOS

I spend most of my time working on BeagleBoard.org community projects, but I do a few projects of my own building IoT devices. There are a number of good software infrastructure tools out there, but I often use Balena. The problem I have faced often is not specific at all to Balena, but I’d like to share with you my Balena solution in case it might be helpful to you.

There is no fully “standardized” way to provide device-tree overlay fragments to Arm board bootloaders in general or for Beagles in specific. The config.txt file was picked up for Raspberry Pi boards by Balena, but they have yet to adopt the extlinux.conf we rely on for the BeagleBoard.org Debian images. While it is possible to customize BalenaOS myself, I find that to be an incredible pain.

In this particular case, all I wanted to do was to disable the loading of the serdev driver attached to the CC1352 so that I could flash it with different firmware and use it with other protocols. While the upstream driver provides a firmware update mechanism, I am using my own custom CC1352 firmware and need to remove the driver to get raw UART access. I suspect this is a fairly common use-case.

Balena has an example for building an out-of-tree kernel module that makes it fairly easy to build a kernel module. This was my starting point.

For the kernel module itself, Pantelis Antoniou proposed a patchset all the way back in 2014 that still works with the mainline dt changeset. It is a shame we haven’t yet included this with the mainline kernel, perhaps in fear of abuse, but given the nature of Arm and other architectures that depend on device tree for expressing hardware that might be dynamic, it feels like critical infrastructure for system developers. I grabbed a version off of the Xilinx Github Linux tree and added the necessary MODULE_LICENSE to enable it to compile out-of-tree as a module.

I built both the kernel module and device tree binaries in separate build phases. The kernel module example was tied to a specific version of Ubuntu and I didn’t want to mess with that. There was nothing special about the version of Ubuntu I grabbed to run the device tree compiler, except that `jammy` is a bit old for my taste. For the example of applying the overlay, I chose a Debian image with the Beagle package feed enabled. All this breakdown is likely entirely unnecessary.

Applying the device tree overlay via configfs is done in start.sh for this example:

#!/bin/bash

# Load the configfs kernel module for dynamic device tree overlays
insmod /opt/lib/modules/of_configfs.ko

# Apply the device tree overlay
mount -t configfs none /sys/kernel/config
mkdir -p /sys/kernel/config/device-tree/overlays/no-cc1352-driver
cat /opt/lib/dtb/k3-am625-beagleplay-bcfserial-no-firmware.dtbo > /sys/kernel/config/device-tree/overlays/no-cc1352-driver/dtbo
cat /sys/kernel/config/device-tree/overlays/no-cc1352-driver/status # applied

# Verify the overlay applied and reload the platform serial driver
cat /proc/device-tree/bus@f0000/serial@2860000/mcu/status # disabled
echo 2860000.serial > /sys/devices/platform/bus@f0000/2860000.serial/driver/unbind
echo 2860000.serial > /sys/devices/platform/bus@f0000/2800000.serial/driver/bind

sleep infinity

Here I am not directly testing that the device tree overlay successfully applied or that the driver was disabled as a result of the device tree overlay being applied, but just showing you the cat operations that would indicate success. To complete the operations, the UART driver bind operation was performed. Note that /dev/ttyS1 may need to be generated manually with mknod and I skipped that operation here.

I have posted this example on Github. This exact version is untested as this was previously integrated into a project of mine, so please post any issues you find.