03 - Extras

Objectives

Contents

Lecture

No new lectures today ;) some demos, maybe!

But if you haven't already, check out the first two: 1, 2.

Tasks

01. Automatically booting our Linux

You are probably be tired of using u-boot CLI to load the Linux FIT uImage into RAM and booting it manually. Surely, there must be something to be done to automate this… and you're right! Let's learn to do this!

First, boot your board into u-boot prompt. Enter env print

env print output…

env print output…

arch=arm
auth_os=auth_cntr ${cntr_addr}
baudrate=115200
board=imx93_frdm
board_name=imx93_frdm
boot_a_script=load ${devtype} ${devnum}:${distro_bootpart} ${scriptaddr} ${prefix}${script}; source ${scriptaddr}
boot_efi_binary=load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} efi/boot/bootaa64.efi; if fdt addr -q ${fdt_addr_r}; then bootefi ${kernel_addr_r} ${fdt_addr_r};else bootefi ${kernel_addr_r} ${fdtcontroladdr};fi
boot_efi_bootmgr=if fdt addr -q ${fdt_addr_r}; then bootefi bootmgr ${fdt_addr_r};else bootefi bootmgr;fi
boot_extlinux=sysboot ${devtype} ${devnum}:${distro_bootpart} any ${scriptaddr} ${prefix}${boot_syslinux_conf}
boot_fit=no
boot_net_usb_start=usb start
boot_os=booti ${loadaddr} - ${fdt_addr_r};
boot_prefixes=/ /boot/
boot_script_dhcp=boot.scr.uimg
boot_scripts=boot.scr.uimg boot.scr
boot_syslinux_conf=extlinux/extlinux.conf
boot_targets=mmc0 mmc1 usb0
bootargs=console=ttyLP0,115200 earlycon,115200 rdinit=/linuxrc clk_ignore_unused
bootcmd=run sr_ir_v2_cmd;run distro_bootcmd;run bsp_bootcmd
bootcmd_mfg=run mfgtool_args;if iminfo ${initrd_addr}; then if test ${tee} = yes; then bootm ${tee_addr} ${initrd_addr} ${fdt_addr}; else booti ${loadaddr} ${initrd_addr} ${fdt_addr}; fi; else echo "Run fastboot ..."; fastboot auto; fi;
bootcmd_mmc0=devnum=0; run mmc_boot
bootcmd_mmc1=devnum=1; run mmc_boot
bootcmd_usb0=devnum=0; run usb_boot
bootdelay=2
bootm_size=0x10000000
bootscript=echo Running bootscript from mmc ...; source
bsp_bootcmd=echo Running BSP bootcmd ...; mmc dev ${mmcdev}; if mmc rescan; then if run loadbootscript; then run bootscript; else if test ${sec_boot} = yes; then if run loadcntr; then run mmcboot; else run netboot; fi; else if run loadimage; then run mmcboot; else run netboot; fi; fi; fi; fi;
cntr_addr=0x98000000
cntr_file=os_cntr_signed.bin
console=ttyLP0,115200 earlycon
cpu=armv8
distro_bootcmd=for target in ${boot_targets}; do run bootcmd_${target}; done
efi_dtb_prefixes=/ /dtb/ /dtb/current/
emmc_dev=0
ethprime=eth1
fdt_addr=0x83000000
fdt_addr_r=0x83000000
fdt_high=0xffffffffffffffff
fdtfile=imx93-11x11-frdm.dtb
image=Image
initrd_addr=0x83800000
initrd_high=0xffffffffffffffff
jh_mmcboot=setenv fdtfile ${jh_root_dtb}; setenv jh_clk clk_ignore_unused mem=1248MB kvm-arm.mode=nvhe; if run loadimage; then run mmcboot;else run jh_netboot; fi;
jh_netboot=setenv fdtfile ${jh_root_dtb}; setenv jh_clk clk_ignore_unused mem=1248MB kvm-arm.mode=nvhe; run netboot;
jh_root_dtb=imx93-11x11-frdm-root.dtb
kboot=booti
kernel_addr_r=0x80400000
load_efi_dtb=load ${devtype} ${devnum}:${distro_bootpart} ${fdt_addr_r} ${prefix}${efi_fdtfile}
loadaddr=0x80400000
loadbootscript=fatload mmc ${mmcdev}:${mmcpart} ${loadaddr} ${script};
loadcntr=fatload mmc ${mmcdev}:${mmcpart} ${cntr_addr} ${cntr_file}
loadfdt=fatload mmc ${mmcdev}:${mmcpart} ${fdt_addr_r} ${fdtfile}
loadimage=fatload mmc ${mmcdev}:${mmcpart} ${loadaddr} ${image}
mfgtool_args=setenv bootargs console=${console},${baudrate} rdinit=/linuxrc clk_ignore_unused
mmc_boot=if mmc dev ${devnum}; then devtype=mmc; run scan_dev_for_boot_part; fi
mmcargs=setenv bootargs ${jh_clk} ${mcore_clk} console=${console} root=${mmcroot}
mmcautodetect=yes
mmcboot=echo Booting from mmc ...; run mmcargs; if test ${sec_boot} = yes; then if run auth_os; then run boot_os; else echo ERR: failed to authenticate; fi; else if test ${boot_fit} = yes || test ${boot_fit} = try; then bootm ${loadaddr}; else if run loadfdt; then run boot_os; else echo WARN: Cannot load the DT; fi; fi;fi;
mmcdev=0
mmcpart=1
mmcroot=/dev/mmcblk1p2 rootwait rw
netargs=setenv bootargs ${jh_clk} ${mcore_clk} console=${console} root=/dev/nfs ip=dhcp nfsroot=${serverip}:${nfsroot},v3,tcp
netboot=echo Booting from net ...; run netargs;  if test ${ip_dyn} = yes; then setenv get_cmd dhcp; else setenv get_cmd tftp; fi; if test ${sec_boot} = yes; then ${get_cmd} ${cntr_addr} ${cntr_file}; if run auth_os; then run boot_os; else echo ERR: failed to authenticate; fi; else ${get_cmd} ${loadaddr} ${image}; if test ${boot_fit} = yes || test ${boot_fit} = try; then bootm ${loadaddr}; else if ${get_cmd} ${fdt_addr_r} ${fdtfile}; then run boot_os; else echo WARN: Cannot load the DT; fi; fi;fi;
nodes=/usbg1 /usbg2 /wdt-reboot /rm67199_panel /dsi-host /display-subsystem /soc@0/bus@44000000/dma-controller@44000000 /soc@0/bus@44000000/sai@443b0000 /soc@0/bus@44000000/mqs1 /soc@0/bus@44000000/bbnsm@44440000 /soc@0/bus@44000000/system-controller@44460000 /soc@0/bus@44000000/tmu@44482000 /soc@0/bus@44000000/micfil@44520000 /soc@0/bus@42000000/dma-controller@42000000 /soc@0/bus@44000000/i3c-master@44330000 /soc@0/bus@42000000/i3c-master@42520000 /soc@0/bus@42000000/sai@42650000 /soc@0/bus@42000000/sai@42660000 /soc@0/bus@42000000/mqs2 /soc@0/bus@42000000/xcvr@42680000 /soc@0/bus@42000000/flexio@425c0000 /soc@0/bus@42800000/epxp@4ae20000 /soc@0/bus@42800000/camera /soc@0/efuse@47510000 /soc@0/system-controller@4ac10000 /soc@0/ldb@4ac10020 /soc@0/phy@4ac10024 /soc@0/ele-mu /soc@0/dsi@4ae10000 /soc@0/lcd-controller@4ae30000 /soc@0/blk-ctrl@4e010000 /soc@0/memory-controller@4e300000 /soc@0/bus@44000000/i2c@44350000/pmic@25 /imx93-lpm
prepare_mcore=setenv mcore_clk clk-imx93.mcore_booted;
scan_dev_for_boot=echo Scanning ${devtype} ${devnum}:${distro_bootpart}...; for prefix in ${boot_prefixes}; do run scan_dev_for_extlinux; run scan_dev_for_scripts; done;run scan_dev_for_efi;
scan_dev_for_boot_part=part list ${devtype} ${devnum} -bootable devplist; env exists devplist || setenv devplist 1; for distro_bootpart in ${devplist}; do if fstype ${devtype} ${devnum}:${distro_bootpart} bootfstype; then part uuid ${devtype} ${devnum}:${distro_bootpart} distro_bootpart_uuid ; run scan_dev_for_boot; fi; done; setenv devplist
scan_dev_for_efi=setenv efi_fdtfile ${fdtfile}; for prefix in ${efi_dtb_prefixes}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${efi_fdtfile}; then run load_efi_dtb; fi;done;run boot_efi_bootmgr;if test -e ${devtype} ${devnum}:${distro_bootpart} efi/boot/bootaa64.efi; then echo Found EFI removable media binary efi/boot/bootaa64.efi; run boot_efi_binary; echo EFI LOAD FAILED: continuing...; fi; setenv efi_fdtfile
scan_dev_for_extlinux=if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${boot_syslinux_conf}; then echo Found ${prefix}${boot_syslinux_conf}; run boot_extlinux; echo EXTLINUX FAILED: continuing...; fi
scan_dev_for_scripts=for script in ${boot_scripts}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${script}; then echo Found U-Boot script ${prefix}${script}; run boot_a_script; echo SCRIPT FAILED: continuing...; fi; done
scriptaddr=0x83500000
sd_dev=1
sec_boot=no
soc=imx9
splashimage=0x90000000
sr_ir_v2_cmd=cp.b ${fdtcontroladdr} ${fdt_addr_r} 0x10000;fdt addr ${fdt_addr_r};fdt resize 0x400;fdt set /soc@0/bus@44000000/i2c@44350000/gpio@34 compatible adi,adp5585;for i in ${nodes}; do fdt rm ${i}; done
usb_boot=usb start; if usb dev ${devnum}; then devtype=usb; run scan_dev_for_boot_part; fi
vendor=freescale
 
Environment size: 7010/16380 bytes

Notice various environment variables? There are various addresses, strings and even scripts! Can you figure out the command executed automatically at boot?

We hope you're getting the same idea… what if we can modify this environment when compiling U-Boot such that it executes our own script?

The KConfig item for doing this is called something like DEFAULT_ENV, try searching for it!

It allows us to specify a custom file that contains our default environment…

First, let's create this file, let's say mydefault.env inside the uboot source directory. Let's start from a minimal example:

/* default u-boot environment variables */
/* this file is passed through the C preprocessor (so we can use C-style macros!) */
/* someone in uboot source code re-defined `linux`... */
#undef linux
arch=arm
baseboard=autodetect
baudrate=115200
mmcdev=0
mmcpart=1
emmc_dev=0
console=ttyLP0
bootargs=console=ttyLP0,115200 earlycon,115200 rdinit=/linuxrc clk_ignore_unused
# TODO: this is the command executed automatically when uboot starts...
bootcmd=echo Fastboot mode... press Ctrl-C to exit; fastboot auto
# This runs instead of bootcmd when booted using `uuu` via USB
bootcmd_mfg=run bootcmd
bootdelay=2
image=linux.itb
loadaddr=TODO
loadimage=TODO
linux=echo Booting Linux ...; run loadimage; bootm ${loadaddr};

Notice in the original environment that you may use the run command to run scripts from other defined variables, here's its reference:

run - run commands in an environment variable

We can also use ${varname} expressions to do variable expansion like in Linux shells!

Enter your boot script inside the bootcmd var and let's proceed with overwriting the default environment.

Recall the DEFAULT_ENV menuconfig item? Modify it to point to your mydefault.env (you can simply use a relative path). Note that you need to enable USE_DEFAULT_ENV_FILE checkbox first to let you supply your value!

Afterwards, [re]compile u-boot, copy the u-boot .bin files again to the imx-mkimage directory and regenerate your flash.bin.

Test it by booting your new firmware using uuu. Try using run linux to run the linux script (if you used that), otherwise edit the bootcmd to do that automatically after the timeout! Did it work? if not, you may need to repeat this process (this is where a script comes in handy!).

Note that real devices have their uboot configuration proceed automatically with booting the OS if a physical button is NOT pressed (remember those old Android phones? you could stop this process by holding several volume keys together!). You can script this using GPIO, but it's out of scope for today.

02. Enabling networking on iMX93

Notice that the FRDM-iMX93 has two Ethernet ports. But if you boot your previous Buildroot distro and try to see the available network: ip link show, notice that they are missing. Instead, you'll only have the loopback interface lo.

In this task, you will have to debug the problem and try to fix it or at least find a workaround that lets you use your network interface.

Here are a few suggestions to get you started:

  • Investigate your kernel's Device Tree Blob (dtb) and search for the ethernet node
    • You can use the Device Tree Compiler tool dtc to convert between dtb and dts formats
    • dtc is compiled automatically with Linux and can be found at linux/scripts/dtc/dtc
    • Alternatively, it is packaged as dtc on Arch or device-tree-compiler on Ubuntu
  • Use dmesg to view the kernel's boot log and search for any relevant messages
    • grep -rn parts of any interesting message in the kernel's source to determine the context
    • If you've compiled the kernel already, you can run ./scripts/clang-tools/gen_compile_commands.py and it will generate a compile_commands.json file. This file can be used by your language server to allow you to *go to definition* or highlight code sections included in #ifdefs.
  • Try to identify the driver responsible for our network controller
    • Remember that on ARM, drivers are matched to devices based on their compatible string from the DTB
    • The driver must also have at least one of these strings written in its source files.

Once you are done, enable the iperf3, iproute2 and ethtool network packages in Buildroot and build them. The compilation should not take more than 1-2 minutes. Re-generate linux.itb and copy it to the board's eMMC.

Connect to a colleague's board with an Ethernet cable.
Use the ip command to add a static IP to your network interface (man ip-address).
Then, use the iperf3 tool to test the throughput and compare it to what ethtool advertises.
Why is it not a full 1Gbps?

Hint #1

Hint #1

This line from dmesg tells us why the Network Interface Controller (NIC) remains unavailable.

[   12.004678] platform 42890000.ethernet: deferred probe pending: platform: wait for supplier /soc@0/efuse@47510000/mac-address@4ec

Hint #2

Hint #2

The driver can be found at linux/drivers/net/ethernet/freescale/fec_main.c.
There, the fec_get_mac() function will tell you how the driver gets the NIC's MAC address.
Try to provide it via another means, other than through the eFuse protected by the EdgeLink Enclave (that doesn't work).

Hint #3

Hint #3

In linux/net/ethernet/eth.c look at the comments of the fwnode_get_mac_address() function.
This will give you some alternatives for the nvmem-cells and nvmem-cell-names properties that block the driver initialization.
Try spoofing the MAC address of some vendor.

03. Writing & packaging a GPIO app on Buildroot

An embedded device is designed to interface with the physical world via sensors and actuators.

The most common way to do this is by means of Generic Purpose Input/Output (GPIO): electrical connectors exposed by the SoC that can be freely controlled by software to be either input or output, read/write a logical signal (0-3.3/5V) with whatever they're connected too.

Moreover, the FRDM-iMX93 board has some GPIOs exposed to a Raspbery PI-like 40-pin header and even has some on-board soldered I/O components (e.g., two user-controllable buttons and a RGB LED). Check out its the manual!

Today, we'll learn how to write a simple Linux application to control GPIOs and integrate it into Buildroot's automatic packaging system!

First, check out the Buildroot manual's table of contents.

Read the 9.9. Adding project-specific packages section for an overview of the process involved in adding new custom packages to our Linux distribution.

Basically, we have two options: either create a subdirectory inside the buildroot's internal packages directory or use a BR2_EXTERNAL directory. Although it will be a tad more work to do, we'll choose the second approach as it has one advantage: we are able to include / exclude the directory at will (useful if you don't intend to finish the task, as it would leave your buildroot source broken!) and being able to share your work with others (e.g., using your own Git repository), i.m.h.o. separation of concerns is always a nice thing to do!

So we'll proceed with the tutorial on Using BR2_EXTERNAL to keep customizations outside of the root.

The External Directory

First, create a directory, let's say, gpio-external somewhere in your home. In there, you must create three files inside this directory:

1.external.desc, containing a name and a description:

external.desc
name: ASS_EXTRA
desc: ARM Summer School Extras

The name is the most important, must be uppercase [A-Z0-9_], as it will be used to prefix your custom packages!

2. Config.in, that will be used to load KConfig menus from your packages:

Config.in
# BR2_EXTERNAL_<NAME>_PATH is automatically defined for your external dir!
# we will create our package later, let's name it `myleds`:
source "$BR2_EXTERNAL_ASS_EXTRA_PATH/package/myleds/Config.in"

3. … and external.mk, also designed to include packages' build scripts:

external.mk
include $(sort $(wildcard $(BR2_EXTERNAL_ASS_EXTRA_PATH)/package/*/*.mk))

The Package Directory

As you probably figured it out, you must also create the package/myleds (or whatever name you want for your Buildroot package be called) inside your external root, containing two important files:

  1. Config.in (uppercase C!) – KConfig menu entries;
  2. myleds.mk: A makefile included by Buildroot, guess why: to compile your program!

Let's start with the package configuration. Recall that, by invoking menuconfig make target, you are presented with lots of BR2_PACKAGE_* offerings. All packages append to this menu by the means of a KConfig file using a custom definition language. For buildroot, it must be called Config.in and must be located inside each package's and external dir directory (you saw it included there!). Let's make one for our package:

package/myleds/Config.in
config BR2_PACKAGE_MYLEDS
        bool "myleds"
        # we can have some dependencies:
        depends on BR2_PACKAGE_LIBGPIOD2
        help
          ARM Summer School LEDS App.

We have given MYLEDS name for our package (the directory name must also be a lower-cased version of this). This is important, as we will need to use it as prefix for all of our Makefile variables defined in our *.mk:

package/myleds/myleds.mk
################################################################################
# ASS Leds Package
################################################################################
 
MYLEDS_VERSION = 1.0
MYLEDS_SITE = $(BR2_EXTERNAL_ASS_EXTRA_PATH)/package/myleds/src
MYLEDS_SITE_METHOD = local
MYLEDS_LICENSE = GPL-3.0+
MYLEDS_INSTALL_STAGING = NO
MYLEDS_INSTALL_TARGET = YES
MYLEDS_DEPENDENCIES = libgpiod2
 
define MYLEDS_BUILD_CMDS
	$(MAKE) $(TARGET_CONFIGURE_OPTS) -C $(@D) all
endef
 
define MYLEDS_INSTALL_TARGET_CMDS
    $(INSTALL) -D -m 0755 $(@D)/myleds $(TARGET_DIR)/usr/bin
endef
 
define MYLEDS_PERMISSIONS
    /usr/bin/myleds f 0755 root root - - - - -
endef
 
$(eval $(generic-package))

We've started from the Generic Package tutorial with our makefile.

Notice that there are ways to specify where to get the source code (it can be downloaded, but we will write our code locally), how to compile the package (by invoking an inner Makefile inside our source directory) and how to install it (we are currently copying the myleds executable to the target rootfs's /usr/bin path, though feel free to custimize it further!) plus permissions (we'll giving anyone the right to execute it!).

The $(generic-package) is a Buildroot macro that will take all of our package name-prefixed variables and generate the final makefile build rules.

For now, create a dummy source directory at package/myleds/src with a main.cpp (or .c if you wish to stay oldschool) file simply printing something:

#include <cstdlib>
#include <iostream>
 
int main()
{
	::std::cout << "Hello from MyLeds!"
		    << ::std::endl;
 
	return 0;
}

And its compilation script:

SRCS = main.cpp
# all is the first target, so it will be invoked when calling `make` simply
all: myleds
# myleds is the resulting executable name using `-o $(@)`
# depend on source files (so it will be recompiled everytime the .cpp changes)
myleds: $(SRCS)
	$(CXX) $(CXXFLAGS) $(SRCS) $(LDFLAGS) -o $(@)
# good practice to have a `clean` rule:
clean:
	rm -f myleds

We're using the recommended make variables: $(CXX) specifying the path to the compiler (it will actually be set to ${CROSS_COMPILE}g++ by Buildroot)!

Same for $(CXXFLAGS) and $(LDFLAGS), they contain compiler arguments supplied by the make caller (we will set them later, we'll leave them empty for now).

Those are actually defined as a convention by GNU Make's manual and used throughout the Open Source ecosystem, so just behave and use them too ;))

Let's test the build process!

We can now do a preliminary test if it builds by first enabling our external directory using:

# activate our external dir (do this inside buildroot's source dir!)
make BR2_EXTERNAL="../gpio-external"

This is a special configuration option automatically saved in the hidden .br2-external.mk file in the output directory!

If you ever wish to disable it, simply invoke a make clearing this variable, e.g.:

make BR2_EXTERNAL=

Now enter menuconfig and search for your package: MYLEDS or however you called it. It should exist! Enable it (together with its declared libgpiod2 dependency! recall: all KConfig names are UPPERCASE!).

After enabling it, invoke make again and check the logs: you should see your makefile being executed!

Find your executable inside the buildroot's output directory:

find output/ -name 'myleds'

Was is correctly installed? Can you execute it (hah, gotcha!)?

Accessing GPIOs from Linux

We need to use a library for this, modern distributions use libgpiod2 which you saw we added as dependency earlier!

Time to finish our code: check out a C++ gpiod2 example here. This sets a GPIO for input, we want it to output. Change the code accordingly. Make a specific GPIO blink every second.

Find the RGB LED's GPIO index inside the board's manual. Either choose a single color or write code to cycle through them all. One more thing: there are multiple gpiochips defined by the board's device tree. The one useable to control the LEDs and RPI GPIO header is /dev/gpiochip0 (the first one, as expected).

Also check more LibGPIOd2 examples on its official repository!

When finally finished, try to build it. It may give you some errors (libgpiod functions/classes not found). This is because you're missing the appropiate LDFLAGS, so be sure to include them and re-try.

Good luck ;)