If you wish to run the build locally, make sure you have a pretty powerful x64 laptop/PC (8-16 powerful cores, 32GB RAM, 50GB SSD free space).

Otherwise, we offer some rather powerful VMs hosted inside the Faculty’s private cloud (OpenStack) for use with this workshop.

It’s going to be a headless (no graphics) server-like environment, so Linux command line knowledge is required. Don’t worry, you have all the tools you need in there (plus a tweaked Zsh, TMux for running persistent/multiplexed terminals and vim with AstroNvim config for enthusiasts).

But first, you will need to gain access to this VM. To do this, you must create a SSH private/public key pair and give the public counterpart to the superviser for account creation & authorization (do this on your personal PC / laptop in a Linux sh-compatible shell or Windows Powershell), generate a public/private SSH keypair:

# first, check if a keypair already exists
you@laptop:~# ls ~/.ssh/
# if the files `id_ed25519` and `id_ed25519.pub` already exist, SKIP everything else!
you@laptop:~# ssh-keygen -t ed25519  # Edwards curves: better & shorter keys!
# answer the defaults (aka just press enter multiple times)
you@laptop:~# ls -l ~/.ssh/
-rw------- 1 root root 2602 Oct 15 12:59 id_ed25519     # <-- THE PRIVATE KEY, keep secret!
-rw-r--r-- 1 root root  571 Oct 15 12:59 id_ed25519.pub # <-- your PUBLIC key to give away!
# print and copy the private key (including the 'ssh-ed25519' prefix!):
you@laptop:~# cat ~/.ssh/id_ed25519.pub
ssh-ed25519 AAAAB3NzaC...  # a longish line containing the public key number + email

Now copy your public key and give it to a supervisor (together with a preferred username to be created) to gain access to the build machine, but wait for confirmation…

When notified, try to connect to your assigned VM and port, e.g.:

ssh youruser@arm2025.root.sx -p 220X  # where X is your assigned VM index

export CROSS_COMPILE = /path/to/your/toolchain/gcc-prefix...-
 
ATF_DIR = imx-atf
ATF_MAKE_FLAGS = SPD=none PLAT=TODO...
atf:
	make -C "$(ATF_DIR)" $(ATF_MAKE_FLAGS)
 
UBOOT_DIR = uboot
UBOOT_MAKE_FLAGS =
uboot:
	make -C "$(UBOOT_DIR)" $(UBOOT_MAKE_FLAGS)
 
.PHONY: uboot atf
# ... and so on !

Well, we want to compile things. We'll be using either a PC / laptop or a VM, which is probably running on a Intel/AMD-based x86-64 CPU architecture.

But we need to compile code for a foreign architecture: Aarch64 (also known as arm64). So we will need a cross compiler: a compiler that runs on your host but generates binary code for another target.

Cross compilers are usually named using this convention <target_arch>-<vendor>-[<os>-]<abi>-gcc. In our case, it will be aarch64-none-linux-gnu-gcc:

• aarch64: The target architecture; ARM 64-bit (ARM Arch 64-bit… bad acronym, we know, but it's the official name…);
• none: The toolchain supplier. Here, we're dealing with the official cross compiler from ARM so the vendor is not specified;
• linux: The target OS; if this field is none or missing, it means the target is bare-metal (used for building microcontroller firmware, which is not the case today);
• gnu: The Application Binary Interface (ABI) defining how the generated code will interact with other software components.

Identify the correct toolchain (be careful with your host architecture: probably Linux x86_64) and download it from here. Yes, you can also install it with your package manger but it's always better to have a project-specific copy. New compiler releases are known to sometime break the build process.

After extracting it (somewhere, let's say… make a new toolchain subdirectory in your home), consider exporting the compiler's prefix (aarch64-none-linux-gnu-) from your newly downloaded toolchain as the CROSS_COMPILE environment variable. Otherwise, you will have to specify the full path of the cross compiler every time when invoking a script. E.g.:

export CROSS_COMPILE=/absolute/path/to/compiler/bin/aarch64-none-linux-gnu-
# now you can use the cross compiler as such:
"${CROSS_COMPILE}gcc" ...
# make will inherit this value, so no need to add a CROSS_COMPILE 
# when invoking it from your terminal ;) 
make

But beware: export only affects the terminal it's run from (so run it in each one / every time you reopen it)! This is why having a make script to run all commands to do is recommended!

/home/user/arm-summer-school/
├── toolchain/aarch64-none-linux-gnu/...
├── atf/            # ARM Trusted Firmware code
├── firmware/       # NXP IMX proprietary firmware BLOBs
├── uboot/          # U-Boot source code
├── imx-mkimage/    # IMX mkimage scripts
├── artifacts/      # You can copy build artifacts here
├── Makefile        # your makefile, highly recommended! :P
└── ...             # and many others to come!

Let's create an artifacts directory somewhere, we'll use it to copy the generated files to a central place.

This package will contain BL2 (U-Boot SPL), BL31 (ARM Trusted Firmware) and BL33 (normal/full U-Boot) and some proprietary firmware (yikes!). The Secure OS (BL32) is outside the scope of this lab; even if we did bother to include it, it would just be there, doing nothing.

Following this exercise, we should be able to reach the first step of the booting sequence that can be interactive; meaning that we'll be able to interact with a shell implemented in U-Boot!

Never mind the fact that we begin with BL31, you can consider it a warmup.

For this, we'll be using the Trusted Firmware-A project. Although it contains reference implementations for the other bootloader components as well, we are going to use it strictly for the BL31 role (trusted firmware initialization).

We recommend reading the documentation before moving forward. You can also use the build options as reference for its various build properties.

In particular, search for the keys to specifying the platform, cross compiler, the secure payload dispatcher (SPD – we will need to tell it that we need none, also read below); additionally, controlling logging levels and specifying console device (UART) is always useful for when you run into problems!

If you're still having problems, you can also read the u-boot mainline steps for FRDM iMX93.

Following a successful build process, you should obtain a bl31.bin file (take note if its location (use find to find it if lost) and copy it to your artifacts/ dir, for you will need it later).

With BL31 out of the way, we are going to tackle BL2 next (of course, BL1 is the first one to be loaded, but, fortunately, it comes carved inside our chip – into Read Only Memory).

BL1 is actually loaded from the SoC's ROM in the first half of the available Static RAM (the On-Chip RAM). This SRAM is just 256KB in size (remember, SRAM is quite expensive, similar to a cache memory), so there's not much space left for loading additional software.

Afterwards, BL1 loads BL2 in the upper half of SRAM and it stops here! the remainder of the firmware image is ignored. At this point it's up to BL2 to enable the rest of the memory (2GB of DRAM) and finish loading the rest of the FIP in main memory.

However, . So BL2 is to initialize the hardware using the proprietary firmware offered by the chip manufacturer. Without this firmware, we don't even have access to the DRAM memory.

You must download both firmware-imx-8.22.bin and firmware-sentinel-0.11.bin (URLs are taken from Yocto recipes) and download the self-extracting archives.

Run the binaries and accept the license agreement in order for it to extract its contents (oh, and since we're on Linux, don't forget to apply to executable bit – chmod +x <filename>).

What we're actually interested in are the following files (find them and put them inside your own artifacts directory, we'll make use of them later):

# from the firmware-imx (8 files!):
lpddr4_imem_1d_v202201.bin     lpddr4_dmem_1d_v202201.bin
lpddr4_imem_2d_v202201.bin     lpddr4_dmem_2d_v202201.bin
lpddr4_pmu_train_1d_dmem.bin   lpddr4_pmu_train_1d_imem.bin
lpddr4_pmu_train_2d_dmem.bin   lpddr4_pmu_train_2d_imem.bin
# and this is from firmware-sentinel (just 1 req., fortunately):
mx93a1-ahab-container.img

For the last two components of our Firmware Package we'll be using this U-Boot fork!

Each of them has a very specific purpose. At first, BL1 will start downloading the FIP (Firmware Image Package, which we'll generate later) using the Serial Download Protocol, running on top of a USB connection. Once it finishes receiving BL2 (SPL, together with the firmware binaries from Task 2.2.), it cedes control to it instead. BL2 will initialize the DRAM using said firmware and then continue where BL1 left off, finishing the download of the FIP.

Afterwards, BL31 (ARM Trusted Firmware) will run and initialize the so-called TrustZone secure hardware features (required for ARMv8) and BL33 (our final U-Boot stage) will be called upon. During this phase we'll finally have an interactive shell and multiple drivers to help interact with the board. With this, we can investigate the board's hardware, read and potentially alter the partitions in the persistent storage (on-board eMMC memory) and most importantly, boot Linux from any number of sources!

Clone this repo using git: The NXP iMX U-Boot fork (obligatory: clone the lf_v2024.04 branch! you can use git checkout to change branch afterwards).

Unfortunately, especially when using NXP's official U-Boot, our FRDM board is not supported! We'll need to download yet another thing (a patch) from NXP's Yocto source code: https://github.com/nxp-imx-support/meta-imx-frdm/blob/lf-6.6.36-2.1.0/meta-imx-bsp/recipes-bsp/u-boot/u-boot-imx/0002-imx-imx93_frdm-Add-basic-board-support.patch.

Save the file to u-boot`s source directory and apply it using:

# the -p1 argument is required to strip out the first path component 
# (which is a/b if you manually inspect the patch)
patch -p1 < 0002-imx-imx93_frdm-Add-basic-board-support.patch

If the patch fails some hunks, it's your fault :P (you haven't checked out the correct git branch as mentioned above).

Alright, let's get to it! U-Boot is based on the same build system as the Linux kernel, namely Kbuild. To get an idea of what functionality it provides, try to run:

$ make help

If you check the configs/ directory, you will find a number of board-specific configuration files. These serve as templates, containing the minimal necessary configuration. By running make some_defconfig the Kbuild system will determine what other unspecified options need to be enabled in order for these features to be functional. The result will be saved in a file called .config.

Generate a .config for your board by running the make <board's name>_defconfig (you can also find the name of the board's configuration inside the patch above or by using git status).

Also, don't forget the CROSS_COMPILE variable from BL31 (you've exported it, right? if not, pass it as KEY=VALUE' argument to make). It's very common across such projects and Kbuild will actually complain if it sees that you're trying to use a x86 compiler.

Let's manually explore our configuration using the menuconfig target! We're especially interested in the vendor/product values for the USB driver.

Open a ncurses-based interface for editing the .config file:

$ make CROSS_COMPILE=... menuconfig

The interface should be fairly intuitive. Use the Arrow keys to navigate the entries, Space to toggle options on or off, Enter to dive into a submenu or open a prompt, and the ? key to get more information about the currently selected entry. If you see a letter highlighted in a different color, pressing the corresponding key will take you to that option. Note that multiple options can have the same keybind; pressing it will cycle you through to the next occurrence.

The search function for a specific option (by name) is the exact same as in less or vim: /[CONFIG_]MY_OPTION <Enter>. This will generate a list of potential matches, each bearing a numeric index. Press the number key corresponding to that index in order to jump to the search result.

For now, change the following config variables and save the changes to .config:

• USB_GADGET_MANUFACTURER: FSL (abbreviated FreeScale Semiconductors – bought by NXP in 2015)
• USB_GADGET_VENDOR_NUM: 0x1fc9 (you can search it inside the USB VID database to see to whom it belongs to)
• USB_GADGET_PRODUCT_NUM: 0x0152

Run the make command (again, don't forget the CROSS_COMPILE argument, if you haven't exported it already)!

The four files you should obtain are:

• spl/u-boot-spl.bin: aka. Secondary Program Loader – BL2;
• u-boot-nodtb.bin: aka. BL33;
• u-boot.bin': still BL33 (with .dtb included).
• arch/arm/dts/imx93-11×11-frdm.dtb: a Device Tree Blob (DTB) which we'll also require – see below.

Copy them all inside your special artifacts directory (which you created earlier, right?).

dtc -I dtb -O dts imx93-11x11-frdm.dtb | less
# press Q to exit the paginator :p

Now that we have all necessary binaries either downloaded or compiled ourselves, all that is left is to combine them in a manner that can be understood by the processor's first boot stage (BL1 – inside ROM memory).

The format is specific to each model and is usually described inside the chip's Technical Reference Manual (note: requires sign in!).

In order to generate a binary image file in this format, we must use a specific tool ofc.

Since 2022, U-Boot's tool of choice for this task is binman. This tool uses a platform-specific config file that specifies what components should be included and where they should be placed in memory. For our platform (i.e.: i.MX93) this file would be arch/arm/dts/imx93-11×11-frdm-u-boot.dtsi.

However, since the U-Boot version that we are using is older and the board manufacturer did not add proper support for binman, we are going to use the older method, based on mkimage (part of the U-Boot repo or as a package on most distros). In order to spare ourselves some pain, we are going to use NXP's imx-mkimage implementation which knows the proper offsets where the images should be loaded… but beware: it's not very pretty to use!

Clone the imx-mkimage project from git and cd to it! In their source tree you will find a number of subdirectories corresponding to different versions of the i.MX platform. Select the one which corresponds to our board:

imx-mkimage
├── iMX8DXL
├── iMX8M
├── iMX8QM
├── iMX8QX
├── iMX8ULP
├── iMX91
├── iMX93
├── iMX94
├── iMX95
├── scripts
└── src

When you get there (that is, inside the iMX93 subdirectory), you will have to copy all the artifacts (remember we had you manage them all inside a new folder?) you compiled so far + the ones downloaded/extracted from proprietary firmware archive (trust us here: make a script/Makefile goal to do all this automatically! you'll need to do it tens – probably hundreds – of times!).

In addition to these, you will have to copy the base mkimage tool generated in the U-Boot directory, see if you can find it ;) you must rename it as mkimage_uboot inside the iMX93 subdirectory.

Once you have all these (see below), run make in the parent directory of imx-mkimage (check and see where it has that Makefile!) with the flash_singleboot target, while specifying the platform in the SOC=iMX93 argument (note that cASE!), and the name of the DTB copied over from U-Boot in the dtbs= argument. The output firmware image should be called flash.bin. Here's the final contents of the directory:

imx-mkimage/iMX93
├── bl31.bin                       # ARM Trusted Firmware
├── boot-spl-container.img         # also generated
├── flash.bin                      # generated by `make SOC=... dtbs=...`
├── head.hash
├── imx93-11x11-frdm.dtb           # copied from u-boot
├── lpddr4_dmem_1d_v202201.bin     #|
├── lpddr4_dmem_2d_v202201.bin#    #|
├── lpddr4_imem_1d_v202201.bin     #|
├── lpddr4_imem_2d_v202201.bin     #|- copied from firmware-imx
├── lpddr4_pmu_train_1d_dmem.bin   #|
├── lpddr4_pmu_train_1d_imem.bin   #|
├── lpddr4_pmu_train_2d_dmem.bin   #|
├── lpddr4_pmu_train_2d_imem.bin   #|
├── mkimage_uboot                  # copied & renamed from u-boot!
├── mx93a1-ahab-container.img      # copied from firmware-sentinel
├── scripts
│   └── autobuild.mak
├── signature.dts
├── soc.mak
├── u-boot-atf-container.img       # generated..
├── u-boot.bin                     # copied from u-boot
├── u-boot-hash.bin
├── u-boot-nodtb.bin               # copied from u-boot
├── u-boot-spl.bin                 # copied from u-boot
└── u-boot-spl-ddr.bin             # guess what? generated!

We told you it won't be pretty, didn't we?

$ dtc -I dtb -O dts u-boot.itb | less

The i.MX93 family of processors can boot from multiple sources: eMMC, SD card, etc. Our board lacks an SD card so this method can be discarded right off the bat. Although booting from eMMC is possible, it would require us to overwrite its persistent storage with a disk image every single time. We will do this at some point later in our workshop but, for now, the most convenient solution is using SDP (Serial Download Protocol) to download flash.bin via our USB connection to the board directly into the chip's SRAM (and execute it there!).

In order to select Serial Download as the preferred method of boot, you will need to set four DIP switches on the board. Find the BOOT label and configure them according to the figure below:

The Universal Update Utility (UUU) is an image deployment tool created by NXP for it's i.MX architecture. For more information about its usage, scripting interface and supported protocols, check out the documentation.

Grab the source code from here and compile uuu. The project uses the cmake build system for the sake of portability. If you haven't encountered it yet, follow these setup steps:

# currently in the mfgtools repo root
mkdir build
cd build
cmake ..
make -j $(nproc)

First things first, connect the three USB-C cables (Power to the AC adapter, Debug and USB_C labeled ports to your PC). While the former will be used to power on the board, the latter will expose two Serial Devices and a USB Bootloader communication line with your computer. The one used for console I/O by the bootloaders should appear to you as /dev/ttyACM0.

Connect to this device using a serial terminal emulation tool of your choice. We recommend picocom. The default baud rate of i.MX devices (actually, most embedded Linux devices) is 115200 by convention. But don't trust us on this: check the manual ;)

Note that nothing will be printed on the console yet, but you need to stay connected to receive the messages that will follow!

Ever since the USB-C cable was plugged in, BL1 has been waiting for the FIP data over serial, thanks to our jumper configuration. Now we can finally provide this data using uuu and it's SDP implementation.

# basic syntax: scp <source> <destination>
# either source or destination MUST be a remote SSH with the form:
# username@host:/path/to/file
# Note: -P specifies the port (your VM index!)
# example (destination is "./" -- your current directory):
scp -P 220X your-user@arm2026.root.sx:path/to/flash.bin ./

# paths for uuu and flash.bin truncated
uuu -b spl flash.bin

About now you should see some debug messages in your serial console tool.

Finally, when everything is in working condition, you should see a Run fastboot … message, press Ctrl + C and you should get the u-boot=> prompt.

You might be wondering: but we don't have an OS installed yet… is this all we can do now?

Don't worry, we'll now get to see why u-boot is the most popular choice for embedded devices (here's another fact: most Android phones also use it, although it's not popular anymore since most manufacturers migrated to proprietary ones).

Now that we finally have access to the interactive shell (we've stopped at BL33), try to run bdinfo for some generic board information. Run help to see what other commands are available to you, and help <command> for detailed information of said command. Note that this may not be an exhaustive list of commands; some may not have been compiled into the final binary, depending on your .config.

Try to perform the following:

• Print the available environment (env u-boot command).
• Check out the vendor and System on Chip (SOC) values.
• Print the available eMMC devices, as well as their partition tables (if any are available).
• Perform a memory test on the first GB of DRAM. Note that U-Boot relocates itself toward the end of the DRAM bank during its initialization phase (check bdinfo for the exact address). Stay away from that region unless you want to overwrite BL33 itself with your test pattern.

Finally, let's test the USB Mass Storage emulation of the eMMC using u-boot (which will allow us to access the board's internal flash memory direcrly from our ;laptops / PCs via USB-C). Try the ums command (use Google/LLMs for its arguments ;) ).