This package will contain BL2 (U-Boot SPL), BL31 (ARM Trusted Firmware) and BL33 (normal/full U-Boot) and some proprietary firmware (yikes!). The Secure OS (BL32) is outside the scope of this lab; even if we did bother to include it, it would just be there, doing nothing.

Following this exercise, we should be able to reach the first step of the booting sequence that can be interactive; meaning that we'll be able to interact with a shell implemented in U-Boot!

Never mind the fact that we begin with BL31, you can consider it a warmup.

For this, we'll be using the Trusted Firmware-A project. Although it contains reference implementations for the other bootloader components as well, we are going to use it strictly for the BL31 role (trusted firmware initialization).

We recommend reading the documentation before moving forward. You can also use the build options as reference for its various build properties.

In particular, search for the keys to specifying the platform, cross compiler, the secure payload dispatcher (SPD – we will need to tell it that we need none, also read below); additionally, controlling logging levels and specifying console device (UART) is always useful for when you run into problems!

If you're still having problems, you can also read the u-boot mainline steps for FRDM iMX93.

Following a successful build process, you should obtain a bl31.bin file (take note if its location (use find to find it if lost) and copy it to your artifacts/ dir, for you will need it later).

With BL31 out of the way, we are going to tackle BL2 next (of course, BL1 is the first one to be loaded, but, fortunately, it comes carved inside our chip – into Read Only Memory).

BL1 is actually loaded from the SoC's ROM in the first half of the available Static RAM (the On-Chip RAM). This SRAM is just 256KB in size (remember, SRAM is quite expensive, similar to a cache memory), so there's not much space left for loading additional software.

Afterwards, BL1 loads BL2 in the upper half of SRAM and it stops here! the remainder of the firmware image is ignored. At this point it's up to BL2 to enable the rest of the memory (2GB of DRAM) and finish loading the rest of the FIP in main memory.

However, . So BL2 is to initialize the hardware using the proprietary firmware offered by the chip manufacturer. Without this firmware, we don't even have access to the DRAM memory.

You must download both firmware-imx-8.22.bin and firmware-sentinel-0.11.bin (URLs are taken from Yocto recipes) and download the self-extracting archives.

Run the binaries and accept the license agreement in order for it to extract its contents (oh, and since we're on Linux, don't forget to apply to executable bit – chmod +x <filename>).

What we're actually interested in are the following files (find them and put them inside your own artifacts directory, we'll make use of them later):

# from the firmware-imx (8 files!):
lpddr4_imem_1d_v202201.bin     lpddr4_dmem_1d_v202201.bin
lpddr4_imem_2d_v202201.bin     lpddr4_dmem_2d_v202201.bin
lpddr4_pmu_train_1d_dmem.bin   lpddr4_pmu_train_1d_imem.bin
lpddr4_pmu_train_2d_dmem.bin   lpddr4_pmu_train_2d_imem.bin
# and this is from firmware-sentinel (just 1 req., fortunately):
mx93a1-ahab-container.img

For the last two components of our Firmware Package we'll be using this U-Boot fork!

Each of them has a very specific purpose. At first, BL1 will start downloading the FIP (Firmware Image Package, which we'll generate later) using the Serial Download Protocol, running on top of a USB connection. Once it finishes receiving BL2 (SPL, together with the firmware binaries from Task 2.2.), it cedes control to it instead. BL2 will initialize the DRAM using said firmware and then continue where BL1 left off, finishing the download of the FIP.

Afterwards, BL31 (ARM Trusted Firmware) will run and initialize the so-called TrustZone secure hardware features (required for ARMv8) and BL33 (our final U-Boot stage) will be called upon. During this phase we'll finally have an interactive shell and multiple drivers to help interact with the board. With this, we can investigate the board's hardware, read and potentially alter the partitions in the persistent storage (on-board eMMC memory) and most importantly, boot Linux from any number of sources!

Clone this repo using git: The NXP iMX U-Boot fork (obligatory: clone the lf_v2024.04 branch! you can use git checkout to change branch afterwards).

Unfortunately, especially when using NXP's official U-Boot, our FRDM board is not supported! We'll need to download yet another thing (a patch) from NXP's Yocto source code: https://github.com/nxp-imx-support/meta-imx-frdm/blob/lf-6.6.36-2.1.0/meta-imx-bsp/recipes-bsp/u-boot/u-boot-imx/0002-imx-imx93_frdm-Add-basic-board-support.patch.

Save the file to u-boot`s source directory and apply it using:

# the -p1 argument is required to strip out the first path component 
# (which is a/b if you manually inspect the patch)
patch -p1 < 0002-imx-imx93_frdm-Add-basic-board-support.patch

If the patch fails some hunks, it's your fault :P (you haven't checked out the correct git branch as mentioned above).

Alright, let's get to it! U-Boot is based on the same build system as the Linux kernel, namely Kbuild. To get an idea of what functionality it provides, try to run:

$ make help

If you check the configs/ directory, you will find a number of board-specific configuration files. These serve as templates, containing the minimal necessary configuration. By running make some_defconfig the Kbuild system will determine what other unspecified options need to be enabled in order for these features to be functional. The result will be saved in a file called .config.

Generate a .config for your board by running the make <board's name>_defconfig (you can also find the name of the board's configuration inside the patch above or by using git status).

Also, don't forget the CROSS_COMPILE variable from BL31 (you've exported it, right? if not, pass it as KEY=VALUE' argument to make). It's very common across such projects and Kbuild will actually complain if it sees that you're trying to use a x86 compiler.

Let's manually explore our configuration using the menuconfig target! We're especially interested in the vendor/product values for the USB driver.

Open a ncurses-based interface for editing the .config file:

$ make CROSS_COMPILE=... menuconfig

The interface should be fairly intuitive. Use the Arrow keys to navigate the entries, Space to toggle options on or off, Enter to dive into a submenu or open a prompt, and the ? key to get more information about the currently selected entry. If you see a letter highlighted in a different color, pressing the corresponding key will take you to that option. Note that multiple options can have the same keybind; pressing it will cycle you through to the next occurrence.

The search function for a specific option (by name) is the exact same as in less or vim: /[CONFIG_]MY_OPTION <Enter>. This will generate a list of potential matches, each bearing a numeric index. Press the number key corresponding to that index in order to jump to the search result.

For now, change the following config variables and save the changes to .config:

• USB_GADGET_MANUFACTURER: FSL (abbreviated FreeScale Semiconductors – bought by NXP in 2015)
• USB_GADGET_VENDOR_NUM: 0x1fc9 (you can search it inside the USB VID database to see to whom it belongs to)
• USB_GADGET_PRODUCT_NUM: 0x0152

Run the make command (again, don't forget the CROSS_COMPILE argument, if you haven't exported it already)!

The four files you should obtain are:

• spl/u-boot-spl.bin: aka. Secondary Program Loader – BL2;
• u-boot-nodtb.bin: aka. BL33;
• u-boot.bin': still BL33 (with .dtb included).
• arch/arm/dts/imx93-11×11-frdm.dtb: a Device Tree Blob (DTB) which we'll also require – see below.

Copy them all inside your special artifacts directory (which you created earlier, right?).

dtc -I dtb -O dts imx93-11x11-frdm.dtb | less
# press Q to exit the paginator :p

Now that we have all necessary binaries either downloaded or compiled ourselves, all that is left is to combine them in a manner that can be understood by the processor's first boot stage (BL1 – inside ROM memory).

The format is specific to each model and is usually described inside the chip's Technical Reference Manual (note: requires sign in!).

In order to generate a binary image file in this format, we must use a specific tool ofc.

Since 2022, U-Boot's tool of choice for this task is binman. This tool uses a platform-specific config file that specifies what components should be included and where they should be placed in memory. For our platform (i.e.: i.MX93) this file would be arch/arm/dts/imx93-11×11-frdm-u-boot.dtsi.

However, since the U-Boot version that we are using is older and the board manufacturer did not add proper support for binman, we are going to use the older method, based on mkimage (part of the U-Boot repo or as a package on most distros). In order to spare ourselves some pain, we are going to use NXP's imx-mkimage implementation which knows the proper offsets where the images should be loaded… but beware: it's not very pretty to use!

Clone the imx-mkimage project from git and cd to it! In their source tree you will find a number of subdirectories corresponding to different versions of the i.MX platform. Select the one which corresponds to our board:

imx-mkimage
├── iMX8DXL
├── iMX8M
├── iMX8QM
├── iMX8QX
├── iMX8ULP
├── iMX91
├── iMX93
├── iMX94
├── iMX95
├── scripts
└── src

When you get there (that is, inside the iMX93 subdirectory), you will have to copy all the artifacts (remember we had you manage them all inside a new folder?) you compiled so far + the ones downloaded/extracted from proprietary firmware archive (trust us here: make a script/Makefile goal to do all this automatically! you'll need to do it tens – probably hundreds – of times!).

In addition to these, you will have to copy the base mkimage tool generated in the U-Boot directory, see if you can find it ;) you must rename it as mkimage_uboot inside the iMX93 subdirectory.

Once you have all these (see below), run make in the parent directory of imx-mkimage (check and see where it has that Makefile!) with the flash_singleboot target, while specifying the platform in the SOC=iMX93 argument (note that cASE!), and the name of the DTB copied over from U-Boot in the dtbs= argument. The output firmware image should be called flash.bin. Here's the final contents of the directory:

imx-mkimage/iMX93
├── bl31.bin                       # ARM Trusted Firmware
├── boot-spl-container.img         # also generated
├── flash.bin                      # generated by `make SOC=... dtbs=...`
├── head.hash
├── imx93-11x11-frdm.dtb           # copied from u-boot
├── lpddr4_dmem_1d_v202201.bin     #|
├── lpddr4_dmem_2d_v202201.bin#    #|
├── lpddr4_imem_1d_v202201.bin     #|
├── lpddr4_imem_2d_v202201.bin     #|- copied from firmware-imx
├── lpddr4_pmu_train_1d_dmem.bin   #|
├── lpddr4_pmu_train_1d_imem.bin   #|
├── lpddr4_pmu_train_2d_dmem.bin   #|
├── lpddr4_pmu_train_2d_imem.bin   #|
├── mkimage_uboot                  # copied & renamed from u-boot!
├── mx93a1-ahab-container.img      # copied from firmware-sentinel
├── scripts
│   └── autobuild.mak
├── signature.dts
├── soc.mak
├── u-boot-atf-container.img       # generated..
├── u-boot.bin                     # copied from u-boot
├── u-boot-hash.bin
├── u-boot-nodtb.bin               # copied from u-boot
├── u-boot-spl.bin                 # copied from u-boot
└── u-boot-spl-ddr.bin             # guess what? generated!

We told you it won't be pretty, didn't we?

$ dtc -I dtb -O dts u-boot.itb | less