Well, we want to compile things. We'll be using the personal PC / laptop, which is probably running on a Intel/AMD-based x86-64 CPU architecture.

But we want to compile code for a foreign architecture: Aarch64 (also known as arm64). So we will need a cross compiler: a compiler that runs on your host but generates binary code for another target.

Cross compilers are usually named using this convention <target_arch>-<vendor>-[<os>-]<abi>-gcc. In our case, it will be aarch64-none-linux-gnu-gcc:

• aarch64: The target architecture; ARM 64-bit (why isn't it called this way? don't ask us :( …).
• none: The toolchain supplier. Here, we're dealing with the official cross compiler from ARM so the vendor is not relevant.
• linux: The target OS (this filed is optional).
• gnu: The Application Binary Interface (ABI) defining how the generated code will interact with other software components. Don't mistake this for gnueabi, which is meant to generate bare metal code.

Identify the correct toolchain and download it from here. Yes, you can also install it with your package manger but it's always better to have a project-specific copy. New compiler releases are known to sometime break the build process.

After extracting it, consider exporting the compiler's prefix (aarch64-none-linux-gnu-) from your newly downloaded toolchain as the CROSS_COMPILE environment variable. Otherwise, you will have to specify the full path of the cross compiler every time when invoking a script. E.g.:

export CROSS_COMPILE=/absolute/path/to/compiler/bin/aarch64-none-linux-gnu-
# now you can use the cross compiler as such:
"${CROSS_COMPILE}gcc" ...
# make will inherit this value, so no need to add a CROSS_COMPILE 
# when invoking it from your terminal ;) 
make

But beware: export only affects the terminal it's run from (so run it in each one :P)!

/home/user/arm-summer-school/
├── toolchain/aarch64-none-linux-gnu/...
├── imx-atf/        # ARM Trusted Firmware code
├── firmware/       # NXP IMX proprietary firmware BLOBs
├── u-boot-tn-imx/  # U-Boot source code
├── imx-mkimage/    # IMX mkimage scripts
├── Makefile        # your makefile, highly recommended! :P
└── ...             # and many others to come!

This package will contain BL2, BL31 and BL33. The Secure OS (BL32) is outside the scope of this lab; even if we did bother to include it, it would just be there, doing nothing.

Following this exercise, we should be able to reach the first step of the booting sequence that can be interactive; meaning that we'll be able to interact with a shell implemented in BL33.

Never mind the fact that we begin with BL31, you can consider it a warmup.

For this, we'll be using the Trusted Firmware-A project. Although it contains reference implementations for the other bootloader components as well, we are going to use it strictly for BL31. The others have more complete and widely recognized alternatives available.

We recommend reading the documentation before moving forward. Also use the build options as reference.

In particular, search for the keys to specifying the platform, cross compiler, the secure payload dispatcher (we will need to tell it that we need none, also read below); additionally, controlling logging levels and specifying console device (UART) is always useful for when you run into problems!

Following a successful build process, you should obtain a bl31.bin file (take note if its location, for you will need it later).

With BL31 out of the way, we are going to tackle BL2 next (of course, BL1 is the first one to be loaded, but, fortunately, it comes carved inside our chip – into Read Only Memory).

BL1 is actually loaded from the SoC's ROM in the first half of the available Static RAM (the On-Chip RAM). This SRAM is just 128KB in size (remember, SRAM is quite expensive, similar to a cache memory), so there's not much space left for loading additional software.

Afterwards, BL1 loads BL2 in the upper half of SRAM and it stops here! the remainder of the firmware image is ignored. At this point it's up to BL2 to enable the rest of the memory (2GB of DRAM) and finish loading the rest of the FIP in main memory.

However, . So BL2 is to initialize the hardware using the proprietary firmware offered by the chip manufacturer. Without this firmware, we don't even have access to the DRAM memory.

Simply go here and download the self-extracting archive. After this, run it and accept the license agreement in order for it to extract its contents (oh, and since we're on Linux, don't forget to apply to executable bit – chmod +x <filename>).

What we're actually interested in are the following files:

• lpddr4_pmu_train_1d_dmem.bin
• lpddr4_pmu_train_1d_imem.bin
• lpddr4_pmu_train_2d_dmem.bin
• lpddr4_pmu_train_2d_imem.bin
• signed_hdmi_imx8m.bin

For the last two components of our Firmware Package we'll be using this U-Boot fork (clone it!). Each of them has a very specific purpose.

At first, BL1 will start downloading the FIP (Firmware Image Package, which we'll generate later) using the Serial Download Protocol, running on top of a USB connection. Once it finishes receiving BL2 (together with the firmware binaries from Task B), it cedes control to it instead. BL2 will initialize the DRAM using said firmware and then continue where BL1 left off, finishing the download of the FIP.

When BL31 runs its course, BL33 will be called upon. During this phase we'll finally have an interactive shell and multiple drivers to help interact with the board. With this, we can investigate the board's hardware, read and potentially override the partitions in the persistent storage and most importantly, boot Linux from any number of sources.

Alright, let's get to it! U-Boot is based on the same build system as the Linux kernel, namely Kbuild. To get an idea of what functionality it provides, try to run:

$ make help

If you check the configs/ directory, you will find a number of board-specific configuration files. These serve as templates, containing the minimal necessary configuration. By running make some_defconfig the Kbuild system will determine what other unspecified options need to be enabled in order for these features to be functional. The result will be saved in a file called .config.

Generate a .config for your board by running the make <board's name>_defconfig.

Also, don't forget the CROSS_COMPILE variable from BL31 (you've exported it, right? if not, pass it as KEY=VALUE' argument to make). It's very common across such projects and Kbuild will actually complain if it sees that you're trying to use a x86 compiler.

The default configuration that you chose (correctly, hopefully) contains a few erroneous values for the USB driver. Normally these would take some time to dig up from the board's / processor's documentation / source code; we took them from the debug prints of the firmware that was pre-configured on the eMMC :P

Open a ncurses-based interface for editing the .config file:

$ make CROSS_COMPILE=... menuconfig

The interface should be fairly intuitive. Use the Arrow keys to navigate the entries, Space to toggle options on or off, Enter to dive into a submenu or open a prompt, and the ? key to get more information about the currently selected entry. If you see a letter highlighted in a different color, pressing the corresponding key will take you to that option. Note that multiple options can have the same keybind; pressing it will cycle you through to the next occurrence.

The search function for a specific option (by name) is the exact same as in less or vim: /[CONFIG_]MY_OPTION <Enter>. This will generate a list of potential matches, each bearing a numeric index. Press the key corresponding to that index in order to jump to the search result.

For now, change the following config variables and save the changes to .config:

• USB_GADGET_MANUFACTURER: ASS
• USB_GADGET_VENDOR_NUM: 0x1fc9
• USB_GADGET_PRODUCT_NUM: 0x012b

Run the make command (again, don't forget the cross compiler argument, if you haven't exported it already)!

The three files you should obtain are:

• spl/u-boot-spl.bin: aka. BL2.
• u-boot-nodtb.bin: aka. BL33.
• arch/arm/dts/imx8mq-pico-pi.dtb: a Device Tree Blob (DTB) which we'll also require; see details below.

# press Q to exit the paginator :p
$ dtc -I dtb -O dts imx8mq-pico-pi.dtb | less

Now that we have all necessary binaries either downloaded or compiled ourselves, all that is left is to combine them in a manner that can be understood by the processor's first boot stage (BL1).

Since 2022, U-Boot's tool of choice for this task is binman. This tool uses a platform-specific config file that specifies what components should be included and where they should be placed in memory. For our platform (i.e.: i.MX8M Quad) this file would be arch/arm/dts/imx8mq-u-boot.dtsi.

However, since the U-Boot version that we are using is older and the board manufacturer did not add proper support for binman, we are going to use the older method, based on mkimage (part of the U-Boot repo or as a package on most distros). In order to spare ourselves some pain, we are going to use NXP's imx-mkimage implementation which knows the proper offsets where the images should be loaded.

In their source tree you will find a number of subdirectories corresponding to different versions of the i.MX platform. Select the one which corresponds to our board (remember, the base model is called iMX8M). When you get there, you will have to copy all the bootloaders you compiled so far, as well as the downloaded firmware (trust us here: make a script to do this automatically! you'll need to do it tens – probably hundreds – of times!).

In addition to these, you will have to copy the base mkimage tool (generated in the U-Boot directory, see if you can find it ;) ); rename it as mkimage_uboot.

Once you have all these, run make with the flash_evk target, while specifying the platform in the SOC= make argument, and the name of the DTB copied over from U-Boot in the dtbs= argument. The output firmware image should be called flash.bin.

$ dtc -I dtb -O dts u-boot.itb | less

The i.MX8M family of processors can boot from multiple sources: eMMC, SD card, etc. Our board lacks an SD card slot so this method can be discarded right off the bat. Although booting from eMMC is possible, it would require us to overwrite its persistent storage with a disk image every single time. We will do this at some point in the following sessions but for now the most convenient solution is using SDP to download flash.bin via a serial connection, over USB.

In order to select Serial Download as the preferred method of boot, you will need to set two jumpers on the board. Find J1 and J2 and configure them according to the figure below.

The Universal Update Utility (UUU) is an image deployment tool created by NXP for it's i.MX architecture. For more information about its usage, scripting interface and supported protocols, check out the documentation.

Grab the source code from here and compile *uuu*. The project uses the cmake build system for the sake of portability. If you haven't encountered it yet, follow these setup steps:

# currently in the mfgtools repo root
 
$ mkdir build && cd build
$ cmake ..
$ make -j $(nproc)

First things first, connect the Micro USB and USB-C cables. While the latter will be used to power on the board, the former will expose two Serial Devices to your computer. The one used for console I/O by the bootloaders should appear to you as /dev/ttyUSB0.

Connect to this device using a serial terminal emulation tool of your choice. We recommend picocom. The default baud rate of i.MX devices is 115200 by convention.

Note that nothing will be printed on the console yet, but you need to stay connected to receive the messages that will follow!

Ever since the USB-C cable was plugged in, BL1 has been waiting for the FIP data over serial, thanks to our jumper configuration. Now we can finally provide this data using uuu and it's SDP implementation:

# paths for uuu and flash.bin truncated
$ uuu -b spl flash.bin

About now you should see some debug messages in your serial console tool.

Unfortunately (as per Murphy's law), notice we have some boot errors (: don't worry, this is expected!

You should get this error:

Trying to boot from USB SDP
alloc space exhausted
failed to initialize gadget
couldn't find an available UDC
g_dnl_register: failed!, error: -19
SDP dnl register failed: -19

The proper debugging procedure is to find the location of the exact error message inside u-boot's source code.

It seems that the malloc() call fails due to memory exhaustion. Recall that BL2 (U-Boot SPL) runs entirely from the internal SRAM of the chip, which is just 128K! But we also saw some messages about 2GB of DRAM becoming initialized, but we still need to properly tell our bootloader to use them!

Thus, go back to Step 4 (Modifying U-Boot configuration) and do some more configuration changes! But first, we must find out which options control the malloc pools. You can do that by reading the source code here and here.

Use menuconfig to search for the following options:

• SYS_SPL_MALLOC – must enable additional malloc pools;
• SPL_MALLOC_ADDR – enter a physical address inside the board's DRAM memory (reminder: DDR starts from 0x40000000);
• SPL_MALLOC_SIZE – make it bigger, e.g. 1MB (in hex!);

CONFIG_SYS_SPL_MALLOC=y
CONFIG_HAS_CUSTOM_SPL_MALLOC_START=y
CONFIG_CUSTOM_SYS_SPL_MALLOC_ADDR=0x42200000
CONFIG_SYS_SPL_MALLOC_SIZE=0x100000
CONFIG_USB_GADGET_MANUFACTURER="ASS"
CONFIG_USB_GADGET_VENDOR_NUM=0x1fc9
CONFIG_USB_GADGET_PRODUCT_NUM=0x012b
CONFIG_USB_FUNCTION_SDP=y

You may enter them inside a simple text file (e.g.: ass-extra.config; don't forget the CONFIG_* prefix) and use the following script to mergem them inside the main .config:

# note: run this inside u-boot's source directory
scripts/kconfig/merge_config.sh ".config" "ass-extra.config"

Return after re-building the entire firmware image (i.e. flash.bin – including the mkimage step – hope you've scripted it in a Makefile!).

Finally, when everything is in working condition, you should see a Run fastboot … message, press Ctrl + C and you should get the u-boot=> prompt.

You might be wondering: but we don't have an OS installed yet… is this all we can do now?

Don't worry, we'll now get to see why u-boot is the most popular choice for embedded devices (here's another fact: most Android phones also use it)!

Now that we finally have access to the interactive shell (we've stopped at BL33), try to run bdinfo for some generic board information. Run help to see what other commands are available to you, and help <command> for detailed information of said command. Note that this may not be an exhaustive list of commands; some may not have been compiled into the final binary, depending on your .config.

Try to perform the following:

• Get the vendor and System on Chip (SOC) name from the environment variables.
• Print the available eMMC devices, as well as their partition tables (if any are available).
• Perform a memory test on the first GB of DRAM. Note that U-Boot relocates itself toward the end of the DRAM bank during its initialization phase (check bdinfo for the exact address). Stay away from that region unless you want to overwrite BL33 itself with your test pattern.