There are three vital components that are required to successfully boot Linux on ARM:

• Kernel image: This should be self-evident :p
• Flattened Device Tree: A structured binary blob of data that the kernel parses to identify hardware elements.
• RootFS: A file system that can be mounted at / in order to load init. Here, we will use a ramdisk image.

Although these components can be manually loaded and configured for boot from U-Boot, it's preferable to package them together in a Flattened μImage Tree (FIT). Similarly to how bl1 and bl2 know how to load the binaries that comprise the FIP we generated in the previous lab, so does bl33 know how to boot the system from a FIT.

In the following tasks we will create each individual component and package them together. Finally, we will create a partition table (with one FAT partition) on the board's eMMC and store the FIT only to then load it and boot it from the bl33 shell.

Clone the kernel from the official GitHub repo, using the v6.6 tag. We recommend you to use this git cloning technique to avoid fetching the entire git history (requiring several GBs of disk space).

Since you've built U-Boot previously, you should be somewhat familiar with the Kbuild system. Start by generating the build configuration from its default configuration template. Optionally, run a menuconfig to inspect the Linux kernel options available.

In order to build the kernel image, simply make with the required variables (see the warning above!).

Note that the kernel image you will be including in the FIP is called Image. Unlike vmlinux which is an ELF file (i.e.: contains useless sections, including .debuginfo if you want to debug the kernel), Image is a boot executable image, made specifically for you to jump straight in and start executing. After the build process finalizes, find the Image file within the <linux-kernel-src>/arch/arm64/boot/ output directory.

While waiting for the build, explore the Linux source using your favorite code editor (especially the device trees)!

./scripts/clang-tools/gen_compile_commands.py

Luckily, the FDT for our platform is also included in Linux. It's name should be imx8mq-pico-pi.dtb (generated from its source counterpart, imx8mq-pico-pi.dts).

If for some reason it wasn't built alongside the kernel Image, check out the dtbs make target.

Usually there are two approaches to generating the root filesystem: bootstrapping from a source of pre-compiled binaries (e.g.: debootstrap, pacstrap, etc.) or building them yourself (e.g.: Yocto, Buildroot.) The former is usually preferred when working on desktop environments. The latter allows you to fine tune everything that is installed on your system. For example, the OpenSSH package available via pacstrap will come with a default build of sshd (the SSH server). On some critical infrastructure however, you might want to harden the SSH server binary by using a custom dynamic loader instead of ld-linux which allows LD_PRELOAD hooks.

Between Yocto and Buildroot, we generally opt for Buildroot. While Yocto has its advantages (e.g.: wide industry adoption, extensive device support, etc.), it also has a very steep learning curve and consumes significant amounts of resources (i.e.: ~50G storage space for a build). Although it has a layer-based build customization feature (think Dockerfiles extending base images), we argue that this makes it more difficult to comprehend the contents of what is being built.

Buildroot on the other hand is geared towards simplicity and ease of use. Being based on Kbuild and Makefile (same as Linux and U-Boot) makes it instantly familiar to most developers. And even if you are new to this, while Yocto requires you to read entire books in order to utilize it properly, Buildroot can be summed up in a 1k LoC Makefile.

Clone the official Buildroot repo.

Find an existing defconfig for our platform (the imx8mq) and take a look at what it contains. Notice how the following config options are enabled:

BR2_LINUX_KERNEL=y
BR2_TARGET_ARM_TRUSTED_FIRMWARE=y
BR2_TARGET_UBOOT=y

Yes, Buildroot also integrates the bootloaders into its build system. However, keeping the components separate makes it easier to appply patches and debug problems. What we want from Buildroot is a minimal root filesystem and nothing more. So, after generating .config, enter menuconfig and make a few changes:

• Disable all components that we've already integrated in our build environment (starting with the ones above).
• Use an external, custom, pre-installed toolchain (remember CROSS_COMPILE? check the menu for it!).
• Use systemd as the init system instead of BusyBox.
  • Make sure systemd-logind is included in the build, or you may have this problem.
• Choose bash as the default shell instead of sh.
• Enable password login for the root user, then set a password.
• Include a text editor PACKAGE (e.g.: vim or nano).
• Include the coreutils PACKAGE.
• Generate an uncompressed CPIO image from the output file system (see the Note below).

$ cd /path/to/rootfs
$ find . | cpio -o > /path/to/ramdisk.cpio

Feel free to add / remove anything else you want. Anything goes as long as you end up with a functioning system. After completing Step 3, maybe return and see if there's anything left to trim down to speed up the build process

Not much to it, really. Once everything's done, check out the output/ directory. What does it contain? Where is your CPIO archive?

Create a staging directory and copy everything that we've obtained from the previous three tasks. Then, create an Image Tree Source (ITS) file. We'll be referring to it as linux.its but the name doesn't really matter. What matters is the content:

/dts-v1/;

/ {
    description = "ASS - Linux FIT image for Pico Pi";
    #address-cells = <1>;

    images {
        kernel {
            description = "Linux kernel";
            data = /incbin/("Image");
            type = "kernel";
            arch = "arm64";
            os = "linux";
            compression = "none";
            load = <XXX>;
            entry = <XXX>;
        };
        fdt {
            description = "Device tree";
            data = /incbin/("imx8mq-pico-pi.dtb");
            type = "flat_dt";
            arch = "arm64";
            compression = "none";
            load = <YYY>;
        };
        initrd {
            description = "Ramdisk";
            data = /incbin/("rootfs.cpio");
            type = "ramdisk";
            arch = "arm64";
            os = "linux";
            compression = "none";
            load = <ZZZ>;
        };
    };

    configurations {
        default = "normal-boot";

        normal-boot {
            description = "Normal boot config";
            kernel = "kernel";
            fdt = "fdt";
            ramdisk = "initrd";
        };
    };
};

Let's unpack this:

• The file starts with /dts-v1/;, identifying it as a Device Tree Source.
• Next, we have a root / node with two child nodes: images and configurations.
  • images contains the description of each binary that will need to be loaded by U-Boot when processing the FIT.
    • Each image entry contains an incbin directive that tells mkimage to copy paste the contents of the specified file into the output DTB.
    • Each image also contains a load property, specifying the address where the data will be placed.
    • In addition to load, the kernel image also has an entry property specifying where U-Boot will jump when relinquishing control to the Linux kernel.
  • configurations contains sets of image configurations that can be chained.
    • The only configuration we have is also the default: normal-boot.
    • Notice how it has pre-defined attributes for kernel, fdt, ramdisk. These are not simple binary blobs; instead, they each have a role to play in the boot sequence. E.g., U-Boot will know to take the load address of the fdt image and pass it via a certain register (decided by convention) to the kernel, informing it where in memory to look for the FDT.

Note how we replaced the load and entry addresses with placeholders such as XXX. Replace these with whatever addresses you want such as the binaries do not overlap once U-Boot starts loading them.

Once all the addresses are filled in, generate the Image Tree Blob (ITB) using mkimage. Best not to use the imx-mkimage; instead, install mkimage using your distro's package manager.

$ mkimage -f linux.its linux.itb