The initial step would be to embed OP-TEE OS into our firmware boot package and re-configure (and recompile) ARM Trusted Firmware-A to load it.

But first, we need to understand the memory organization of our board. This is required since BL31 (TF-A) has to configure the TZASC peripheral to reserve a trusted region inside the main memory (DRAM) for use by our Secure OS & TAs.

In the figure to the right, we use the upper end of the DDR1 memory (36 MB) for the TrustZone OS (BL32). We will need to calculate the addresses and supply them as configuration parameters when compiling the software.

In order to get the actual physical addresses, one needs to check out the official NXP iMX8MQ Reference Manual, Chapter 2.1.2 (Cortex-A53 Memory Map, pg. 20). We note that the first DRAM (DDR1) is mapped at 0x40000000 (exactly 1GB from the beginning the address space) and ends at 0xFFFFFFFF (the difference of the two addresses totals to 3072MB == 3 * 1024MB == 3GB).

But, since our boards only come with 2GB of LPDDR chips installed, we need to recalculate the end of our DRAM memory region: DRAM_END = 0x40000000 + 2*1024^3 == 0xC0000000 (in hex)).

Next, you must do the math to find out the beginning of the TrustZone space: subtract 32 + 4 = 36 MB = 36 * 1024^2 from the end of the RAM!

Finally, we split the TEE space in two: first 32MB are reserved as secure memory (OP-TEE uses the term TZDRAM to describe this region), the last 4MB are used as shared memory between the TEE and the Secure Monitor (thus, it is called SHMEM).

Write those values out as Makefile variables (use the hexadecimal format for the calculation results, no spaces inside the number since space is the shell argument separator!), e.g.:

TEE_TZDRAM_START = 0xbdc00000
TEE_TZDRAM_SIZE = <32MB as number of bytes, hex is accepted!>
TEE_SHMEM_START = <calc TEE_TZDRAM_START + 32MB, resulting value as hex please>
TEE_SHMEM_SIZE = <4MB in bytes please>
TEE_RAM_TOTAL_SIZE = <36MB as number of bytes>

These will be passed as compilation flags to the OP-TEE + TF-A build scripts (also makefile-based!) and will get inserted as C expressions into their code (thus, you can use valid mathematical expressions, but we recommend against it due to arcane makefile ↔ shell argument passing issues).

(oh, the angular brackets, <…>, are NOT PART OF THE Makefile syntax, they are used as commentary placeholder and should be removed, too!)

Now we're ready to run some compilation commands!

Clone the official OP-TEE OS repository inside your working directory's root (where you have all other projects like U-Boot, ATF/TF-A, Linux kernel etc.). Suggested directory name: optee_os (well… git automatically implies it, since it's also the name of the project).

Now cd into OP-TEE's freshly downloaded source dir and compile it using GNU Make! Here's the official build documentation: https://optee.readthedocs.io/en/latest/building/gits/optee_os.html.

Well, not really… since the official instructions lack many of the required configuration flags (especially for our platform) and figuring them out requires some time-consuming source code studying, here are some pointers:

• set the CROSS_COMPILE as in the labs before!
• use the DEBUG, CFG_TEE_BENCHMARK and CFG_TEE_CORE_LOG_LEVEL values from the official example; set the O=… (output directory) to wherever you like;
• the PLATFORM is imx-mx8mqevk; well, not really, but it's the closest one available, we start from here and override some of its configuration defaults, see next vars ;)
• set the CFG_TZDRAM_START, CFG_TZDRAM_SIZE, CFG_TEE_SHMEM_START, CFG_TEE_SHMEM_SIZE configuration flags to the values calculated above (reminder: you can reference another Makefile variable using $(VARIABLE_NAME) syntax);
• finally, you will also need to pass CFG_DDR_SIZE=0x80000000 (yep, that's 2GB, our board's actual installed memory).

After a successful build, check the output (O) directory's core subdirectory for the tee.bin, tee-raw.bin and many other files! We will keep the raw one for later inclusion into the firmware package using mkimage ;)

Remember the TF-A component (BL31) from Lab 1? We need to re-configure + rebuild it (hopefully, you still got the Makefile script)!

This should be straightforward, since you've done it before ;) You need to set the following additional configuration variables to its make invocation:

• SPD=opteed – this is the Secure Payload Dispatcher module, aka: who does ATF's Secure Monitor need to talk with? Our OP-TEE, of course!
• ATF also has to know the memory region where we've put OP-TEE (set the BL32_BASE variable to lower limit, in hexadecimal); we also need to specify its total size: BL32_SIZE (remember: we allocated 36MB, but give it in bytes, either in base 10 or 16 using C integer notation);
• We want ATF to print some debug messages over the first serial peripheral so, finally, set the LOG_LEVEL=40 and IMX_BOOT_UART_BASE=0x30860000 (if you look in iMX8MQ's Memory Map (the Reference Manual), this is the physical address of our Universal Asynchronous Transmitter/Receiver - i.e. our serial communication module!).

Before we can take a look at the fruits of our effort so far, we need to re-build the firmware package with these last two components.

This is easy if you saved your mkimage script (in your Makefile). You will need JUST one additional file copied: tee-raw.bin from optee_os build output directory, core subdirectory (as mentioned in the subtask above!) inside mkimage's build directory, but rename it as tee.bin. The bundled scripts will see that this file exists and add it to the image automatically!

Also make sure to replace bl31.bin with the newly recompiled one from ARM Trusted Firmware-A!

Proceed to load this image over the serial boot protocol using the IMX uuu utility. Check the serial console (the one provided by the board's Micro USB port) for confirmation!

Found header at 0x404089a0
# Now, this is some output from ATF / BL31; never mind the warnings :D
D/TC:0   plat_get_aslr_seed:118 Warning: no ASLR seed
D/TC:0   add_phys_mem:665 VCORE_UNPG_RX_PA type TEE_RAM_RX 0xbdc00000 size 0x000cb000
D/TC:0   add_phys_mem:665 VCORE_UNPG_RW_PA type TEE_RAM_RW 0xbdccb000 size 0x00135000
D/TC:0   add_phys_mem:665 ta_base type TA_RAM 0xbde00000 size 0x01e00000
D/TC:0   add_phys_mem:665 CONSOLE_UART_BASE type IO_NSEC 0x30800000 size 0x00400000
...
# aaaaanddd our main dish, OP-TEE (BL32):
I/TC: OP-TEE version: 3.22.0-12-ga012b9923 (gcc version 13.1.0 (GCC)) #1 Fri Jul 14 19:24:34 UTC 2023 aarch64
I/TC: WARNING: This OP-TEE configuration might be insecure!
I/TC: WARNING: Please check https://optee.readthedocs.io/en/latest/architecture/porting_guidelines.html
I/TC: Primary CPU initializing
D/TC:0 0 boot_init_primary_late:1478 Executing at offset 0 with virtual load address 0xbdc00000
...
I/TC: Primary CPU switching to normal world boot
...
# BL33 (Normal U-Boot loads afterwards, as expected)
U-Boot 2022.04-g1f940d6213 (Jul 14 2023 - 22:22:25 +0300)

CPU:   i.MX8MQ rev2.1 1300 MHz (running at 800 MHz)
CPU:   Industrial temperature grade (-40C to 105C) at 52C
...

In the second part of our lab, it's time to run some Trusted Applications (TAs)!

We will use the official optee_examples as starting point.

Two questions arise: how can one compile a TA? + how to test it on our board?

If we wish to communicate with OP-TEE from Linux, we need to configure its driver (don't worry, we don't need to recompile the kernel, it's included in defconfig').

Of course, the intended way to do that is by modifying the Linux Device Tree. Do it as documented here ;)

make ARCH=... dtbs

We can use our workstation / laptop to cross-compile a trusted application!

Read the official instructions here.

Remember the secure boot process? The Secure OS (OP-TEE) will also verify each TA before being able to load it!

Fortunately, OP-TEE comes with a predefined key pair, used to facilitate development / testing (warning: highly insecure since anybody can retrieve that key from the source repository!).

Read on for the TA signing procedure!

Even better: the TA Makefile automatically signs the .ta using the development key. So we won't need to do anything more! Yay…

Our rootfs image already contains the OP-TEE client library.

You just need to copy to the cross-compiled binaries to your boot partition.

For this, put u-boot in USB Mass Storage mode:

u-boot=> ums mmc 0

Do not Ctrl+C yet, leave it running and mount the newly appeared USB device in your PC/VM!

After copying the files, boot the Linux package (you can use the Lab02 boot commands, see Readme.md).

Mount the boot partition and run the TA (you might need to copy it somewhere else and chmod +x)! Then execute the program ;)