Before beginning the tasks, please check out the lecture slides & notes here.
These tasks assume you have a proper boot image + base Linux system working (U-Boot SPL + ARM TrustedFirmware-A + Normal U-Boot, Linux Kernel + Buildroot filesystem).
We will continue writing the build scripts (Makefile-based) to augment our firmware boot package with a freshly compiled OP-TEE (BL32), modify ATF / TF-A (BL31) to load OP-TEE and, finally, write our own Trusted Applications and deploy them on our boards (with the help of our previously bootstrapped Linux, ofc.)!
The initial step would be to embed OP-TEE OS into our firmware boot package and re-configure (and recompile) ARM Trusted Firmware-A to load it.
But first, we need to understand the memory organization of our board. This is required since BL31 (TF-A) has to configure the TZASC peripheral to reserve a trusted region inside the main memory (DRAM) for use by our Secure OS & TAs.
In the figure to the right, we use the upper end of the DDR1 memory (36 MB) for the TrustZone OS (BL32). We will need to calculate the addresses and supply them as configuration parameters when compiling the software.
In order to get the actual physical addresses, one needs to check out the official NXP iMX93 Reference Manual's Memory Map.
We note that the first DRAM (DDR1) is mapped at 0x80000000
(exactly 1GB from the beginning the address space) and ends at 0xFFFFFFFF
(the difference of the two addresses totals 2GB
).
Next, you must do the math to find out the beginning of the TrustZone space: subtract 32 + 4 = 36 MB = 36 * 1024^2
from the end of the RAM!
Finally, we split the TEE space in two: first 32MB
are reserved as secure memory (OP-TEE uses the term TZDRAM to describe this region), the last 4MB
are used as shared memory between the TEE and the Secure Monitor (thus, it is called SHMEM).
Write those values out as Makefile variables (use the hexadecimal format for the calculation results, no spaces inside the number since space is the shell argument separator!), e.g.:
TEE_TZDRAM_START = 0xfdc00000 TEE_TZDRAM_SIZE = <32MB as number of bytes, hex is accepted!> TEE_SHMEM_START = <calc TEE_TZDRAM_START + 32MB, resulting value as hex please> TEE_SHMEM_SIZE = <4MB in bytes please> TEE_RAM_TOTAL_SIZE = <36MB as number of bytes>
These will be passed as compilation flags to the OP-TEE + TF-A build scripts (also makefile-based!) and will get inserted as C expressions into their code (thus, you can use valid mathematical expressions, but we recommend against it due to arcane makefile ↔ shell argument passing issues).
(oh, the angular brackets, <…>
, are NOT PART OF THE Makefile syntax, they are used as commentary placeholder and should be removed, too!)
Now we're ready to run some compilation commands!
Clone the NXP fork imx-optee-os repository inside your working directory's root (where you have all other projects like U-Boot, ATF/TF-A, Linux kernel etc.).
Now cd
into OP-TEE's freshly downloaded source dir and compile it using GNU Make!
Here's the official build documentation: https://optee.readthedocs.io/en/latest/building/gits/optee_os.html.
Well, not really… since the official instructions lack many of the required configuration flags (especially for our platform) and figuring them out requires some time-consuming source code studying, here are some pointers:
CROSS_COMPILE
as in the labs before!DEBUG
, CFG_TEE_BENCHMARK
and CFG_TEE_CORE_LOG_LEVEL
values from the official example; set the O=…
(output directory) to wherever you like;PLATFORM
can be found exclusively inside the fork's source code! but there's a catch: the platform prefix must be imx-
, continued by model name without the leading i
(check some examples here).CFG_TZDRAM_START
, CFG_TZDRAM_SIZE
, CFG_TEE_SHMEM_START
, CFG_TEE_SHMEM_SIZE
configuration flags to the values calculated above (reminder: you can reference another Makefile variable using $(VARIABLE_NAME)
syntax);CFG_DDR_SIZE=0x80000000
(yep, that's 2GB, our board's actual installed memory).
As workaround, we can disable the ELE RNG driver and enable a software one using the following additional make configuration options: CFG_WITH_SOFTWARE_PRNG=y CFG_IMX_ELE=n
. So make sure to also set them!
After a successful build, check the output (O
) directory's core
subdirectory for the tee.bin
, tee-raw.bin
and many other files!
We will keep the raw one for later inclusion into the firmware package using mkimage
;)
Remember the TF-A component (BL31) from Lab 1? We need to re-configure + rebuild it (hopefully, you still got the Makefile script)!
This should be straightforward, since you've done it before ;) You need to set the following additional configuration variables to its make invocation:
SPD=opteed
– this is the Secure Payload Dispatcher module, aka: who does ATF's Secure Monitor need to talk with? Our OP-TEE, of course!BL32_BASE
variable to lower limit, in hexadecimal); we also need to specify its total size: BL32_SIZE
(remember: we allocated 36MB
, but give it in bytes, either in base 10 or 16 using C integer notation);LOG_LEVEL=40
and IMX_BOOT_UART_BASE=0x44380000
(if you look in iMX93's Memory Map (the Reference Manual), this is the physical address of our LPUART1 - i.e. our serial communication module!).Before we can take a look at the fruits of our effort so far, we need to re-build the firmware package with these last two components.
This is easy if you saved your mkimage script (in your Makefile). You will need JUST one additional file copied: tee-raw.bin
from optee
build output directory, core
subdirectory (as mentioned in the subtask above!) inside mkimage
's build directory, but rename it as tee.bin
.
The bundled scripts will see that this file exists and add it to the image automatically!
TEE_LOAD_ADDR=<TrustZone DRAM start address>
.
The default value is wrong for our SoC.
Also make sure to replace bl31.bin
with the newly recompiled one from ARM Trusted Firmware-A!
Proceed to load this image over the serial boot protocol using the IMX uuu
utility.
Check the serial console (the one provided by the board's DEBUG USB port) for confirmation!
In the second part of our lab, it's time to run some Trusted Applications (TAs)!
We will use the official optee_examples as starting point.
Two questions arise: how can one compile a TA? + how to test it on our board?
If we wish to communicate with OP-TEE from Linux, we need to configure its driver (don't worry, we don't need to recompile the kernel, it's included in defconfig
').
Of course, the intended way to do that is by modifying the Linux Device Tree. Do it as documented here ;)
arch/arm64/
.
We promised that you won't need to recompile the kernel, you just need to recompile its DTB ;) use:
make ARCH=... dtbs
Oh, and re-enable Buildroot if disabled for the previous lab (you backed up your ITS file, hopefully).
Then enter its menuconfig
and search for PACKAGE_OPTEE_CLIENT
. Enable it and rebuild your rootfs!
Afterwards, you need to copy the new .dtb
into your staging/
directory and re-build the Linux FIT (e.g., linux.itb
). And upload it to your emmc (use u-boot's ums
and simply copy the file on the FAT32 boot partition).
We can use our workstation / laptop to cross-compile a trusted application!
Read the official instructions here.
Build both the host app and the Trusted Application need to be compiled TOGETHER with the TEE Client Library, and the OPTEE OS exported SDK, respectively.
Make sure to read the examples documentation to see the make variables to set!
But, since we're using Buildroot and have enabled it, you can find it already compiled for the target system at <buildroot-dir>/output/build/optee-client-<version>
.
Touugh we need to give a TEEC_EXPORT
install path when invoking the TA makefile… Use find for tee_client_api.h
and see where it's found (hint: sysroot
)!
TA_DEV_KIT_DIR
of a Trusted Application, it must point to an SDK generated inside BL32 (optee_os) source directory (something like export-ta…
'.
Also note that for building Buildroot-targeted applications, your classic aarch64-none-gnu-
toolchain won't work since Buildroot uses custom ucLibC by default. But, fortunately, you may find the cross compilation prefix at <buildroot-dir>/output/host/bin/aarch64-buildroot-linux-gnu-
(see the difference?)!
Remember the secure boot process? The Secure OS (OP-TEE) will also verify each TA before being able to load it!
Fortunately, OP-TEE comes with a predefined key pair, used to facilitate development / testing (warning: highly insecure since anybody can retrieve that key from the source repository!).
Read on for the TA signing procedure!
Even better: the TA Makefile automatically signs the .ta using the development key. So we won't need to do anything more! Yay…
Our rootfs image already contains the OP-TEE client library.
You just need to copy to the cross-compiled binaries to your boot partition.
For this, put u-boot in USB Mass Storage mode:
u-boot=> ums mmc 0
Do not Ctrl+C yet, leave it running and mount the newly appeared USB device in your PC/VM!
After copying the files, boot Linux.
Now check if tee-supplicant
is running… let's mount devtmpfs then start it:
mount -t devtmpfs devtmpfs /dev tee-supplicant -d
Mount the boot partition and run the TA (you might need to copy it somewhere else and chmod +x
)!
Then execute the program ;)
.ta
file inside a trusted memory or REE.
For this, you will need to copy the signed <UUID>.ta
file to /lib/optee_armtz/
(create it if it doesn't exist), as (very badly) documented.