Inquiries about TEE memory

[SangsubLim]

I am studying memory in TEE, and I have a few questions.
I'm working on QEMU v8 environment. (CFG_WITH_PAGER=n)

1. CFG_TZDRAM_SIZE.
   Refer to the generic_ram_layout.h file for details.
   Checking TEE RAM layout without CFG_WITH_PAGER, there is TEE core secure RAM (TEE_RAM).
   Does this include both the code area and data area of optee-os?

2. If so, the Trusted Application RAM (TA_RAM) seems to be the area for TA's code and data.
   It is thought that several TAs share and use this area. I wonder how memory is managed when several TAs are running.
   For example, if TA A has allocated a large amount of heap memory, and TA B also requests a large amount of heap, how is it managed?
   I understand that general Linux systems are managed through PAGING and SWAP. In optee, is out of memory returned in the above situation or is there a separate management method?

3. The heap size of optee os is designated as "CFG_CORE_HEAP_SIZE" when building optee-os, and the heap size of TA is designated as property (TA_DATA_SIZE) when building TA.
   Looking at TEE RAM layout without CFG_WITH_PAGER, it seems that there is no CFG_XXXX that distinguishes between TEE core secure RAM (TEE_RAM) and Trusted Application RAM (TA_RAM).
   Is it used as contiguous memory as shown below?

----------------------------------------|----------------- -- <-- CFG_TZDRAM_START
TEE core secure RAM (TEE_RAM)           | optee-os code area
                                        | optee-os data area
----------------------------------------|----------------- --
Trusted Application RAM (TA_RAM)        | TAs code area
                                        | TAs data area
----------------------------------------|-------------------- <-- CFG_TZDRAM_START + CFG_TZDRAM_SIZE

If so, I think increasing the size of "CFG_CORE_HEAP_SIZE" will lower the maximum allowed value of TA_DATA_SIZE. Am I right?
And in the picture above, do other areas other than code and data exist?

4. Can the OPTEE paging system be used only in an environment where SRAM exists?
   I saw in the link below that it was written as a solution for SRAM 128 to 256 KiB environment.
   https://optee.readthedocs.io/en/latest/architecture/core.html#pager

Again, I'm studying on QEMU v8 right now, and I've confirmed that CFG_WITH_PAGER=n is set.

BR,

[etienne-lms]

When CFG_WITH_PAGER is disabled, CFG_TZDRAM_START/_SIZE cover TEE_RAM + TA_RAM.
TEE_RAM is code/data (including heap of size CFG_CORE_HEAP_SIZE).
TA_RAM is memory used for TAs and some OP-TEE core big buffer allocation.

For example, if TA A has allocated a large amount of heap memory, and TA B also requests a large amount of heap, how is it managed?

Each TA declares, when instancied, how much heap it requires (image + TA_DATA_SIZE + TA_STACK_SIZE). That memory is provisioned specifically to that TA and cannot be used/claimed by other TAs or OP-TEE core itslef.

3. (...) Looking at TEE RAM layout without CFG_WITH_PAGER, it seems that there is no CFG_XXXX that distinguishes between TEE core secure RAM (TEE_RAM) and Trusted Application RAM (TA_RAM).

When CFG_WITH_PAGER is disabled, CFG_TZDRAM_START/_SIZE cover TEE_RAM + TA_RAM.

TEE_RAM size is given by CFG_TEE_RAM_VA_SIZE is defined, or falback to a pagedir entry (CORE_MMU_PGDIR_SIZE: e.g. 2MB on armv8-A). TA_RAM size uses the remaining.

See core/arch/arm/include/mm/generic_ram_layout.h.

If so, I think increasing the size of "CFG_CORE_HEAP_SIZE" will lower the maximum allowed value of TA_DATA_SIZE. Am I right?

CFG_CORE_HEAP_SIZE consumes memory in TEE_RAM but does not affect CFG_TEE_RAM_VA_SIZE (see above) that must be set specifically if needed. When CFG_CORE_HEAP_SIZE > CFG_TEE_RAM_VA_SIZE, the configuration will obviosuly not work.

4. Can the OPTEE paging system be used only in an environment where SRAM exists?

CFG_WITH_PAGING can be used with with or without SRAM.
When CFG_WITH_PAGER is enabled, TEE_RAM range is expected to be defined by platform configuration CFG_TZSRAM_START/_SIZE (could be SRAM or DDR), in which case TA_RAM range is fully defined by CFG_TZDRAM_START/_SIZE.

Platforms testing pager (CFG_WITH_PAGER=y) with DDR only (no SRAM) usually don't set CFG_TZSRAM_START/_SIZE.
In this case, the begining of TZDRAM is used for TEE_RAM: CFG_TZDRAM_START/CFG_CORE_TZSRAM_EMUL_SIZE.
The remaining TZDRAM is used for TA_RAM (CFG_TZDRAM_SIZE - CFG_CORE_TZSRAM_EMUL_SIZE).

Note a 128kB SRAM for OP-TEE with pager is quite a challenge. 256kB or more is recommended.

[github-actions]

This issue has been marked as a stale issue because it has been open (more than) 30 days with no activity. Remove the stale label or add a comment, otherwise this issue will automatically be closed in 5 days. Note, that you can always re-open a closed issue at any time.

[TaihuLight]

How to check the available TEE memory size when SBC boot?
@etienne-lms @SangsubLim

[etienne-lms]

When core is built with CFG_WITH_STATS=y, you can use xtest --stats --alloc to get core heap (TEE_RAM heap) and secure DDR (TA_RAM) pools info (value in bytes). See the dump below, from a recent qemu_virt run :

(...)
Welcome to Buildroot, type root or test to login
buildroot login: root
# xtest --stats --alloc
Pool:                Heap
Bytes allocated:                       17704
Max bytes allocated:                   17784
Size of pool:                          77880
Number of failed allocations:          0
Size of larges allocation failure:     0
Total bytes allocated at that failure: 0

Pool:                
Bytes allocated:                       0
Max bytes allocated:                   0
Size of pool:                          0
Number of failed allocations:          0
Size of larges allocation failure:     0
Total bytes allocated at that failure: 0

Pool:                Secure DDR
Bytes allocated:                       253952
Max bytes allocated:                   253952
Size of pool:                          13631488
Number of failed allocations:          0
Size of larges allocation failure:     0
Total bytes allocated at that failure: 0

Pool:                
Bytes allocated:                       0
Max bytes allocated:                   0
Size of pool:                          0
Number of failed allocations:          0
Size of larges allocation failure:     0
Total bytes allocated at that failure: 0

Pool:                
Bytes allocated:                       0
Max bytes allocated:                   0
Size of pool:                          0
Number of failed allocations:          0
Size of larges allocation failure:     0
Total bytes allocated at that failure: 0
#