Skip to content

Latest commit

 

History

History
78 lines (63 loc) · 3.2 KB

README.md

File metadata and controls

78 lines (63 loc) · 3.2 KB
Raw

Info

Arm(R) Ethos(TM)-U is a new class of machine learning processors, called a microNPU, specifically designed to accelerate ML inference in area-constrained embedded and IoT devices. This readme briefly describes how to integrate Ethos-U related hardware and software into TFLM. See also Ethos-U ML Evaluation kit examples.

To enable the Ethos-U software stack, add CO_PROCESSOR=ethos_u to the make command. Use ETHOSU_ARCH to specify the architecture. See examples below.

Requirements:

  • Armclang 6.14 or later
  • GCC 10.2.1 or later

Ethos-U custom operator

The TFLM runtime will dispatch workloads to Ethos-U when it encounters an Ethos-U custom op in the tflite file. See an ASCII art example below. The Ethos-U custom op is added by a tool called Ethos-U Vela and contains information the Ethos-U hardware need to execute the workload. More info in the Vela repository.

     | tensor0
     |
     v
+------------+
| ethos-u    |
| custom op  |
+------------+
     +
     |
     | tensor1
     |
     v
+-----------+
| transpose |
|           |
+----|------+
     |
     | tensor2
     |
     v

Note that the ethousu_init() API of the Ethos-U driver need to be called at startup, before calling the TFLM API. More info in the Ethos-U driver repo.

For even more info regarding Vela and Ethos-U, checkout Ethos-U landing page.

Some examples of compiling a binary and running a network with Ethos-U support.

In order to run a test with Ethos-U55 enabled, a platform with corresponding hardware support is required. One such platform is the fixed virtual platform (FVP) based on Arm Corstone-300 software. See Corstone-300 readme for more info.

On top of that the .tflite model needs to be modified according subchapter "Ethos-U custom operator" above.

The log level of the Ethos-U driver can be set in the build command. For example: ETHOSU_LOG_SEVERITY=ETHOSU_LOG_INFO.

Example using network tester

See tensorflow/lite/micro/examples/network_tester/README.md for more info.

make -f tensorflow/lite/micro/tools/make/Makefile network_tester_test CO_PROCESSOR=ethos_u ETHOSU_ARCH=u55 TARGET=cortex_m_generic TARGET_ARCH=cortex-m55 microlite

For the Arm Corstone-300 target, ETHOSU_ARCH is defined in cortex_m_corstone_300_makefile.inc so it doesn't need to be defined on the command line.

make -f tensorflow/lite/micro/tools/make/Makefile network_tester_test CO_PROCESSOR=ethos_u TARGET=cortex_m_corstone_300 TARGET_ARCH=cortex-m55 test_network_tester_test NETWORK_MODEL=path/to/network_model.h INPUT_DATA=path/to/input_data.h OUTPUT_DATA=path/to/expected_output_data.h

make -f tensorflow/lite/micro/tools/make/Makefile network_tester_test CO_PROCESSOR=ethos_u TARGET=cortex_m_corstone_300 TARGET_ARCH=cortex-m55 test_network_tester_test