Skip to content

Commit

Permalink
[Hexagon] Add hexagon launcher to apps and add to TVM's build system (a…
Browse files Browse the repository at this point in the history
…pache#9220)

* Add USE_HEXAGON_LAUNCHER cmake configuration to build the android
hexagon launcher along with a standard TVM build.

* Update hexagon launcher README.md to include
instructions on how to build the launcher
alongside TVM.

* Move Hexagon launcher into top-level apps directory.

* Refactor hexagon launcher cmake directory
structure and group common code into
cmake/HexagonLauncher.cmake.

* Address CRs from @kparzysz-quic.
  • Loading branch information
csullivan authored and ylc committed Jan 13, 2022
1 parent 4f926f5 commit 53f485d
Show file tree
Hide file tree
Showing 15 changed files with 339 additions and 170 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,7 @@
## Compilation

The launcher consists of two parts: part running on Hexagon, and part running
on Android. They need to be compiled separately. Since some source files are
shared between these two parts, make sure to delete all object files between
compilations. Compile the Hexagon code first.
on Android. Each component must be compiled separately.

The supported Snapdragon architectures are 855, 865, and 888.

Expand All @@ -33,7 +31,46 @@ The supported Snapdragon architectures are 855, 865, and 888.
Android NDK can be downloaded from https://developer.android.com/ndk.
Hexagon SDK is available at //developer.qualcomm.com/software/hexagon-dsp-sdk.

### Compilation of the Hexagon part
### Compilation with TVM

Building the Hexagon launcher application as a component of the main TVM build
used for Hexagon codegen can be achieved by setting `USE_HEXAGON_LAUNCHER=ON`.
This option will compile core tvm, the android launcher binary and its corresponding
tvm_runtime, as well as the Hexagon launcher shared library and its corresponding
tvm_runtime. As described in the [Manual compilation](#Manual compilation) section
each component requires Hexagon and android dependencies. When building the launcher
along with TVM these configurations must be providing when invoking cmake. A minimal
example invocation for compiling TVM along with the Hexagon launcher is included below,

```
cmake -DCMAKE_MAKE_PROGRAM=make \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DCMAKE_CXX_FLAGS='-stdlib=libc++' \
-DCMAKE_CXX_STANDARD=14 \
-DUSE_LLVM=/path/to/hexagon/llvm/bin/llvm-config \
-DUSE_HEXAGON_LAUNCHER=ON \
-DUSE_ANDROID_TOOLCHAIN=/path/to/android-ndk/build/cmake/android.toolchain.cmake \
-DANDROID_PLATFORM=android-28 \
-DANDROID_ABI=arm64-v8a \
-DUSE_HEXAGON_ARCH=v68 \
-DUSE_HEXAGON_SDK=/path/to/hexagon/SDK \
-DUSE_HEXAGON_TOOLCHAIN=/path/to/hexagon/Toolchain/ ..
```

The Hexagon launcher application is an android binary and thus requires the use
of an android toolchain for compilation. Similarly, the Hexagon tvm runtime
requires the use of the Hexagon toolchain and depends on the Hexagon SDK. The
resulting hexagon launcher binaries can be found in the `launcher` subdirectory
of the cmake build directory.

### Manual compilation

Since some source files are shared between the Hexagon and android builds,
make sure to delete all object files between compilations. Compile the Hexagon
code first.

#### Compilation of the Hexagon part

1. Build the static version of TVM runtime for Hexagon. Use Hexagon clang
from the Hexagon SDK. This step is the same as building the shared version,
Expand All @@ -55,7 +92,7 @@ Hexagon SDK is available at //developer.qualcomm.com/software/hexagon-dsp-sdk.

3. Run `make`. This will create `liblauncher_rpc_skel.so`.

### Compilation of the Android part
#### Compilation of the Android part

1. Build TVM runtime for Android, using clang for AArch64 from the Android
NDK. Unlike in the Hexagon case, this should be the dynamic library (which
Expand Down
61 changes: 61 additions & 0 deletions apps/hexagon_launcher/cmake/HexagonLauncher.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

if(NOT "${FASTRPC_LIBS}" STREQUAL "SKEL" AND
NOT "${FASTRPC_LIBS}" STREQUAL "STUB")
message(SEND_ERROR "Please set FASTRPC_LIBS to either SKEL or STUB")
endif()

if(NOT DEFINED USE_HEXAGON_SDK)
message(SEND_ERROR "Please set USE_HEXAGON_SDK to the location of Hexagon SDK")
endif()
if (NOT DEFINED USE_HEXAGON_ARCH)
message(SEND_ERROR "Please set USE_HEXAGON_ARCH to the Hexagon architecture version")
endif()

set(TVM_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../../../../")

include(ExternalProject)
include("${TVM_SOURCE_DIR}/cmake/modules/HexagonSDK.cmake")

find_hexagon_sdk_root("${USE_HEXAGON_SDK}" "${USE_HEXAGON_ARCH}")

include_directories(SYSTEM ${HEXAGON_SDK_INCLUDES} ${HEXAGON_REMOTE_ROOT})

set(QAIC_EXE "${HEXAGON_QAIC_EXE}")
foreach(INCDIR IN LISTS HEXAGON_SDK_INCLUDES HEXAGON_REMOTE_ROOT)
list(APPEND QAIC_FLAGS "-I${INCDIR}")
endforeach()

set(LAUNCHER_SRC "${CMAKE_CURRENT_SOURCE_DIR}/../../")
set(CMAKE_SKIP_RPATH TRUE)

# Qaic for the domain header.
#
# Don't add paths to these filenames, or otherwise cmake may spontaneously
# add -o option to the qaic invocation (with an undesirable path).
set(LAUNCHER_RPC_IDL "launcher_rpc.idl")
set(LAUNCHER_RPC_H "launcher_rpc.h")
set(LAUNCHER_RPC_SKEL_C "launcher_rpc_skel.c")
set(LAUNCHER_RPC_STUB_C "launcher_rpc_stub.c")

include_directories(
"${LAUNCHER_SRC}"
"${TVM_SOURCE_DIR}/include"
"${TVM_SOURCE_DIR}/3rdparty/dlpack/include"
"${TVM_SOURCE_DIR}/3rdparty/dmlc-core/include"
)
77 changes: 77 additions & 0 deletions apps/hexagon_launcher/cmake/android/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

cmake_minimum_required(VERSION 3.2)
project(HexagonAndroidLauncher C CXX)

include("${CMAKE_CURRENT_SOURCE_DIR}/../HexagonLauncher.cmake")

add_custom_command(
OUTPUT ${LAUNCHER_RPC_STUB_C}
COMMAND ${QAIC_EXE} ${QAIC_FLAGS}
"${LAUNCHER_SRC}/${LAUNCHER_RPC_IDL}"
COMMAND ${CMAKE_COMMAND} -E rename "${LAUNCHER_RPC_H}"
"${LAUNCHER_SRC}/${LAUNCHER_RPC_H}"
MAIN_DEPENDENCY "${LAUNCHER_SRC}/${LAUNCHER_RPC_IDL}"
)

include_directories(SYSTEM
"${HEXAGON_SDK_INCLUDES}"
"${HEXAGON_RPCMEM_ROOT}/inc"
)

link_directories(${HEXAGON_REMOTE_ROOT})

add_definitions(-DDMLC_USE_LOGGING_LIBRARY=<tvm/runtime/logging.h>)

set(STUB_SRCS
"${LAUNCHER_SRC}/launcher_android.cc"
"${LAUNCHER_SRC}/launcher_core.cc"
"${LAUNCHER_SRC}/launcher_main.cc"
"${LAUNCHER_SRC}/launcher_util.cc"
)

add_executable(launcher_android
"${STUB_SRCS}"
"${LAUNCHER_RPC_STUB_C}"
)

ExternalProject_Add(android_tvm_runtime
SOURCE_DIR "${TVM_SOURCE_DIR}"
BUILD_COMMAND $(MAKE) runtime
CMAKE_ARGS
"-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE}"
"-DANDROID_PLATFORM=${ANDROID_PLATFORM}"
"-DANDROID_ABI=${ANDROID_ABI}"
"-DCMAKE_CXX_STANDARD=14"
"-DUSE_LIBBACKTRACE=OFF"
"-DUSE_LLVM=OFF"
"-DUSE_RPC=OFF"
INSTALL_COMMAND ""
BUILD_ALWAYS ON
)
ExternalProject_Get_Property(android_tvm_runtime BINARY_DIR)
ExternalProject_Add_Step(android_tvm_runtime copy_binaries
COMMAND ${CMAKE_COMMAND} -E copy_directory ${BINARY_DIR} ${CMAKE_INSTALL_PREFIX}
DEPENDEES install
)

add_dependencies(launcher_android android_tvm_runtime)
add_library(tvm_runtime SHARED IMPORTED)
set_target_properties(tvm_runtime PROPERTIES IMPORTED_LOCATION "${BINARY_DIR}/libtvm_runtime.so")

target_link_libraries(launcher_android cdsprpc log tvm_runtime)
84 changes: 84 additions & 0 deletions apps/hexagon_launcher/cmake/hexagon/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

cmake_minimum_required(VERSION 3.2)
project(HexagonLauncherRPCSkel C CXX)

include("${CMAKE_CURRENT_SOURCE_DIR}/../HexagonLauncher.cmake")

add_custom_command(
OUTPUT ${LAUNCHER_RPC_SKEL_C} ${LAUNCHER_RPC_H}
COMMAND ${QAIC_EXE} ${QAIC_FLAGS}
"${LAUNCHER_SRC}/${LAUNCHER_RPC_IDL}"
MAIN_DEPENDENCY "${LAUNCHER_SRC}/${LAUNCHER_RPC_IDL}"
)

include_directories(SYSTEM ${HEXAGON_QURT_INCLUDES})

link_directories(${HEXAGON_QURT_LIBS})

add_definitions(-D_MACH_I32=int)
add_definitions(-DDMLC_CXX11_THREAD_LOCAL=0)
add_definitions(-DDMLC_USE_LOGGING_LIBRARY=<tvm/runtime/logging.h>)

# Extra compile flags (both C and C++).
set(EXTRA_COMP_FLAGS
"-O3"
"-m${USE_HEXAGON_ARCH}"
)
string(REGEX REPLACE ";" " " EXTRA_COMP_FLAGS_STR "${EXTRA_COMP_FLAGS}")
set(CMAKE_C_FLAGS "${EXTRA_COMP_FLAGS_STR} ${CMAKE_C_FLAGS}")
set(CMAKE_CXX_FLAGS "${EXTRA_COMP_FLAGS_STR} ${CMAKE_CXX_FLAGS}")

set(SKEL_SRCS
"${LAUNCHER_SRC}/launcher_core.cc"
"${LAUNCHER_SRC}/launcher_hexagon.cc"
)
add_library(launcher_rpc_skel SHARED
"${LAUNCHER_SRC}/${LAUNCHER_RPC_H}"
"${LAUNCHER_RPC_SKEL_C}"
"${SKEL_SRCS}"
)

ExternalProject_Add(static_hexagon_tvm_runtime
SOURCE_DIR "${TVM_SOURCE_DIR}"
BUILD_COMMAND $(MAKE) runtime
CMAKE_ARGS
"-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}"
"-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}"
"-DUSE_HEXAGON_ARCH=${USE_HEXAGON_ARCH}"
"-DCMAKE_CXX_STANDARD=14"
"-DUSE_LIBBACKTRACE=OFF"
"-DUSE_LLVM=OFF"
"-DUSE_RPC=OFF"
"-DBUILD_STATIC_RUNTIME=ON"
"-DUSE_HEXAGON_SDK=${USE_HEXAGON_SDK}"
INSTALL_COMMAND ""
BUILD_ALWAYS ON
)
ExternalProject_Get_Property(static_hexagon_tvm_runtime BINARY_DIR)
ExternalProject_Add_Step(static_hexagon_tvm_runtime copy_binaries
COMMAND ${CMAKE_COMMAND} -E copy_directory ${BINARY_DIR} ${CMAKE_INSTALL_PREFIX}
DEPENDEES install
)

add_dependencies(launcher_rpc_skel static_hexagon_tvm_runtime)
add_library(static_tvm_runtime STATIC IMPORTED)
set_target_properties(static_tvm_runtime PROPERTIES IMPORTED_LOCATION "${BINARY_DIR}/libtvm_runtime.a")

target_link_libraries(launcher_rpc_skel -Wl,--whole-archive static_tvm_runtime -Wl,--no-whole-archive)

File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
3 changes: 3 additions & 0 deletions cmake/config.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,9 @@ set(USE_FALLBACK_STL_MAP OFF)
set(USE_HEXAGON_DEVICE OFF)
set(USE_HEXAGON_SDK /path/to/sdk)

# Whether to build the hexagon launcher
set(USE_HEXAGON_LAUNCHER OFF)

# Hexagon architecture to target when compiling TVM itself (not the target for
# compiling _by_ TVM). This applies to components like the TVM runtime, but is
# also used to select correct include/library paths from the Hexagon SDK when
Expand Down
81 changes: 72 additions & 9 deletions cmake/modules/Hexagon.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -53,23 +53,86 @@ if(BUILD_FOR_HEXAGON)
include_directories(SYSTEM ${HEXAGON_SDK_INCLUDES} ${HEXAGON_QURT_INCLUDES})
endif()

if(USE_HEXAGON_DEVICE STREQUAL "OFF")
list(APPEND COMPILER_SRCS src/target/opt/build_hexagon_off.cc)
return()
elseif(NOT USE_HEXAGON_DEVICE STREQUAL "${PICK_SIM}" AND
NOT USE_HEXAGON_DEVICE STREQUAL "${PICK_HW}")
set(ERROR_MSG
if(USE_HEXAGON_LAUNCHER STREQUAL "ON")
set(USE_HEXAGON_DEVICE "${PICK_SIM}")
else()
if(USE_HEXAGON_DEVICE STREQUAL "OFF")
list(APPEND COMPILER_SRCS src/target/opt/build_hexagon_off.cc)
return()
elseif(NOT USE_HEXAGON_DEVICE STREQUAL "${PICK_SIM}" AND
NOT USE_HEXAGON_DEVICE STREQUAL "${PICK_HW}")
set(ERROR_MSG
"USE_HEXAGON_DEVICE must be one of [${PICK_NONE}|${PICK_SIM}|${PICK_HW}]")
message(SEND_ERROR "${ERROR_MSG}")
return()
message(SEND_ERROR "${ERROR_MSG}")
return()
endif()
endif()
# If USE_HEXAGON_DEVICE is set to a valid value, make sure that USE_HEXAGON_SDK

# If USE_HEXAGON_DEVICE/LAUNCHER is set to a valid value, make sure that USE_HEXAGON_SDK
# is defined.
if(NOT USE_HEXAGON_SDK)
message(SEND_ERROR "Please set USE_HEXAGON_SDK to the Hexagon SDK root")
return()
endif()

if(USE_HEXAGON_LAUNCHER STREQUAL "ON")

if(DEFINED USE_ANDROID_TOOLCHAIN)
if(NOT DEFINED ANDROID_PLATFORM)
message(SEND_ERROR "Please set ANDROID_PLATFORM "
"when providing an Android cmake toolchain.")
endif()
if(NOT DEFINED ANDROID_ABI)
message(SEND_ERROR "Please set ANDROID_ABI "
"when providing an Android cmake toolchain.")
endif()
else()
message(SEND_ERROR "Please set USE_ANDROID_TOOLCHAIN to build the android "
" launcher for hexagon.")
endif()

set(LAUNCHER_BINARY_DIR "${CMAKE_BINARY_DIR}/launcher")
ExternalProject_Add(launcher_android
SOURCE_DIR "${CMAKE_SOURCE_DIR}/apps/hexagon_launcher/cmake/android"
INSTALL_DIR "${LAUNCHER_BINARY_DIR}"
BUILD_ALWAYS ON
CMAKE_ARGS
"-DCMAKE_TOOLCHAIN_FILE=${USE_ANDROID_TOOLCHAIN}"
"-DANDROID_PLATFORM=${ANDROID_PLATFORM}"
"-DANDROID_ABI=${ANDROID_ABI}"
"-DFASTRPC_LIBS=STUB"
"-DUSE_HEXAGON_ARCH=v68"
"-DUSE_HEXAGON_SDK=${USE_HEXAGON_SDK}"
"-DCMAKE_INSTALL_PREFIX:PATH=<INSTALL_DIR>"
INSTALL_COMMAND ""
)
ExternalProject_Get_Property(launcher_android BINARY_DIR)
ExternalProject_Add_Step(launcher_android copy_binaries
COMMAND ${CMAKE_COMMAND} -E copy_directory ${BINARY_DIR} ${LAUNCHER_BINARY_DIR}
DEPENDEES install
)
ExternalProject_Add(launcher_hexagon
SOURCE_DIR "${CMAKE_SOURCE_DIR}/apps/hexagon_launcher/cmake/hexagon"
INSTALL_DIR "${LAUNCHER_BINARY_DIR}"
BUILD_ALWAYS ON
CMAKE_ARGS
"-DCMAKE_C_COMPILER=${USE_HEXAGON_TOOLCHAIN}/Tools/bin/hexagon-clang"
"-DCMAKE_CXX_COMPILER=${USE_HEXAGON_TOOLCHAIN}/Tools/bin/hexagon-clang++"
"-DFASTRPC_LIBS=SKEL"
"-DUSE_HEXAGON_ARCH=v68"
"-DUSE_HEXAGON_SDK=${USE_HEXAGON_SDK}"
"-DCMAKE_INSTALL_PREFIX:PATH=<INSTALL_DIR>"
INSTALL_COMMAND ""
)
ExternalProject_Get_Property(launcher_hexagon BINARY_DIR)
ExternalProject_Add_Step(launcher_hexagon copy_binaries
COMMAND ${CMAKE_COMMAND} -E copy_directory ${BINARY_DIR} ${LAUNCHER_BINARY_DIR}
DEPENDEES install
)

set_directory_properties(PROPERTIES ADDITIONAL_MAKE_CLEAN_FILES "${LAUNCHER_BINARY_DIR}")
endif()

if(USE_HEXAGON_DEVICE STREQUAL "${PICK_SIM}")
find_hexagon_toolchain()
message(STATUS "Hexagon toolchain: ${HEXAGON_TOOLCHAIN}")
Expand Down
Loading

0 comments on commit 53f485d

Please sign in to comment.