Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zephyr Build info file #79118

Merged
merged 6 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2235,3 +2235,10 @@ add_subdirectory_ifdef(
)

toolchain_linker_finalize()

yaml_context(EXISTS NAME build_info result)
if(result)
build_info(zephyr version VALUE ${PROJECT_VERSION_STR})
build_info(zephyr zephyr-base VALUE ${ZEPHYR_BASE})
yaml_save(NAME build_info)
endif()
3 changes: 3 additions & 0 deletions cmake/modules/FindHostTools.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -114,3 +114,6 @@ set_ifndef(TOOLCHAIN_KCONFIG_DIR ${TOOLCHAIN_ROOT}/cmake/toolchain/${ZEPHYR_TOOL

set(HostTools_FOUND TRUE)
set(HOSTTOOLS_FOUND TRUE)
build_info(toolchain name VALUE ${ZEPHYR_TOOLCHAIN_VARIANT})
string(TOUPPER ${ZEPHYR_TOOLCHAIN_VARIANT} zephyr_toolchain_variant_upper)
build_info(toolchain path VALUE "${${zephyr_toolchain_variant_upper}_TOOLCHAIN_PATH}")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Causing old compilation errors as ???_TOOLCHAIN_PATH is invalid.

-- Found toolchain: gnuarmemb (C:\gnu_arm_embedded_10)
CMake Error at C:/Users/BillyBob/zephyrGoodX/zephyrNew/cmake/modules/yaml.cmake:313 (string):
string sub-command JSON failed parsing json string: * Line 1, Column 1

Bad escape sequence in string

See Line 1, Column 6 for detail.

I changed it to:

build_info(toolchain path VALUE "${TOOLCHAIN_ROOT}")
# build_info(toolchain path VALUE "${${zephyr_toolchain_variant_upper}_TOOLCHAIN_PATH}")

Copy link
Member

@fabiobaltieri fabiobaltieri Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @WilliamGFish, could you open a pull request with the fix? This is the contribution guideline documentation. Thanks!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, found out its a Linux / Windows path issue.
I'll raise a 'fix' via a PR.

Copy link
Collaborator Author

@tejlmand tejlmand Oct 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a couple of comments regarding this, first related to the issue itself.

Please note that a CMake path officially is with forward slashes /, and that windows path separator in principle should be escaped.
This info seems to be a bit hidden in various places of the docs, like here:
https://cmake.org/cmake/help/latest/command/cmake_path.html#modification

If <input> is a native path, it is converted into a cmake-style path with forward-slashes (/).
...
...
For example:
set(native_path "c:\\a\\b/..\\c")

or in CMake questions:
https://discourse.cmake.org/t/window-env-variable-and-cmake-path/5410

Paths should normally be written with forward slashes in CMake code, even for Windows. Alternatively, if you want a backslash, you need to escape it, so your string would have to be either "D:/test/fmt/release" or "D:\\test\\fmt\\release".

So the use of a single back-slash in general with CMake is not guaranteed to work.
That said, it seems like the find_program() supports the use of single \. The <toolchain>_TOOLCHAIN_PATH is generally used together with find_program() and therefore working with single \.
Other CMake settings / paths may not work with single \ separator.

Regarding the build_info() / yaml feature, which uses CMake's json support under the hood, then path should of course be properly sanitized in case they contain single \.

Note that TOOLCHAIN_ROOT != <toolchain>_TOOLCHAIN_PATH.
One is the toolchain itself, the other points to the location of the toolchain CMake integration code.
Those two variables points to very different things.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed here: #79647

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right they are two different things just needed stop compile failures. I have addressed this by replacing the \ with / in the CMake script like this:

string(REGEX REPLACE "\\\\" "/" toolchain_path "${${zephyr_toolchain_variant_upper}_TOOLCHAIN_PATH}")
build_info(toolchain path VALUE "${toolchain_path}")

As on windows machines they will continue to use backslashes in config files the will need to be replaced.

1 change: 1 addition & 0 deletions cmake/modules/FindZephyr-sdk.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -175,3 +175,4 @@ if(LOAD IN_LIST Zephyr-sdk_FIND_COMPONENTS)
endif()
endif()
endif()
set(ZEPHYR_TOOLCHAIN_PATH ${ZEPHYR_SDK_INSTALL_DIR})
4 changes: 4 additions & 0 deletions cmake/modules/boards.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -369,3 +369,7 @@ if(BOARD_EXTENSIONS)
list(APPEND BOARD_EXTENSION_DIRS ${board_extension_dir})
endforeach()
endif()
build_info(board name VALUE ${BOARD})
string(REGEX REPLACE "^/" "" qualifiers "${BOARD_QUALIFIERS}")
build_info(board qualifiers VALUE ${qualifiers})
build_info(board revision VALUE ${BOARD_REVISION})
2 changes: 2 additions & 0 deletions cmake/modules/configuration_files.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -99,3 +99,5 @@ zephyr_boilerplate_watch(DTC_OVERLAY_FILE)
zephyr_get(EXTRA_CONF_FILE SYSBUILD LOCAL VAR EXTRA_CONF_FILE OVERLAY_CONFIG MERGE REVERSE)
zephyr_get(EXTRA_DTC_OVERLAY_FILE SYSBUILD LOCAL MERGE REVERSE)
zephyr_get(DTS_EXTRA_CPPFLAGS SYSBUILD LOCAL MERGE REVERSE)
build_info(application source-dir VALUE ${APPLICATION_SOURCE_DIR})
build_info(application configuration-dir VALUE ${APPLICATION_CONFIG_DIR})
6 changes: 6 additions & 0 deletions cmake/modules/dts.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,7 @@ set(dts_files
if(DTC_OVERLAY_FILE)
zephyr_list(TRANSFORM DTC_OVERLAY_FILE NORMALIZE_PATHS
OUTPUT_VARIABLE DTC_OVERLAY_FILE_AS_LIST)
build_info(devicetree user-files VALUE ${DTC_OVERLAY_FILE_AS_LIST})
list(APPEND
dts_files
${DTC_OVERLAY_FILE_AS_LIST}
Expand All @@ -197,6 +198,7 @@ endif()
if(EXTRA_DTC_OVERLAY_FILE)
zephyr_list(TRANSFORM EXTRA_DTC_OVERLAY_FILE NORMALIZE_PATHS
OUTPUT_VARIABLE EXTRA_DTC_OVERLAY_FILE_AS_LIST)
build_info(devicetree extra-user-files VALUE ${EXTRA_DTC_OVERLAY_FILE_AS_LIST})
list(APPEND
dts_files
${EXTRA_DTC_OVERLAY_FILE_AS_LIST}
Expand Down Expand Up @@ -413,3 +415,7 @@ elseif(stderr)
message(WARNING "dtc raised one or more warnings:\n${stderr}")
endif()
endif(DTC)

build_info(devicetree files VALUE ${dts_files})
build_info(devicetree include-dirs VALUE ${DTS_ROOT_SYSTEM_INCLUDE_DIRS})
build_info(devicetree bindings-dirs VALUE ${DTS_ROOT_BINDINGS})
106 changes: 106 additions & 0 deletions cmake/modules/extensions.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ include(CheckCXXCompilerFlag)
# 7.1 llext_* configuration functions
# 7.2 add_llext_* build control functions
# 7.3 llext helper functions
# 8. Script mode handling

########################################################
# 1. Zephyr-aware extensions
Expand Down Expand Up @@ -3656,6 +3657,67 @@ function(topological_sort)
set(${TS_RESULT} "${sorted_targets}" PARENT_SCOPE)
endfunction()

# Usage:
# build_info(<tag>... VALUE <value>...)
#
# This function populates updates the build_info.yml info file with exchangable build information
# related to the current build.
#
# Example:
# build_info(devicetree files VALUE file1.dts file2.dts file3.dts)
# Will update the 'devicetree files' key in the build info yaml with the list
# of files, file1.dts file2.dts file3.dts.
#
# build_info(vendor-specific foo VALUE bar)
# Will place the vendor specific key 'foo' with value 'bar' in the vendor specific section
# of the build info file.
#
# <tag>...: One of the pre-defined valid CMake keys supported by build info or vendor-specific.
# See 'scripts/schemas/build-schema.yml' CMake section for valid tags.
# VALUE <value>... : value(s) to place in the build_info.yml file.
function(build_info)
set(arg_list ${ARGV})
list(FIND arg_list VALUE index)
if(index EQUAL -1)
message(FATAL_ERROR "${CMAKE_CURRENT_FUNCTION}(...) missing a required argument: VALUE")
endif()

yaml_context(EXISTS NAME build_info result)
if(NOT result)
yaml_load(FILE ${ZEPHYR_BASE}/scripts/schemas/build-schema.yml NAME build_info_schema)
if(EXISTS ${CMAKE_BINARY_DIR}/build_info.yml)
yaml_load(FILE ${CMAKE_BINARY_DIR}/build_info.yml NAME build_info)
else()
yaml_create(FILE ${CMAKE_BINARY_DIR}/build_info.yml NAME build_info)
endif()
yaml_set(NAME build_info KEY version VALUE "0.1.0")
endif()

list(SUBLIST arg_list 0 ${index} keys)
list(SUBLIST arg_list ${index} -1 values)
list(POP_FRONT values)

if(ARGV0 STREQUAL "vendor-specific")
set(type VALUE)
else()
set(schema_check ${keys})
list(TRANSFORM schema_check PREPEND "mapping;")
yaml_get(check NAME build_info_schema KEY mapping cmake ${schema_check})
if(check MATCHES ".*-NOTFOUND")
message(FATAL_ERROR "${CMAKE_CURRENT_FUNCTION}(...) called with invalid tag: ${keys}")
endif()

yaml_get(type NAME build_info_schema KEY mapping cmake ${schema_check} type)
if(type MATCHES "seq|sequence")
set(type LIST)
else()
set(type VALUE)
endif()
endif()

yaml_set(NAME build_info KEY cmake ${keys} ${type} "${values}")
endfunction()

########################################################
# 4. Devicetree extensions
########################################################
Expand Down Expand Up @@ -5759,3 +5821,47 @@ function(llext_filter_zephyr_flags filter flags outvar)

set(${outvar} ${zephyr_filtered_flags} PARENT_SCOPE)
endfunction()

########################################################
# 8. Script mode handling
########################################################
#
# Certain features are not available when CMake is used in script mode.
# For example custom targets, and thus features related to custom targets, such
# as target properties are not available in script mode.
#
# This section defines behavior for functions whose default implementation does
# not work correctly in script mode.
#
# The script mode function can be a simple stub or a more complex solution
# depending on the exact use of the function in script mode.
#
# Current Zephyr CMake scripts which includes `extensions.cmake` in script mode
# are: package_helper.cmake, verify-toolchain.cmake
#

if(CMAKE_SCRIPT_MODE_FILE)
# add_custom_target and set_target_properties are not supported in script mode.
# However, Zephyr CMake functions like `zephyr_get()`, `zephyr_create_scope()`,
# llext functions creates or relies on custom CMake targets.
function(add_custom_target)
# This silence the error: 'add_custom_target command is not scriptable'
endfunction()

function(set_target_properties)
# This silence the error: 'set_target_properties command is not scriptable'
endfunction()

function(zephyr_set variable)
# This silence the error: zephyr_set(... SCOPE <scope>) doesn't exists.
endfunction()

# Build info creates a custom target for handling of build info.
# build_info is not needed in script mode but still called by Zephyr CMake
# modules. Therefore disable build_info(...) in when including
# extensions.cmake in script mode.
function(build_info)
# This silence the error: 'YAML context 'build_info' does not exist.'
# 'Remember to create a YAML context'
endfunction()
endif()
3 changes: 3 additions & 0 deletions cmake/modules/kconfig.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -95,11 +95,13 @@ set(PARSED_KCONFIG_SOURCES_TXT ${PROJECT_BINARY_DIR}/kconfig/sources.txt)
if(CONF_FILE)
string(CONFIGURE "${CONF_FILE}" CONF_FILE_EXPANDED)
string(REPLACE " " ";" CONF_FILE_AS_LIST "${CONF_FILE_EXPANDED}")
build_info(kconfig user-files VALUE ${CONF_FILE_AS_LIST})
endif()

if(EXTRA_CONF_FILE)
string(CONFIGURE "${EXTRA_CONF_FILE}" EXTRA_CONF_FILE_EXPANDED)
string(REPLACE " " ";" EXTRA_CONF_FILE_AS_LIST "${EXTRA_CONF_FILE_EXPANDED}")
build_info(kconfig extra-user-files VALUE ${EXTRA_CONF_FILE_AS_LIST})
endif()

zephyr_file(CONF_FILES ${BOARD_EXTENSION_DIRS} KCONF board_extension_conf_files SUFFIX ${FILE_SUFFIX})
Expand Down Expand Up @@ -354,6 +356,7 @@ endif()
if(CREATE_NEW_DOTCONFIG)
set(input_configs_flags --handwritten-input-configs)
set(input_configs ${merge_config_files} ${FORCED_CONF_FILE})
build_info(kconfig files VALUE ${input_configs})
else()
set(input_configs ${DOTCONFIG} ${FORCED_CONF_FILE})
endif()
Expand Down
1 change: 1 addition & 0 deletions cmake/modules/unittest.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ cmake_minimum_required(VERSION 3.20.0)

include(extensions)
include(west)
include(yaml)
include(root)
include(zephyr_module)
include(boards)
Expand Down
25 changes: 0 additions & 25 deletions cmake/package_helper.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -44,20 +44,6 @@

cmake_minimum_required(VERSION 3.20.5)

# add_custom_target and set_target_properties are not supported in script mode.
# However, several Zephyr CMake modules create custom target for user convenience
# like menuconfig, boards, shields, etc.
# As we are not generating a build system with this tool, only running part of
# the modules, then we simply override those functions to allow running those
# modules.
function(add_custom_target)
# This silence the error: 'add_custom_target command is not scriptable'
endfunction()

function(set_target_properties)
# This silence the error: 'set_target_properties command is not scriptable'
endfunction()

# Find last `-B` and `-S` instances.
foreach(i RANGE ${CMAKE_ARGC})
if(CMAKE_ARGV${i} MATCHES "^-B(.*)")
Expand Down Expand Up @@ -111,16 +97,5 @@ if(NOT DEFINED MODULES)
)
endif()

# Loading Zephyr CMake extension commands, which allows us to overload Zephyr
# scoping rules.
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE} COMPONENTS extensions)

# Zephyr scoping creates custom targets for handling of properties.
# However, custom targets cannot be used in CMake script mode.
# Therefore disable zephyr_set(... SCOPE ...) in package helper as it is not needed.
function(zephyr_set variable)
# This silence the error: zephyr_set(... SCOPE <scope>) doesn't exists.
endfunction()

string(REPLACE ";" "," MODULES "${MODULES}")
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE} COMPONENTS zephyr_default:${MODULES})
1 change: 1 addition & 0 deletions modules/hal_nordic/nrfx/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -229,3 +229,4 @@ mdk_svd_ifdef(CONFIG_SOC_NRF9160 nrf9160.svd)
mdk_svd_ifdef(CONFIG_SOC_NRF9230_ENGB_CPUAPP nrf9230_engb_application.svd)
mdk_svd_ifdef(CONFIG_SOC_NRF9230_ENGB_CPUPPR nrf9230_engb_ppr.svd)
mdk_svd_ifdef(CONFIG_SOC_NRF9230_ENGB_CPURAD nrf9230_engb_radiocore.svd)
build_info(vendor-specific nordic svdfile VALUE ${SOC_SVD_FILE})
108 changes: 108 additions & 0 deletions scripts/schemas/build-schema.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# SPDX-License-Identifier: Apache-2.0
#
# Copyright (c) 2024, Nordic Semiconductor ASA

# A pykwalify schema for basic validation of the Zephyr build info YAML file.

type: map
mapping:
version:
required: true
type: str
cmake:
type: map
mapping:
application:
type: map
mapping:
source-dir:
type: str
configuration-dir:
type: str
board:
type: map
mapping:
name:
required: true
type: str
qualifiers:
type: str
revision:
type: str
path:
type: seq
sequence:
- type: str
devicetree:
type: map
mapping:
files:
type: seq
sequence:
- type: str
user-files:
type: seq
sequence:
- type: str
extra-user-files:
type: seq
sequence:
- type: str
include-dirs:
type: seq
sequence:
- type: str
bindings-dirs:
type: seq
sequence:
- type: str
kconfig:
type: map
mapping:
files:
type: seq
sequence:
- type: str
user-files:
type: seq
sequence:
- type: str
extra-user-files:
type: seq
sequence:
- type: str
sysbuild:
type: bool
toolchain:
type: map
mapping:
name:
type: str
version:
type: str
path:
type: str
zephyr:
type: map
mapping:
zephyr-base:
type: str
version:
type: str
vendor-specific:
type: map
mapping:
regex;(.*):
type: map
mapping:
regex;(.*):
type: str
west:
type: map
mapping:
command:
type: str
topdir:
type: str
version:
type: str
Loading
Loading