Make Kconfig data visible in documentation

[chrysn]

Description

We have good metadata about our configuration options in Kconfig, let's use them to make doc.riot-os.org better.

Currently, we have duplication of both the defaults (which may become unused even, going more towards Kconfig for tunables ([issue reference needed], @MrKevinWeiss is working on it AIU) and documentation.

Progress so far

• There is a usable Python library that makes Kconfig easily accessible.
• Our Kconfigs generally expect to be bound to applications or even boards AIU (referencing APPDIR or CPU). Seems they still produce good input without it, provided the environment at least has RIOTBOARD=$PWD/boards/ RIOTCPU=$PWD/cpu/ RIOTPKG=$PWD/pkg KCONFIG_GENERATED_DEPENDENCIES=/does-not-exist KCONFIG_EXTERNAL_MODULE_CONFIGS=/does-not-exist KCONFIG_EXTERNAL_PKG_CONFIGS=/does-not-exist.
• Even while those do suffice as dummy variables, for docs it would be great to have access to the unevaluated original values. For example, LWM2M_DEVICE_MODEL defaults to "${BOARD}". (As a workaround, setting BOARD="${BOARD}" may make sense).
• Doxygen can't reference into the source Kconfig files yet, filed at Support #line doxygen/doxygen#10697

import kconfiglib
import re

print("""/**
        @defgroup kconfig_uncategorized Uncategorized Kconfig tunables
        @brief KConfig values
        @{
         */""")

c = kconfiglib.Kconfig()
for s in c.syms.values():
    if not s.nodes:
        # Not sure what these are, but they look more like artifacts than like
        # actual choices
        continue

    if s.name.split("_", 1)[0] in ("MODULE", "HAS", "HAVE", "USEPKG") or s.name.startswith("KCONFIG_USEMODULE_"):
        continue

    if s.nodes[0].filename == 'sys/net/gnrc/transport_layer/tcp/Kconfig':
        group = "net_gnrc_tcp_conf"
    elif s.nodes[0].filename == 'sys/net/gnrc/application_layer/dhcpv6/Kconfig':
        group = None # no 1:1 matching: eg. GNRC_DHCPV6_CLIENT_6LBR_UPSTREAM is in the gnrc_dhcpv6_client_simple_pd group
    if s.nodes[0].filename == 'sys/net/credman/Kconfig':
        group = "net_credman_conf"
    elif (matched := re.match('pkg/([a-z]*)/Kconfig', s.nodes[0].filename)):
        group = "pkg_" + matched.groups()[0]
    else:
        group = None

    # See https://github.com/doxygen/doxygen/issues/10697
    cleaned_filename = re.sub("/+", "/", s.nodes[0].filename) # and Doxygen would think the `//` is a comment marker
    print(f'#line {s.nodes[0].linenr} "{cleaned_filename}"')

    doctype = {kconfiglib.BOOL: "bool", kconfiglib.INT: "int_from_int", kconfiglib.HEX: "int_from_hex"}.get(s.type, "opaque")
    print(f"{doctype} {c.config_prefix}{s.name}; /**<")

    if group:
        print(f"@ingroup {group}")

    for n in s.nodes:
        prompt = n.prompt and n.prompt[0]
        if prompt:
            print("@brief", prompt)
        if n.help:
            print()
            print(n.help)
    if s.defaults:
        print()
        print("Default value: ", " or ".join(f"{kconfiglib.expr_str(default)} if {kconfiglib.expr_str(cond)}" for (default, cond) in s.defaults))
    print("*/")

print("/** @} */")

This has a bunch of issues still -- it prints some literal symbols (@var opaque(?) 5120), and I've seen some duplication here and there, and not all items in here are really tunables, but

@var bool CONFIG_DEVELHELP
@brief Development Help

Set to enable code in RIOT that does safety checking which might not be
needed in a production environment but helps in the development
process.

@var bool CONFIG_LORAMAC_DEFAULT_ADR                                                                                                 
@brief Enable ADR                                                                                                             
                                                                                                                              
Enable or disable Adaptive Data Rate (ADR). If enabled the end node will                                                      
inform the network server about the status of ADR using the ADR field in                                                      
uplink data packet. The network server will then optimize the data rate                                                       
and the transmission power of the end node based on the information                                                           
collected from the network.

doesn't look half bad.

[edit: The updated snippet produces C source code, because Doxygen is very stubborn around having entities defined in non-code files; manually labeling them as @ref targets would work fine too]

[leandrolanzieri]

I'd like to point to https://github.com/leandrolanzieri/RIOT/tree/pr/doc/kconfig_symbols where quite some time ago I started generating documentation from Kconfig. The approach was to generate header files when make doc is called, which would contain the configuration symbols, descriptions, types, defaults and constraints (e.g., ranges for integers).