Releases: clicon/clixon
Clixon 5.9.0
The 5.9 release features with-defaults (RFC 6243) and RESTCONF Call-home (RFC 8071) along with numerous bugfixes and usability improvements.
New features
- With-defaults capability
- Standards
- RFC 6243 "With-defaults Capability for NETCONF"
- RFC 8040 "RESTCONF Protocol": Section 4.8.9
- Features:
:with-defaults
capabilityexplicit
basic mode- Full retrieval modes:
explicit
,trim
,report-all
,report-all-tagged
. - RESTCONF with-defaults query for XML and JSON
- Assigned meta-data for the
ietf-netconf-with-defaults:default
attribute for JSON (RFC8040 Sec 5.3.2)
- Assigned meta-data for the
- Added
withdefault
option to cli show commands- See User manual
- See CLISPEC changes
- See Netconf With-defaults Capability
- Standards
- RESTCONF call home
- Standard: RFC 8071 "NETCONF Call Home and RESTCONF Call Home"
- clixon-restconf.yang extended with callhome inspired by ietf-restconf-server.yang
- See e.g., draft-ietf-netconf-restconf-client-server-26.txt
- The
<socket>
list has been extended with acall-home
presence container including:- reconnect-strategy/max-attempts
- connection-type: either persistent or periodic
- idle-timeout for periodic call-homes.
- An example util client is
clixon_restconf_callhome_client.c
used in test cases - See call-home user guide
API changes on existing protocol/config features
Users may have to change how they access the system
- NETCONF error handling for
data-not-unique
andmissing-choice
follows standard more closely- data-not-unique:
- Added YANG namespace to
<non-unique>
tag - Changed
yang-unique
value to path insytead of single name - See RFC 7950 Sec 15.1
- Added YANG namespace to
- missing-choice:
- Added
<error-path>
- Added YANG namespace to
<missing-choice>
tag - See RFC 7950 Sec 15.6
- Added
- data-not-unique:
- Backend transaction plugins: edit of choice node will always result in a "del/add" event for all edits of change nodes, never a "change" event.
- Before, some cases were using a "change" event if the "yang ordering" happended to be the same.
- See more details in: Clixon backend transactions for choice/case is not logical
- Constraints on number of elements have been made stricter (ie unique, min/max-elements)
- Usecases that passed previously may now return error
- This includes:
- Check of incoming RPCs
- Check of non-presence containers
C/CLI-API changes on existing features
Developers may need to change their code
- C API changes
- Added
defaults
parameter toclicon_rpc_get()
andclicon_rpc_get_config()
- For upgrade, add new sixth parameter and set it to
NULL
.
- For upgrade, add new sixth parameter and set it to
- Added
- CLISPEC changes of cli show functions
- For details of updated API, see https://clixon-docs.readthedocs.io/en/latest/cli.html#show-commands
- Changed
cli_show_auto()
- Added parameters for pretty-print, state and with-default
- If the
<prefix>
is used, you need to change the call as follows:cli_show_auto(<db>, <format>, <prefix>)
->cli_show_auto(<db>, <format>, true, false, NULL, <prefix>)
- Otherwise the API is backward-compatible
- Changed
cli_show_config()
- Added parameters for pretty-print, state and with-default
- If the
<prefix>
parameter is used, you need to change the call as follows:cli_show_config(<db>, <format>, <xpath>, <ns>, <prefix>)
->cli_show_auto(<db>, <format>, <xpath>, <ns>, true, false, NULL, <prefix>)
- Otherwise the API is backward-compatible
- Removed
cli_show_auto_state()
, replace withcli_show_auto
with state set totrue
- Removed
cli_show_config_state()
, replace withcli_auto_show
with state set totrue
- Replaced
cli_auto_show()
withcli_show_auto_mode()
- The first argument is removed. You need to change all calls as follows:
cli_show_config(<treename>, <db>, ...
->cli_show_auto_menu(<db>, ...)
- The
cli_auto_show()
callback remains in 5.9.0 for backward compatible reasons, but will be removed in later releaes.
- The first argument is removed. You need to change all calls as follows:
Minor features
- Restconf:
- Openssl 3.0 is supported
- Refactoring of code closing sockets. Some cornercase bugs have been removed.
Corrected Bugs
- Fixed: Leak in restconf http/1 data path: when multiple packets in same connection.
- Fixed: Replace operation
- Fixed: When multiple lists have same key name, need more elaborate error message in case of configuration having duplicate keys
- Solved by implementing RFC7950 Sec 5.1 correctly
- Fixed: All values in list don't appear when writing "show " in cli
- Fixed: yang regular char \w not include underline char
- Fixed: Clixon backend transactions for choice/case is not logical
- Fixed: Clixon backend transaction callback fails for empty types
- Fixed: with-defaults=trim does not work due to dodgy handling of state data marked as default
- Fixed: YANG ordering fails for nested choice and action
- Fixed: YANG min-elements within non-presence container does not work
- Fixed: Issues with ietf-snmp modules
- Fixed: Missing/no namespace error in YANG augments with default values
- Fixed: Validation of mandatory in choice/case does not work in some cases
Clixon 5.8.0
5.8.0
28 July 2022
New features in Clixon 5.8.0 include a new SNMP frontend, YANG action and parse-able TEXT syntax.
New features
-
New SNMP frontend
- Support for SNMP for retreiving and setting values via net-snmp using MIB-YANG mapping defined in RFC6643.
- For details, see SNMP section of user manual
- YANG
clixon-config@2022-03-21.yang
changes:- Added options:
CLICON_SNMP_AGENT_SOCK
CLICON_SNMP_MIB
- Added options:
- New configure options:
--enable-netsnmp
--with-mib-generated-yang-dir=DIR
(test only)
- Thanks: Siklu Communications LTD for sponsoring this work
-
YANG Action (RFC 7950 Section 7.15)
- Register action callback with
action_callback_register()
- The main example contains example code
- Remains: check list keys, validate output
- See Support for "action" statement
- Register action callback with
-
TEXT syntax is now parseable
- This means you can save and load TEXT syntax files, as additions to XML/JSON/CLI formats
- Previously only output was supported.
- TEXT output format changed (see API changes)
- FOr more info, see user manual
- See Support performant load_config_file(...) for TEXT format
API changes on existing protocol/config features
Users may have to change how they access the system
- TEXT file format changed
- With new parsing of TEXT format, the output is changed
- Namespace/modulename added to top-level
- Leaf-list support:
a [ x y z ]
- List key support:
a x y { ... }
- See compile-time option
TEXT_LIST_KEYS
- Keep backward-compatible non-top-level prefix with compile-time option
TEXT_SYNTAX_NOPREFIX
- With new parsing of TEXT format, the output is changed
- Augmented XML uses default namespace
- Instead of using prefixes for augmented XML, assign the default namespace
- This does not change the semantics, but changes the way XML prefixes are used
- Example augmented ipv4 into interface:
- Previously:
<interface><ip:ipv4 xmlns:ip="urn:...:ietf-ip"><ip:enabled>...
- Now:
<interface><ipv4 xmlns="urn:...:ietf-ip"><enabled>...
- Previously:
C/CLI-API changes on existing features
Developers may need to change their code
- Changed C-API for xml translation/print the internal
cxobj
tree data structure to other formats.- Functions are merged, ie removed and replaced with more generic functions
- Added
skiptop
parameter, if set only apply to children of a node, skip top node- default is 0
- The new API is as follows:
clixon_xml2file()
/clixon_xml2cbuf()
- Print internal tree as XML to file or buffer, respectivelyclixon_json2file()
/clixon_json2cbuf()
- Print internal tree as JSON to file or buffer, respectivelyclixon_cli2file()
- Print internal tree as CLI format to fileclixon_txt2file()
- Print internal tree as text format to file
- As developer, you need to replace the old functions to the new API as follows:
clicon_xml2file(f, x, l, p)
->clixon_xml2file(f, x, l, p, NULL, 0, 0)
clicon_xml2file_cb(f, x, l, p, fn)
->clixon_xml2file(f, x, l, p, fn, 0, 0)
cli_xml2file(x, l, p, fn)
->clixon_xml2file(stdout, x, l, p, fn, 0, 0)
clicon_xml2cbuf(c, x, l, p, d)
->clixon_xml2cbuf(c, x, l, p, d, 0)
clicon_xml2str(x)
-> Rewrite using cbufs andclixon_xml2cbuf()
xml2json(f, x, p)
->clixon_json2file(f, x, p, NULL, 0, 0)
xml2json_cb(f, x, p, fn)
->clixon_json2file(f, x, p, fn, 0, 0)
xml2json_cbuf(c, x, p)
->clixon_json2cbuf(c, x, p, 0, 0)
xml2cli(h, f, x, p, fn)
->clixon_cli2file(h, f, x, p, fn, 0)
cli_xml2txt(x, fn, l)
->clixon_txt2file(stdout, x, l, NULL, 0, 0)
xml2txt(f, x, l)
->clixon_txt2file(f, x, l, NULL, 0, 0)
xml2txt_cb(f, x, fn)
->clixon_txt2file(f, x, 0, NULL, 0, 0)
Minor features
- Break-out RFC 7950 Section 6.1 tokenization
- This enables full string lexical parsing of some rules previously not fully compliant, including:
- refine, uses-augment, augment, if-feature, type, base.
- Also fixes some previous tokenization issues
- This enables full string lexical parsing of some rules previously not fully compliant, including:
- Full RFC 7950 if-feature-expr support (Section 7.20.2)
- Previous implementation did not handle nested if-feature expressions
- As part of fixing: YANG if-feature does not support nested boolean expression
- Added new yacc/lex parser for if-feature-expr string
- Added XPATH function
boolean()
- This caused problem for new NTP YANG in RFC 9249
- Feature Request: Log SSL events
- Added syslog NOTICE on failed user certs
Corrected Bugs
- Fixed: Clixon CLI issue: when I try to print the value of the leaf node nothing appeared
- Fixed: Can't use + symbol in the enum statement without quotes
- Fixed: String concatenation in YANG model leads to syntax error
- Fixed: "autocli:hide-show" extension cause bug in xmldb_put method #343
- Fixed: Schema Ambiguity Error with openconfig-system re: NTP
- Fixed: YANG mandatory statements within case nodes do not work
- Fixed: Nested YANG choice does not work
- Fixed: YANG if-feature does not support nested boolean expression
- Fixed: RPC edit-config payloads are not fully validated
Clixon 5.7.0
17 May 2022
The Clixon 5.7 release introduces (long overdue) NETCONF chunked framing as defined
in RFC 6242. It also introduces a limited http data service and lots of bugfixes.
New features
- Implementation of "chunked framing" according to RFC6242 for Netconf 1.1.
- First hello is 1.0 EOM framing, then successing rpc is chunked framing
- See
- Extended the Restconf implementation with a limited http-data static service
- Added two new config options to clixon-config.yang:
CLICON_HTTP_DATA_PATH
CLICON_HTTP_DATA_ROOT
- Added feature http-data to restconf-config.yang and the following option that needs to be true
enable-http-data
- Added
HTTP_DATA_INTERNAL_REDIRECT
compile-time option for internal redirects toindex.html
- For more info, see user manual documentation
- Added two new config options to clixon-config.yang:
API changes on existing protocol/config features
Users may have to change how they access the system
-
CLI
clixon_cli
reconnects to backend if backend restarts with a warning- Note that edits to the candidate database or locks will be lost
- To force the CLI to exit if backend restarts, undef
PROTO_RESTART_RECONNECT
- This is an effect of the fix of Broken pipe error seen in client (cli) when backend restarts and CLICON_SOCK is recreated, the CLI behavior on backend restart is changed.
- Expansion of YANG leafref type default behavior has changed
- In the autocli and handcrafted CLI:s using
expand_dbvar()
the CLI expansion followed the leafrefs to the sources, ie the origin of the leafrefs - Instead leafref expansion now expands according to existing leafrefs by default
- Example:
- Assume leafref with leafref pointing to source if values:
<if>a</if><if>b</if><if>c</if> <ifref>b</ifref>
- Existing behavior: expand to:
a, b, c
- New default behavior: expand to:
b
- Assume leafref with leafref pointing to source if values:
- To keep existing behavior, set
<CLICON_CLI_EXPAND_LEAFREF>true<CLICON_CLI_EXPAND_LEAFREF>
- In the autocli and handcrafted CLI:s using
-
Restconf
- Added 404 return without body if neither restconf, data or streams prefix match
-
Netconf:
- Usage of chunked framing
- To keep existing end-of-message encoding, set
CLICON_NETCONF_BASE_CAPABILITY
to0
- Added
clixon_netconf
command-line option-0
and changed-H
to-1
-0
means dont send hello, but fix netconf base version to 0 and use EOM framing-1
means dont send hello, but fix netconf base version to 1 and use chunked framing
- To keep existing end-of-message encoding, set
- Error message
data-not-unique
changed to return schema nodes instead of XML for RFC7950 compliance
- Usage of chunked framing
-
YANG
- Instead of removing YANG which is disabled by
if-feature
, replace it with an yanganydata
node.- See Adding feature to top level container doesn't work
- This means XML specified by such YANG is ignored, and it is not an error to access it
- Note the similarity with
CLICON_YANG_UNKNOWN_ANYDATA
- New
clixon-config@2022-03-21.yang
revision- Added option:
CLICON_RESTCONF_API_ROOT
CLICON_NETCONF_BASE_CAPABILITY
CLICON_HTTP_DATA_PATH
CLICON_HTTP_DATA_ROOT
CLICON_CLI_EXPAND_LEAFREF
- Added option:
- New
clixon-restconf@2022-03-21.yang
revision- Added option:
enable-http-data
- Added feature:
http-data
- Added option:
- Instead of removing YANG which is disabled by
C/CLI-API changes on existing features
Developers may need to change their code
- Added
nsc
parameter toxml2xpath()
and ensured the xpath uses prefixes.- Old code: add
NULL
as second parameter
- Old code: add
- Added
eof
parameter toclicon_rpc()
andclicon_rpc1()
and error handling modified
Minor features
- Command-line option: Extended
-l
of all clixon commands with-l n
which directs logging to/dev/null
- New: CLI load command for CLI syntax files (not only XML and JSON)
- New: Do not load clixon-restconf YANG file by default
- See prevent clixon-restconf@2021-05-20.yang module from loading
- Instead of always loading it, load it to datastore YANGs only if
CLICON_BACKEND_RESTCONF_PROCESS
istrue
- YANG unique: added single descendant node ids as special case
- This means that two variants are supported:
- unique "a b c", ie multiple direct children
- unique "a/b/c", ie single descendants
- RFC 7950 Sec 7.8.3 is somewhat unclear
- The combination is not supported
- This means that two variants are supported:
Corrected Bugs
- XPath parser: fixed some lexical issues
- Some complexities in Section 3.7 Lexical Structure of XPath 1.0 spec as follows
- There used to be some cornercases where function-names could not be used as nodes
- For example,
node()
is a nodetest, so/node/
caused an error. - In the grammar these include: axisnames, nodetests, functionnames
- The NCNames vs functionnames is now implemented according to the lexical structure section
- Fixed: Keywords containing '-' hyphen are missing from the auto-completion list
- Fixed by disabling
cligen_preference_mode
. This may have other side effects.
- Fixed by disabling
- Fixed: Returning a string while Querying leaf-list for single entry
- Fixed: A long TLS+HTTP/2 request such as by a browser causing block of other requests.
- Fixed: Error message seen twice in some cases
- Fixed: if choice is declared with multiple elements or leaf-list with in a case scope , addition or updation is not happening as expected
- This includes several choice/case adjustments to follow RFC 7950 Sec 7.9 better
- Fixed: HTTP/1 parse error for '/' path
- Fixed: YANG if-feature in config file of disables feature did not work, was always on
- This does not apply to the datastore, only the config file itself.
- Fixed: YANG key list check bad performance
- List key check did unique "xpath" lookup instead of direct child traverse
- Fixed: YANG unique single schema-nodeid required "canonical" namespace
- E.g.,
a/b
did not work even if there was default namespace in XML
- E.g.,
- Disabled xpath optimization for hierarchical list
- When
XPATH_LIST_OPTIMIZE
is set, patterns likey[k='3']
is optimized - But hierarchical lists should not be, ie when
a/y[k='3']
anda
is a list
- When
- Fixed: Removed warning at startup:
No YANG spec for module-set
- Fixed: HTTP/1 multiple write requests in single session appended data between writes, eg PUT+PUT.
- Fixed: Broken pipe error seen in client (cli) when backend restarts and CLICON_SOCK is recreated
- Fixed: Xpath API do not support filter data by wildcard
- Fixed: SEGV in cli show yang
5.6.0
Clixon 5.6 removes the dependency of libevhtp and libevent2 for native HTTP/1
RESTCONF, module-state has been upgraded to RFC8525 and a lot of bugs
have been fixed, thanks to the community for all feedback.
New features
- Yang library upgraded from RFC7895 to RFC 8525
- See API changes for more info
- RESTCONF Internal HTTP/1 native parser
- Removed dependency of libevhtp/libevent2
- Replace configure option
--disable-evhtp
with--disable-http1
for disabling HTTP/1 (default enabled)
API changes on existing protocol/config features
Users may have to change how they access the system
- Module state upgrade: RFC7895 to RFC8525:
- To upgrade to RFC8525:
- Set
CLICON_YANG_LIBRARY
totrue
andCLICON_MODULE_LIBRARY_RFC7895
tofalse
- Set
- To keep RFC7895:
- Set both
CLICON_YANG_LIBRARY
andCLICON_MODULE_LIBRARY_RFC7895
totrue
- Set both
- Following RFC8525, the upgrade means that the state-data returned using GET is changed:
- Preamble changed from:
<modules-state>...
to:<yang-library><module-set>...
module-state-id
changed tocontent-id
conformance-type
removed
- Preamble changed from:
- Note that the datastore feature
CLICON_XMLDB_MODSTATE
is backward compatible with RFC8525.
- To upgrade to RFC8525:
- New
clixon-config@2022-02-11.yang
revision- Added option:
CLICON_LOG_STRING_LIMIT
CLICON_YANG_LIBRARY
- Changed default value:
CLICON_MODULE_LIBRARY_RFC7895
to false
- Removed (previosly marked) obsolete options:
CLICON_RESTCONF_PATH
CLICON_RESTCONF_PRETTY
CLICON_CLI_GENMODEL
CLICON_CLI_GENMODEL_TYPE
CLICON_CLI_GENMODEL_COMPLETION
CLICON_CLI_AUTOCLI_EXCLUDE
CLICON_CLI_MODEL_TREENAME
- Added option:
- RESTCONF replies on the form:
{"data":...}
changed to:{"ietf-restconf:data":...}
- YANG leafref
require-instance
default changed totrue
- This makes leafref validation stricter
- See statement: require-instance should be true if not present according to rfc7950 Sec 9.9.3
- Autotools/configure changes
configure --with-wwwdir=<dir>
is removed- Configure option
--disable-evhtp
with--disable-http1
for disabling HTTP/1 (default enabled)
- Command field of
clixon-lib:process-control
RPC reply used CDATA encoding but now uses regular XML encoding
C/CLI-API changes on existing features
- Added RFC7951 parameter to
clixon_json_parse_string()
andclixon_json_parse_file()
- If set, honor RFC 7951: JSON Encoding of Data Modeled with YANG, eg it requires module name prefixes
- If not set, parse as regular JSON
Minor features
- Added: Strict auto completion for CLI argument expansion #163
- Added: Convert int64, uint64 and decimal64 to string in xml to json #310
- Backend ignore of
SIGPIPE
. This occurs if client quits unexpectedly over the UNIX socket.- This is a timing issue but occurs more frequently in large RESTCONF messages.
- Added option:
CLICON_LOG_STRING_LIMIT
configure option- Limit the length of log and debug messages. Some log messages are dependendent on sizes that can be very large, such as packet lengths. This new option constrains the length of all messgaes. By default no limits.
Corrected Bugs
- Fixed: Validate error when appending module B grouping to module A item use augment statement #308
- Fixed: Restconf PATCH method request failed on item defined by submodule #306
- Fixed: restconf GET json response does not encode top level node with namespace as per rfc #303
- Fixed: statement: require-instance should be true if not present according to rfc7950 Sec 9.9.3
- See also API changes
- Fixed: input RPC validation of YANG
choice
, more specifically, withoutcase
keyword - Fixed: More than one unknown/extension in combination with augment of extension resulted in extension being skipped.
5.5.0
5.5.0
20 January 2022
This release introduces a new autocli design with a clixon-autocli YANG file
New features
- Changed auto-cli design
- See autocli documentation for overview
- Added new YANG
clixon-autocli.yang
moving all autocli options there- Default rules for module exclusion, list-keywords, edit-modes, treeref-state and completion
- Specialized rules for module exclusion and compression
- Replaced separate autocli trees with a single
@basemodel
tree by using filter labels- Filter labels are added to the basemodel tree and then filtered out using
@remove:<label>
- This method reduces memory usage and is more generic
- Backward compatible: can continue use the "old" trees.
- Note: while @datamodel etc are backward compatible, the autocli redesign is NOT backward compatible
- see API changes
- Filter labels are added to the basemodel tree and then filtered out using
- New autocli edit-mode design
- Control which modes to use with
edit-mode-default
- Default is create edit-mode for all containers and list entries
- New edit-mode tree:
@datamodelmode
- Control which modes to use with
- Moved hide extensions from
clixon-lib
toclixon-autocli
API changes on existing protocol/config features
Users may have to change how they access the system
- Auto-cli edit-modes changed
- CLI-spec variable
CLICON_PROMPT
%W
changed semantics due to long prompt- From "Full Working edit path" to "Last element of working path"
- Use
%w
if you want to keep "Full working path"
- Edit modes only for list and container nodes
- Change cli spec entry to
edit @datamodelmode, cli_auto_edit("basemodel");
- CLI-spec variable
- New
clixon-lib@2021-12-05.yang
revision- Extension
autocli-op
obsoleted and no longer supported, use clixon-autoclihide
andhide-show
instead as follows:cl:autocli-op hide
->autocli:hide
cl:autocli-op hide-database
->autocli:hide-show
cl:autocli-op hide-database-auto-completion
->autocli:hide; autocli:hide-show
- Extension
- New
clixon-config@2021-12-05.yang
revision- Removed obsolete options:
CLICON_YANG_LIST_CHECK
- Fixed: Configure option
CLICON_RESTCONF_PRETTY
was marked as obsolete but was still used.CLICON_RESTCONF_PRETTY
is now obsolete for sure- Use:
restconf/pretty
- Fixed: Configure option
CLICON_RESTCONF_PATH
was marked as obsolete but was still used.CLICON_RESTCONF_PATH
is now obsolete for sure- Instead if you use fgci/nginx:
- Use
restconf/fcgi-socket
- Ensure
<CLICON_FEATURE>clixon-restconf:fcgi</CLICON_FEATURE>
is set
- Use
- Marked as obsolete and moved autocli config options from clixon-config.yang to clixon-autocli.yang
- Use:
<config><autocli>...
for configuring the autocli - For details, see autocli upgrade documentation
- Use:
- Removed obsolete options:
C/CLI-API changes on existing features
Developers may need to change their code
-
Removed
#ifdef __GNUC__
around printf-like prototypes since both clang and gcc have format/printf macros defined -
Test changes
- Use
YANG_STANDARD_DIR
from./configure --with-yang-standard-dir=DIR
instead ofYANGMODELS
from site.sh - Remove dependency of IETF YANGs on most tests
- Remove dependency of example/main in most tests, instead make local copy of example yang
- Changed
configure --with-yang-standard-installdir
toconfigure --with-yang-standard-dir
- Use
Corrected Bugs
- Fixed: Autocli YANG patterns including
"
were not properly escaped:\"
- Ensure auto-cli can be run with config option
CLICON_CLI_VARONLY=1
- Fixed: SEGV in backend callback for user-defined RFC:
- rpc_callback_call(): Check if nrp parameter is NULL