From 29a9245d152fe8451f2b565c67accc57b71d174f Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Sat, 9 Mar 2024 13:00:58 +0200 Subject: [PATCH 1/5] added gnmi depth ext documentation --- rpc/gnmi/gnmi-depth.md | 290 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 290 insertions(+) create mode 100644 rpc/gnmi/gnmi-depth.md diff --git a/rpc/gnmi/gnmi-depth.md b/rpc/gnmi/gnmi-depth.md new file mode 100644 index 0000000..a087e3f --- /dev/null +++ b/rpc/gnmi/gnmi-depth.md @@ -0,0 +1,290 @@ +# gNMI Depth Extension + +**Contributors:** Roman Dodin, Matthew MacDonald + +**Date:** March 9, 2024 + +**Version:** 0.1.0 + +## 1. Purpose + +The implicit recursive data fetching used by the gNMI servers makes the implementations simple. Whilst the implementation simplicity is important, the lack of server-side filtering for the data requested in gNMI RPCs may be considered a limitation for some gNMI users and systems. + +The gNMI Depth Extension allows a client to control the depth of the recursion when the server evaluates a group of paths in the Subscribe or Get RPC. + +Orchestration, Network Management and Monitoring Systems can benefit from this extension as it: + +1. reduces the load on the server when data is to be fetched from the Network OS during the recursive data extraction +2. reduces the bytes on the wire payload, by sending fewer data + +gNMI Depth Extension proto specification is defined in [gnmi_ext.proto](https://github.com/openconfig/gnmi/blob/master/proto/gnmi_ext/gnmi_ext.proto). + +## 2. Demo model + +To explain the concept of the depth-based filtering, consider the following model that is used in the implementation examples throughout this document: + +```yang + container basket { + leaf name { + type string; + } + + leaf-list contents { + type string; + } + + list fruits { + key "name"; + + leaf name { + type string; + } + + leaf-list colors { + type string; + } + + leaf size { + type string; + } + + container origin { + leaf country { + type string; + } + + leaf city { + type string; + } + } + } + + container description { + leaf fabric { + type string; + } + + } + container broken { + presence "This container is broken"; + leaf reason { + type string; + } + } + } +``` + +It's tree representation: + +``` +module: app + +--rw basket + +--rw name? string + +--rw contents* string + +--rw fruits* [name] + | +--rw name string + | +--rw colors* string + | +--rw size? string + | +--rw origin + | +--rw country? string + | +--rw city? string + +--rw description + | +--rw fabric? string + +--rw broken! + +--rw reason? string +``` + +We populate this data schema with the following values: + +```json + basket { + contents [ + fruits + vegetables + ] + fruits apples { + size XL + colors [ + red + yellow + ] + origin { + country NL + city Amsterdam + } + } + fruits orange { + size M + } + description { + fabric cotton + } + broken { + reason "too heavy" + } + } +``` + +## 3. Concepts + +The Depth extension allows clients to specify the depth of the subtree to be returned in the response. The depth is specified as the number of levels below the specified path. + +The extension itself has a single field that controls the depth level: + +```proto +message Depth { + uint32 level = 1; +} +``` + +### 3.1 Depth level values + +#### 3.1.1 Value 0 + +Depth value of 0 means no depth limit and behaves the same as if the extension was not specified at all. + +#### 3.1.2 Value 1 + +Value of 1 means only the specified path and its direct children will be returned. See Children section for more info. + +#### 3.1.2 Value of N+ + +Value of N+ where N>1 means all elements of the specified path up to N level and direct children of N-th level. + +### 3.2 Children nodes + +The Depth extension operates the value of "direct children of a schema node". What we understand by direct children: + +1. leafs +2. leaf-lists + +Only these elements are to be returned if depth extension with non-0 value is specified for a specified depth level. + +### 3.3 RPC support + +The Depth extension applies to Get and Subscribe requests only. When used with Capability and Set RPC the server should return an error. + +## 4 Examples + +Using the data model from Section 2 we will run through a set of examples using the patched version of [openconfig/gnmic](https://gnmic.openconfig.net/) client with the added Depth extension support. We can provide the patched gnmic binary for Linux x86_84 if you want to try it out. + +### 4.1 depth 1, path `/basket` + +The most common way to use the depth extension (as we see it) is to use it with level=1. This gets you the immediate child nodes of the schema node targeted by a path. + +Consider the following gnmic command targeting `/basket` path: + +```bash +gnmic -e json_ietf get --path /basket --depth 1 +``` + +```json +[ + { + "contents": [ + "fruits", + "vegetables" + ] + } +] +``` + +As per the design, only the leaf and leaf-list nodes are returned. Since our `/basket` container has only `leaf-list` elements (no leafs) a single element `contents` is returned. + +You can see how this makes it possible to reduce the amount of data extracted by the server and sent over the wire. Many applications might require fetching only leaf values of a certain container to make some informed decision without requiring any of the nested data. + +### 4.2 depth 1, path `/basket/fruits` + +When the path targets the list schema node, all elements of this list is returned with their children nodes + +```bash +gnmic -e json_ietf get --path /basket/fruits --depth 1 +``` + +```json +[ + { + "fruits": [ + { + "colors": [ + "red", + "yellow" + ], + "name": "apples", + "size": "XL" + }, + { + "name": "orange", + "size": "M" + } + ] + } +] +``` + +Again, please keep in mind that only leafs and leaf-lists are returned for every list element. + +### 4.3 depth 2, path `/basket` + +When the depth level is set to values >1, all elements from the path to the provided level value are returned in full with the last level including only leafs and leaf-lists. + +```bash +gnmic -e json_ietf get --path /basket --depth 2 +``` + +```json +[ + { + "broken": { + "reason": "too heavy" + }, + "contents": [ + "fruits", + "vegetables" + ], + "description": { + "fabric": "cotton" + }, + "fruits": [ + { + "colors": [ + "red", + "yellow" + ], + "name": "apples", + "size": "XL" + }, + { + "name": "orange", + "size": "M" + } + ] + } +] +``` + +Here is what happens: + +image + +The 1st level elements are returned, since depth level is 2. +On the 2nd level we return only leafs and leaf-lists, hence the `.fruits.origin` is not present. + +## 5 Prior art + +Netconf standardized `max-depth` in [RFC 85226](https://datatracker.ietf.org/doc/html/rfc8526#section-3.1.1): + +> The "max-depth" parameter can be used by the client to limit the +> number of subtree levels that are returned in the reply. + +The NETCONF way of using the max-depth differs in a sense that `depth=1` returns the element pointed by the path, but not its children. depth=2 returns children of the element pointed by the path. + +I find this behavior strange, as I don't see an operational reason to return the element itself when depth is 1. + +## 6 Summary + +We believe that the Depth extension has a generic applicability whilst not bein a burden for the implementation (henceforth no subtree filtering with XPath or anything of sorts). + +Yet it delivers important quality of life improvements for consuming systems that may get the required data nodes faster and with less processing time spent. + +This is assuming that cumulative time of fetching only leaf/leaf-lists values by the server is smaller than the recursive data retrieval combined with payload unmarshalling on the client side. From 4ca30e8117b228007ee866b6ca86cc315fa33933 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Sat, 9 Mar 2024 13:02:52 +0200 Subject: [PATCH 2/5] remove json syntax --- rpc/gnmi/gnmi-depth.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rpc/gnmi/gnmi-depth.md b/rpc/gnmi/gnmi-depth.md index a087e3f..3e40e64 100644 --- a/rpc/gnmi/gnmi-depth.md +++ b/rpc/gnmi/gnmi-depth.md @@ -96,7 +96,7 @@ module: app We populate this data schema with the following values: -```json +``` basket { contents [ fruits From 39d98af22f0842573f183dfc5deb420ee4e30954 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Sun, 10 Mar 2024 21:14:25 +0200 Subject: [PATCH 3/5] remove sections that don't bear signigicant meaning for the doc --- rpc/gnmi/gnmi-depth.md | 19 ------------------- 1 file changed, 19 deletions(-) diff --git a/rpc/gnmi/gnmi-depth.md b/rpc/gnmi/gnmi-depth.md index 3e40e64..c4a80b7 100644 --- a/rpc/gnmi/gnmi-depth.md +++ b/rpc/gnmi/gnmi-depth.md @@ -269,22 +269,3 @@ Here is what happens: The 1st level elements are returned, since depth level is 2. On the 2nd level we return only leafs and leaf-lists, hence the `.fruits.origin` is not present. - -## 5 Prior art - -Netconf standardized `max-depth` in [RFC 85226](https://datatracker.ietf.org/doc/html/rfc8526#section-3.1.1): - -> The "max-depth" parameter can be used by the client to limit the -> number of subtree levels that are returned in the reply. - -The NETCONF way of using the max-depth differs in a sense that `depth=1` returns the element pointed by the path, but not its children. depth=2 returns children of the element pointed by the path. - -I find this behavior strange, as I don't see an operational reason to return the element itself when depth is 1. - -## 6 Summary - -We believe that the Depth extension has a generic applicability whilst not bein a burden for the implementation (henceforth no subtree filtering with XPath or anything of sorts). - -Yet it delivers important quality of life improvements for consuming systems that may get the required data nodes faster and with less processing time spent. - -This is assuming that cumulative time of fetching only leaf/leaf-lists values by the server is smaller than the recursive data retrieval combined with payload unmarshalling on the client side. From 12334b5f597bd69e1523d81d49f1b715b2ef2a6c Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Tue, 26 Mar 2024 20:13:35 +0100 Subject: [PATCH 4/5] added missing backquotes Co-authored-by: Rob Shakir --- rpc/gnmi/gnmi-depth.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rpc/gnmi/gnmi-depth.md b/rpc/gnmi/gnmi-depth.md index c4a80b7..c4d1a79 100644 --- a/rpc/gnmi/gnmi-depth.md +++ b/rpc/gnmi/gnmi-depth.md @@ -10,7 +10,7 @@ The implicit recursive data fetching used by the gNMI servers makes the implementations simple. Whilst the implementation simplicity is important, the lack of server-side filtering for the data requested in gNMI RPCs may be considered a limitation for some gNMI users and systems. -The gNMI Depth Extension allows a client to control the depth of the recursion when the server evaluates a group of paths in the Subscribe or Get RPC. +The gNMI Depth Extension allows a client to control the depth of the recursion when the server evaluates a group of paths in the `Subscribe` or `Get` RPC. Orchestration, Network Management and Monitoring Systems can benefit from this extension as it: From f28e8faffc585824cdb2bdf968267a5c95c01f67 Mon Sep 17 00:00:00 2001 From: Roman Dodin Date: Tue, 26 Mar 2024 21:24:09 +0200 Subject: [PATCH 5/5] addressed review comments --- rpc/gnmi/gnmi-depth.md | 83 ++++++++++++++++++++++++++++++------------ 1 file changed, 60 insertions(+), 23 deletions(-) diff --git a/rpc/gnmi/gnmi-depth.md b/rpc/gnmi/gnmi-depth.md index c4d1a79..7d637cb 100644 --- a/rpc/gnmi/gnmi-depth.md +++ b/rpc/gnmi/gnmi-depth.md @@ -8,20 +8,29 @@ ## 1. Purpose -The implicit recursive data fetching used by the gNMI servers makes the implementations simple. Whilst the implementation simplicity is important, the lack of server-side filtering for the data requested in gNMI RPCs may be considered a limitation for some gNMI users and systems. +The implicit recursive data fetching used by the gNMI servers makes the +implementations simple. Whilst the implementation simplicity is important, the +lack of server-side filtering for the data requested in gNMI RPCs may be +considered a limitation for some gNMI users and systems. -The gNMI Depth Extension allows a client to control the depth of the recursion when the server evaluates a group of paths in the `Subscribe` or `Get` RPC. +The gNMI Depth Extension allows a client to control the depth of the recursion +when the server evaluates a group of paths in the `Subscribe` or `Get` RPC. -Orchestration, Network Management and Monitoring Systems can benefit from this extension as it: +Orchestration, Network Management and Monitoring Systems can benefit from this +extension as it: -1. reduces the load on the server when data is to be fetched from the Network OS during the recursive data extraction +1. reduces the load on the server when data is to be fetched from the Network +OS during the recursive data extraction 2. reduces the bytes on the wire payload, by sending fewer data -gNMI Depth Extension proto specification is defined in [gnmi_ext.proto](https://github.com/openconfig/gnmi/blob/master/proto/gnmi_ext/gnmi_ext.proto). +gNMI Depth Extension proto specification is defined in +[gnmi_ext.proto]( +nmi_ext.proto). ## 2. Demo model -To explain the concept of the depth-based filtering, consider the following model that is used in the implementation examples throughout this document: +To explain the concept of the depth-based filtering, consider the following +model that is used in the implementation examples throughout this document: ```yang container basket { @@ -127,7 +136,9 @@ We populate this data schema with the following values: ## 3. Concepts -The Depth extension allows clients to specify the depth of the subtree to be returned in the response. The depth is specified as the number of levels below the specified path. +The Depth extension allows clients to specify the depth of the subtree to be +returned in the response. The depth is specified as the number of levels below +the specified path. The extension itself has a single field that controls the depth level: @@ -141,36 +152,51 @@ message Depth { #### 3.1.1 Value 0 -Depth value of 0 means no depth limit and behaves the same as if the extension was not specified at all. +Depth value of 0 means no depth limit and behaves the same as if the extension +was not specified at all. #### 3.1.2 Value 1 -Value of 1 means only the specified path and its direct children will be returned. See Children section for more info. +Value of 1 means only the specified path and its direct children will be +returned. See Children section for more info. -#### 3.1.2 Value of N+ +#### 3.1.2 Value of N -Value of N+ where N>1 means all elements of the specified path up to N level and direct children of N-th level. +Value of N where N>1 means all elements of the specified path up to N level and +direct children of N-th level. ### 3.2 Children nodes -The Depth extension operates the value of "direct children of a schema node". What we understand by direct children: +The Depth extension operates the value of "direct children of a schema node". +When YANG data modelling language is used, the following elements are +considered as children nodes of a schema node: 1. leafs 2. leaf-lists -Only these elements are to be returned if depth extension with non-0 value is specified for a specified depth level. +In a generic data model language, the children nodes are the elements of scalar +type, and arrays of these elements, that are direct descendants of the schema +node. + +Only these elements are to be returned if depth extension with non-0 value is +specified for a specified depth level. ### 3.3 RPC support -The Depth extension applies to Get and Subscribe requests only. When used with Capability and Set RPC the server should return an error. +The Depth extension applies to Get and Subscribe requests only. When used with +Capability and Set RPC the server should return an error. ## 4 Examples -Using the data model from Section 2 we will run through a set of examples using the patched version of [openconfig/gnmic](https://gnmic.openconfig.net/) client with the added Depth extension support. We can provide the patched gnmic binary for Linux x86_84 if you want to try it out. +Using the data model from Section 2 we will run through a set of examples using +the patched version of [openconfig/gnmic](https://gnmic.openconfig.net/) client +with the added Depth extension support. ### 4.1 depth 1, path `/basket` -The most common way to use the depth extension (as we see it) is to use it with level=1. This gets you the immediate child nodes of the schema node targeted by a path. +The most common way to use the depth extension (as we see it) is to use it with +level=1. This gets you the immediate child nodes of the schema node targeted by +a path. Consider the following gnmic command targeting `/basket` path: @@ -189,13 +215,19 @@ gnmic -e json_ietf get --path /basket --depth 1 ] ``` -As per the design, only the leaf and leaf-list nodes are returned. Since our `/basket` container has only `leaf-list` elements (no leafs) a single element `contents` is returned. +As per the design, only the leaf and leaf-list nodes are returned. Since our +`/basket` container has only `leaf-list` elements (no leafs) a single element +`contents` is returned. -You can see how this makes it possible to reduce the amount of data extracted by the server and sent over the wire. Many applications might require fetching only leaf values of a certain container to make some informed decision without requiring any of the nested data. +You can see how this makes it possible to reduce the amount of data extracted +by the server and sent over the wire. Many applications might require fetching +only leaf values of a certain container to make some informed decision without +requiring any of the nested data. ### 4.2 depth 1, path `/basket/fruits` -When the path targets the list schema node, all elements of this list is returned with their children nodes +When the path targets the list schema node, all elements of this list is +returned with their children nodes ```bash gnmic -e json_ietf get --path /basket/fruits --depth 1 @@ -222,11 +254,14 @@ gnmic -e json_ietf get --path /basket/fruits --depth 1 ] ``` -Again, please keep in mind that only leafs and leaf-lists are returned for every list element. +Again, please keep in mind that only leafs and leaf-lists are returned for +every list element. ### 4.3 depth 2, path `/basket` -When the depth level is set to values >1, all elements from the path to the provided level value are returned in full with the last level including only leafs and leaf-lists. +When the depth level is set to values >1, all elements from the path to the +provided level value are returned in full with the last level including only +leafs and leaf-lists. ```bash gnmic -e json_ietf get --path /basket --depth 2 @@ -265,7 +300,9 @@ gnmic -e json_ietf get --path /basket --depth 2 Here is what happens: -image +image The 1st level elements are returned, since depth level is 2. -On the 2nd level we return only leafs and leaf-lists, hence the `.fruits.origin` is not present. +On the 2nd level we return only leafs and leaf-lists, hence the +`.fruits.origin` is not present.