[Inspur][HLD] Add cli_wrapper hld #1091

inspurSDN · 2022-09-30T07:33:28Z

Description:
    cli_wrapper high-level design

Description: cli_wrapper high-level design

Description: Add test results

Description: Update test result

zhangyanzhao · 2023-05-30T15:20:12Z

Community review recording https://zoom.us/rec/share/lL5n1Vb4rnSsL7RgVZf9zW8ZAVOCEX7xoNHpYjs69H-15O3wa6ovrMp8njJJN9l6.PjTc_BJfJNMv2_AO

jeffhtgt · 2023-05-30T15:27:27Z

doc/cli_wrapper/shd-cli_wrapper_hld.md

+
+* Disable CLI wrapper service
+
+![Encode format](images/wrapper_disable.jpg)


How is this done? Does a user have to manually rename the file and stop the server? A global rules flag set to "n" should take care of these steps instead of manual intervention on every switch.

zhangyanzhao · 2023-05-30T15:51:15Z

@inspurSDN can you please update the HLD by using the HLD template https://github.com/sonic-net/SONiC/blob/master/doc/hld_template.md? Some required info/sections are missed in your current HLD. Thanks.

zhangyanzhao · 2023-05-30T16:01:03Z

@venkatmahalingam registered as reviewer.

venkatmahalingam · 2023-05-31T00:46:47Z

doc/cli_wrapper/shd-cli_wrapper_hld.md

+
+  CLI_wrapper server is responsible for the execution of click command. It starts when system boots up, and loads all needed libraries to server. Then setup a socket to wait client's request. Once it accepts a request, it spawns a CLI_worker process to deal with this request. 
+
+  In order to avoid that some command may stuck in CLI worker and cause resource leakage, a watchdog thread is designed to watch the execution time of CLI worker, once the execution time exceeds maximum execution time, the CLI worker will be gracefully terminated. But if the CLI worker is still alive after one minute, a kill signal will be sent to CLI worker to tear down it. 


There are config/show commands that can go beyond 1 min, hope you have a configuration to increase the timeout if required.

When the show output has more than 1 page, how are you handling the pagination? can we make sure the current behavior is not disturbed with this new design?

venkatmahalingam · 2023-05-31T00:52:15Z

doc/cli_wrapper/shd-cli_wrapper_hld.md

+
+  In order to avoid that some command may stuck in CLI worker and cause resource leakage, a watchdog thread is designed to watch the execution time of CLI worker, once the execution time exceeds maximum execution time, the CLI worker will be gracefully terminated. But if the CLI worker is still alive after one minute, a kill signal will be sent to CLI worker to tear down it. 
+
+  The local DB is used to store the record of spawned CLI worker, it records the steady timestamp and pid of the CLI worker. 


Is this local DB is some kind of a Q?

When there are more than client, how is server handling the requests from clients at a time? How's config/show errors are being responded back to the client?

venkatmahalingam · 2023-05-31T00:55:19Z

doc/cli_wrapper/shd-cli_wrapper_hld.md

+
+![Command execution flow](images/command_execution_flow.jpg)
+
+* System ctrl bring up CLI_wrapper server when system starts


Please keep this feature disabled by default, if we're comfortable making it enabled some point in time in the future, we can have it enabled by default.

Do we have an option to use current click commands (without wrapper) even after this feature is enabled? I just want to make sure we're not blocked because of some bug in the wrapper routines?

venkatmahalingam · 2023-05-31T01:02:46Z

doc/cli_wrapper/shd-cli_wrapper_hld.md

+
+### 1.3.2 Command execution flow
+
+![Command execution flow](images/command_execution_flow.jpg)


Please change the diagram to include more than one client, also, not sure how the context for the particular command is carried end to end, please clarify.

venkatmahalingam · 2023-05-31T01:04:42Z

doc/cli_wrapper/shd-cli_wrapper_hld.md

+* CLI worker decodes the command to get parameters and environment setting, then execute this command based on those settings. In the end, it sends return code and data back to CLI wrapper client
+
+### 1.3.3 Watchdog execution flow
+


Is it possible to accept optional value for each command for execution time? sometimes, it's hard to predict the execution time of the command.

venkatmahalingam · 2023-05-31T01:06:18Z

doc/cli_wrapper/shd-cli_wrapper_hld.md

+
+  * For less powerful CPUs, cli_wrapper can significantly reduce execution time.
+
+# 3. Open questions


Please clarify the design around the interactive commands (yes/no).

dgsudharsan · 2023-06-01T23:14:38Z

doc/cli_wrapper/shd-cli_wrapper_hld.md

+
+* CLI_wrapper server
+
+  CLI_wrapper server is responsible for the execution of click command. It starts when system boots up, and loads all needed libraries to server. Then setup a socket to wait client's request. Once it accepts a request, it spawns a CLI_worker process to deal with this request. 


Please check the case of warmboot/fastboot while spawning CLI wrapper server. Are there any CLI commands that are executed during the warmboot/fastboot? If not I recommend delaying the CLI wrapper server until reconciliation is complete.

dgsudharsan · 2023-06-02T01:56:57Z

doc/cli_wrapper/shd-cli_wrapper_hld.md

+
+  CLI_wrapper server is responsible for the execution of click command. It starts when system boots up, and loads all needed libraries to server. Then setup a socket to wait client's request. Once it accepts a request, it spawns a CLI_worker process to deal with this request. 
+
+  In order to avoid that some command may stuck in CLI worker and cause resource leakage, a watchdog thread is designed to watch the execution time of CLI worker, once the execution time exceeds maximum execution time, the CLI worker will be gracefully terminated. But if the CLI worker is still alive after one minute, a kill signal will be sent to CLI worker to tear down it. 


I believe this needs to be rephrased. If I am not mistaken the watchdog will terminate only after 10 minutes and send kill at 11th minute if there is no resposne.

dgsudharsan · 2023-06-02T02:16:01Z

doc/cli_wrapper/shd-cli_wrapper_hld.md

+
+![Encode format](images/wrapper_enable.jpg)
+
+    1. Replace main.py with wrapped main.py


Can we enable disable at run time through a CLI?

dgsudharsan · 2023-06-02T02:18:00Z

doc/cli_wrapper/shd-cli_wrapper_hld.md

+
+  * TODO:
+
+    * Dynamically generating wrapped files


How will this feature work with auto CLI generation feature? Please check https://github.com/sonic-net/SONiC/blob/master/doc/cli_auto_generation/cli_auto_generation.md

dgsudharsan · 2023-06-02T02:19:32Z

doc/cli_wrapper/shd-cli_wrapper_hld.md

+  * TODO:
+
+    * Dynamically generating wrapped files
+


Please also check the interoperability with GCU feature. Since GCU loads the yang models once for every command, can we think about loading the entire yang only once and then validating?

zhangyanzhao · 2023-06-12T06:34:09Z

@dgsudharsan registered as reviewer of this feature.

zhangyanzhao · 2024-02-05T08:55:51Z

@inspurSDN can you please help to add the code PR by referring to #806? Thanks.

zhangyanzhao · 2024-05-06T02:36:54Z

code PR is not ready, move to backlog for future release

[Inspur][HLD] Add cli_wrapper hld

db969b7

Description: cli_wrapper high-level design

ganglyu requested review from qiluo-msft, ganglyu, lguohan and richardyu-ms September 30, 2022 11:13

lihuay and others added 5 commits October 2, 2022 09:18

Merge branch 'master' into cli_wrapper

8d2876e

[Inspur][HLD] Modify cli_wrapper hld

6b398e3

Description: Add test results

[Inspur][HLD] Modify cli_wrapper hld

73cc33d

Description: Update test result

Update cli_wrapper HLD

79ab09b

Update cli_wrapper HLD

cf22c13

zhangyanzhao removed the request for review from richardyu-ms May 30, 2023 15:20

jeffhtgt reviewed May 30, 2023

View reviewed changes

zhangyanzhao requested a review from venkatmahalingam May 30, 2023 16:01

venkatmahalingam reviewed May 31, 2023

View reviewed changes

dgsudharsan suggested changes Jun 2, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Inspur][HLD] Add cli_wrapper hld #1091

[Inspur][HLD] Add cli_wrapper hld #1091

inspurSDN commented Sep 30, 2022

zhangyanzhao commented May 30, 2023

jeffhtgt May 30, 2023

zhangyanzhao commented May 30, 2023

zhangyanzhao commented May 30, 2023

venkatmahalingam May 31, 2023

venkatmahalingam May 31, 2023

venkatmahalingam May 31, 2023 •

edited

Loading

venkatmahalingam May 31, 2023

venkatmahalingam May 31, 2023 •

edited

Loading

venkatmahalingam May 31, 2023

venkatmahalingam May 31, 2023 •

edited

Loading

venkatmahalingam May 31, 2023

dgsudharsan Jun 1, 2023

dgsudharsan Jun 2, 2023

dgsudharsan Jun 2, 2023

dgsudharsan Jun 2, 2023

dgsudharsan Jun 2, 2023

zhangyanzhao commented Jun 12, 2023

zhangyanzhao commented Feb 5, 2024

zhangyanzhao commented May 6, 2024


		* Disable CLI wrapper service

		![Encode format](images/wrapper_disable.jpg)


		CLI_wrapper server is responsible for the execution of click command. It starts when system boots up, and loads all needed libraries to server. Then setup a socket to wait client's request. Once it accepts a request, it spawns a CLI_worker process to deal with this request.

		In order to avoid that some command may stuck in CLI worker and cause resource leakage, a watchdog thread is designed to watch the execution time of CLI worker, once the execution time exceeds maximum execution time, the CLI worker will be gracefully terminated. But if the CLI worker is still alive after one minute, a kill signal will be sent to CLI worker to tear down it.


		In order to avoid that some command may stuck in CLI worker and cause resource leakage, a watchdog thread is designed to watch the execution time of CLI worker, once the execution time exceeds maximum execution time, the CLI worker will be gracefully terminated. But if the CLI worker is still alive after one minute, a kill signal will be sent to CLI worker to tear down it.

		The local DB is used to store the record of spawned CLI worker, it records the steady timestamp and pid of the CLI worker.


		![Command execution flow](images/command_execution_flow.jpg)

		* System ctrl bring up CLI_wrapper server when system starts


		### 1.3.2 Command execution flow

		![Command execution flow](images/command_execution_flow.jpg)

		* CLI worker decodes the command to get parameters and environment setting, then execute this command based on those settings. In the end, it sends return code and data back to CLI wrapper client

		### 1.3.3 Watchdog execution flow


		* For less powerful CPUs, cli_wrapper can significantly reduce execution time.

		# 3. Open questions


		* CLI_wrapper server

		CLI_wrapper server is responsible for the execution of click command. It starts when system boots up, and loads all needed libraries to server. Then setup a socket to wait client's request. Once it accepts a request, it spawns a CLI_worker process to deal with this request.


		![Encode format](images/wrapper_enable.jpg)

		1. Replace main.py with wrapped main.py

[Inspur][HLD] Add cli_wrapper hld #1091

Are you sure you want to change the base?

[Inspur][HLD] Add cli_wrapper hld #1091

Conversation

inspurSDN commented Sep 30, 2022

zhangyanzhao commented May 30, 2023

Choose a reason for hiding this comment

zhangyanzhao commented May 30, 2023

zhangyanzhao commented May 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

venkatmahalingam May 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

venkatmahalingam May 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

venkatmahalingam May 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhangyanzhao commented Jun 12, 2023

zhangyanzhao commented Feb 5, 2024

zhangyanzhao commented May 6, 2024

venkatmahalingam May 31, 2023 •

edited

Loading

venkatmahalingam May 31, 2023 •

edited

Loading

venkatmahalingam May 31, 2023 •

edited

Loading