database-chassis python http-server implementation #13345

anamehra · 2023-01-12T05:43:13Z

Signed-off-by: anamehra anamehra@cisco.com

  Python HTTP Server service.
  This service runs in database-chassis docker based on the configuration
  provided in device/<platform>/chassisdb.conf

  Configuration parameters provided by chassisdb.conf are as follows:

  start_http_server: if "yes" or "1", http-server will be started on this
      node
  chassis_db_address   : IP address for http-server. This is the same IP used
   for the chassis db server
  http_server_port : Port to bind to, default: 8000
  http_server_dir  : HTTP server home directory path, default: /var/www/

# Example of config:
'''
start_http_server=yes
chassis_db_address=127.0.0.10
http_server_dir=/var/www/tftp
'''

Why I did it

Enable http-server as database-chassis service on Chassis supervisor to enable LCs to download image and config files from Supervisor.

How I did it

Added python3 http-server based implementation in database-chassis as a docker service run by supervisord.

How to verify it

Add device//chassisdb.conf with the following data:
start_http_server=yes
chassis_db_address=127.0.0.10
http_server_dir=/var/www/tftp

Boot with the image with the above data.

put an image file at /var/www/tftp/filename
check 'systemctl status http-service'
curl http://http-server:8000/filename -o out_file
compare file_out with /var/www/tftp/filename

Which release branch to backport (provide reason below if selected)

Description for the changelog

Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Python HTTP Server service. This service runs in database-chassis docker based on the configuration provided in device/<platform>/chassisdb.conf Configuration parameters provided by chassisdb.conf are as follows: start_http_server: if "yes" or "1", http-server will be started on this node chassis_db_address : IP address for http-server. This is the same IP used for the chassis db server http_server_port : Port to bind to, default: 8000 http_server_dir : HTTP server home directory path, default: /var/www/ Signed-off-by: anamehra <anamehra@cisco.com>

anamehra · 2023-01-12T05:45:57Z

@abdosi , FYI-
Thanks

abdosi

LGTM

abdosi · 2023-01-19T21:01:19Z

@arlakshm / @judyjoseph please help review.

anamehra · 2023-01-27T18:14:41Z

@judyjoseph , @arlakshm , please let me know if you have any comments. I resolved all comments from our last meeting discussion in this PR. moved the http-server in database-chassis docker for chassis.

dockers/docker-database/http-server

qiluo-msft · 2023-02-04T04:39:51Z

dockers/docker-database/supervisord.conf.j2

+
+{% if INSTANCES %}
+{%     for redis_inst, redis_items in INSTANCES.items() %}
+{% if redis_inst == 'redis_chassis' %}


if

Need 2 level indentation #Closed

Thanks for the review @qiluo-msft !
I will fix this.

qiluo-msft · 2023-02-04T04:42:50Z

dockers/docker-database/Dockerfile.j2

@@ -44,5 +44,6 @@ COPY ["files/sysctl-net.conf", "/etc/sysctl.d/"]
 COPY ["critical_processes", "/etc/supervisor"]
 COPY ["files/update_chassisdb_config", "/usr/local/bin/"]
 COPY ["flush_unused_database", "/usr/local/bin/"]
+COPY ["http-server", "/usr/local/bin/"]


http-server

Why put this service in database container? I do not see any dependency

My initial PR was to run this as host side service as a feature but in the last review in Chassis group, it was decided to use it inside database-chassis for now as it's only for internal image/data hosting for chassis modules. It runs only on Supervisor on Chassis and binds to same internal ip as redis_chassis.server.

@qiluo-msft yes we had discussion on this and if we want to run on host there was comment from Guohan to add this as Feature in Feature Table. But that was little deviation from current model where each feature is like docker with we can enable/disable the feature by doing container stop/start. So in chassis subgroup we discussed and thought since that will be some infrastructure change for this scenario to unblock chassis use case we thought to make it part of database-chassis docker which is specific to chassis use case. Also in future if this is needed in host we can revisit.

qiluo-msft · 2023-02-04T04:46:00Z

dockers/docker-database/http-server

+# Example of config:
+'''
+start_http_server=yes
+chassis_db_address=127.0.0.10


127.0.0.10

hard-coded IP? #Closed

This is an example of the config. The chassis_db_address is used for redis_chassis.server and is hardcoded as of today, provided by chassisd.conf file by vendor in dir.
This http-serevr use the same IP to bind to service requests on internal midplane network.

qiluo-msft · 2023-02-04T04:46:37Z

dockers/docker-database/supervisord.conf.j2

+[program: http-server]
+command=python3 /usr/local/bin/http-server
+priority=3
+autostart=true


true

no way to disable it #Closed

The process starts when the database chassis starts but the server will run only if start_http_server flag is set in chassisd.conf file by the vendor. If the flag is not set, the http-server won't run and this service will exit. autorestart is false.

qiluo-msft · 2023-02-04T04:47:01Z

dockers/docker-database/http-server

+
+from pathlib import Path
+
+HTTP_DEFAULT_BIND_PORT = "8000"


8000

Do we have rules to protect this service? #Pending

I will investigate more on this and respond/implement. Thanks

@qiluo-msft your point is to have Control plane acl (ip table rules) to explicitly permit this ?

yes, I believe you also want to limit the access, for example by some rules on 5-tuple

@abdosi , @qiluo-msft , this port is on the internal network. Do we have any rules on other internal ip/ports which I can refer to?

Hi @qiluo-msft , Abhishek and I had a discussion on this. The current Sonic implementation does not have any rule defined for the midplane subnet yet and we need to have a rule for all kinds of access in the midplane subnet. We will have to define platform subnet parameters that caclmgrd could use to define the rules for midplane access for all kinds of midplane traffic in the chassis. I will open a case and we can use a different PR to define the general subnet access rule. Let me know if that is fine.

@qiluo-msft on chassis setup we have additional interface eth1-midplane both on supervisor and LC for internal traffic (eg: redis and http) . We will add iptable rule to allow all traffic on that interface. That way http traffic will be protected from outside network (eth0) and will get permit only if coming from internal network. I will create separate for that

Created PR : sonic-net/sonic-host-services#42 . With this PR any port 8000 traffic that is not coming on eth1-midplane will get drop and thus protected from external network access.

abdosi · 2023-02-22T22:29:38Z

@qiluo-msft can you please review this again.

If the start_http_server is not set, the process exit with status 0 but it exits before reaching RUNNING state which happens after running for more than a sec. In such a scenario, supervisord restarts the process and mark state as FATAL. For exit with status 0, we do not want to restart the process. Fixed indentation

gechiang · 2023-07-13T00:24:38Z

@anamehra , can you resolve the conflict since the changes has been a while and require rebase/merge to resolve conflicts.
We can then ask @qiluo-msft to approve/merge.
Thanks!

abdosi · 2023-07-13T00:36:00Z

@anamehra , can you resolve the conflict since the changes has been a while and require rebase/merge to resolve conflicts. We can then ask @qiluo-msft to approve/merge. Thanks!

we don;t need this for 202205. This will gets merged to master directly.

Signed-off-by: anamehra <anamehra@cisco.com>

anamehra · 2023-07-13T03:02:48Z

@anamehra , can you resolve the conflict since the changes has been a while and require rebase/merge to resolve conflicts. We can then ask @qiluo-msft to approve/merge. Thanks!

resolved. Thanks!

arlakshm · 2023-11-29T18:40:43Z

Lower priority now. Will revisit in the future

anamehra requested a review from lguohan as a code owner January 12, 2023 05:43

This was referenced Jan 12, 2023

python http-server host service implementation sonic-net/sonic-host-services#17

Closed

python http-server host service implementation #11987

Closed

abdosi previously approved these changes Jan 19, 2023

View reviewed changes

abdosi requested review from arlakshm and judyjoseph January 19, 2023 20:59

abdosi added Request for 202205 Branch Chassis 🤖 Modular chassis support labels Jan 19, 2023

judyjoseph reviewed Feb 1, 2023

View reviewed changes

dockers/docker-database/http-server Outdated Show resolved Hide resolved

Removed commented code

a99715b

anamehra requested review from judyjoseph and removed request for lguohan and arlakshm February 1, 2023 16:05

rlhui assigned anamehra and abdosi Feb 1, 2023

judyjoseph previously approved these changes Feb 2, 2023

View reviewed changes

qiluo-msft reviewed Feb 4, 2023

View reviewed changes

abdosi added the Request for 202211 Branch label Feb 15, 2023

anamehra mentioned this pull request Mar 16, 2023

Sonic Chassis http-server HLD sonic-net/SONiC#1289

Open

Make /var/www accessible from database docker

7185cf6

anamehra requested a review from xumia as a code owner March 31, 2023 15:54

liat-grozovik dismissed stale reviews from judyjoseph and abdosi via 7185cf6 April 13, 2023 11:18

Merge branch 'sonic-net:master' into anamehra/http_server_chassisdb

a1397a8

gechiang requested a review from qiluo-msft June 23, 2023 05:32

gechiang removed the Request for 202205 Branch label Jun 23, 2023

anamehra and others added 2 commits July 12, 2023 20:00

Resolved conflict

4130990

Signed-off-by: anamehra <anamehra@cisco.com>

Merge branch 'master' into anamehra/http_server_chassisdb

bd9d920

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

database-chassis python http-server implementation #13345

database-chassis python http-server implementation #13345

anamehra commented Jan 12, 2023 •

edited

Loading

anamehra commented Jan 12, 2023

abdosi left a comment

abdosi commented Jan 19, 2023

anamehra commented Jan 27, 2023

qiluo-msft Feb 4, 2023 •

edited

Loading

anamehra Feb 8, 2023

anamehra Feb 23, 2023

qiluo-msft Feb 4, 2023

anamehra Feb 8, 2023

abdosi Feb 9, 2023

qiluo-msft Feb 4, 2023 •

edited

Loading

anamehra Feb 8, 2023

qiluo-msft Feb 4, 2023 •

edited

Loading

anamehra Feb 8, 2023

qiluo-msft Feb 4, 2023 •

edited

Loading

anamehra Feb 8, 2023

abdosi Feb 9, 2023

qiluo-msft Feb 9, 2023

anamehra Feb 11, 2023

anamehra Feb 15, 2023

abdosi Feb 15, 2023

abdosi Feb 15, 2023

abdosi commented Feb 22, 2023

gechiang commented Jul 13, 2023

abdosi commented Jul 13, 2023

anamehra commented Jul 13, 2023

arlakshm commented Nov 29, 2023

database-chassis python http-server implementation #13345

Are you sure you want to change the base?

database-chassis python http-server implementation #13345

Conversation

anamehra commented Jan 12, 2023 • edited Loading

Why I did it

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

anamehra commented Jan 12, 2023

abdosi left a comment

Choose a reason for hiding this comment

abdosi commented Jan 19, 2023

anamehra commented Jan 27, 2023

qiluo-msft Feb 4, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qiluo-msft Feb 4, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qiluo-msft Feb 4, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qiluo-msft Feb 4, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abdosi commented Feb 22, 2023

gechiang commented Jul 13, 2023

abdosi commented Jul 13, 2023

anamehra commented Jul 13, 2023

arlakshm commented Nov 29, 2023

anamehra commented Jan 12, 2023 •

edited

Loading

qiluo-msft Feb 4, 2023 •

edited

Loading

qiluo-msft Feb 4, 2023 •

edited

Loading

qiluo-msft Feb 4, 2023 •

edited

Loading

qiluo-msft Feb 4, 2023 •

edited

Loading