-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
database-chassis python http-server implementation #13345
base: master
Are you sure you want to change the base?
Changes from all commits
a4bff50
a99715b
221bb96
7185cf6
a1397a8
4130990
bd9d920
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,119 @@ | ||
#!/usr/bin/python3 | ||
# | ||
########################################################################### | ||
# http-server | ||
# | ||
# Python HTTP Server service. | ||
# This service runs based on the configuration provided in | ||
# device/<platform>/chassisdb.conf | ||
# | ||
# Configuration parameters provided by chassisdb.conf are as follows: | ||
# | ||
# start_http_server: if "yes" or "1", http-server will be started on this | ||
# node | ||
# chassis_db_address : IP address for http-server. This is the same IP used | ||
# for the chassis db server | ||
# http_server_port : Port to bind to, default: 8000 | ||
# http_server_dir : HTTP server home directory path, default: /var/www/ | ||
# | ||
########################################################################### | ||
# Copyright (c) 2021-2022 Cisco Systems, Inc. and its affiliates. | ||
# All rights reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
########################################################################### | ||
|
||
# Example of config: | ||
''' | ||
start_http_server=yes | ||
chassis_db_address=127.0.0.10 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is an example of the config. The chassis_db_address is used for redis_chassis.server and is hardcoded as of today, provided by chassisd.conf file by vendor in dir. |
||
http_server_port=8000 | ||
http_server_dir=/var/www/tftp/ | ||
''' | ||
|
||
import argparse | ||
import http.server | ||
import os | ||
import socketserver | ||
import subprocess | ||
import syslog | ||
|
||
from pathlib import Path | ||
|
||
HTTP_DEFAULT_BIND_PORT = "8000" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I will investigate more on this and respond/implement. Thanks There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @qiluo-msft your point is to have Control plane acl (ip table rules) to explicitly permit this ? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yes, I believe you also want to limit the access, for example by some rules on 5-tuple There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @abdosi , @qiluo-msft , this port is on the internal network. Do we have any rules on other internal ip/ports which I can refer to? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hi @qiluo-msft , Abhishek and I had a discussion on this. The current Sonic implementation does not have any rule defined for the midplane subnet yet and we need to have a rule for all kinds of access in the midplane subnet. We will have to define platform subnet parameters that caclmgrd could use to define the rules for midplane access for all kinds of midplane traffic in the chassis. I will open a case and we can use a different PR to define the general subnet access rule. Let me know if that is fine. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @qiluo-msft on chassis setup we have additional interface There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Created PR : sonic-net/sonic-host-services#42 . With this PR any port 8000 traffic that is not coming on eth1-midplane will get drop and thus protected from external network access. |
||
HTTP_DEFAULT_DIR_PATH = '/var/www' | ||
http_dir = None | ||
|
||
class HTTPRequestHandler(http.server.SimpleHTTPRequestHandler): | ||
def __init__(self, *args, **kwargs): | ||
super().__init__(*args, directory=http_dir, **kwargs) | ||
|
||
class HttpServer(): | ||
''' HTTP server class implementation ''' | ||
|
||
def __init__(self, args, **kw): | ||
self.http_config = {} | ||
self.start_http_server = None | ||
|
||
if args.config_file: | ||
# Use config file provided from command line | ||
platform_conf = args.config_file | ||
else: | ||
platform_conf=os.path.join('/usr/share/sonic/platform', 'chassisdb.conf') | ||
|
||
syslog.syslog(syslog.LOG_INFO,'HTTP Server config file:{}'.format(platform_conf)) | ||
if os.path.isfile(platform_conf): | ||
with open(platform_conf) as f: | ||
for line in f.readlines(): | ||
(key, _, value) = line.strip().replace('"','').partition("=") | ||
self.http_config[key] = value | ||
else: | ||
syslog.syslog(syslog.LOG_INFO, 'HTTP server config file {} not present - exiting...'.format(platform_conf)) | ||
exit(0) | ||
|
||
self.http_server_ip = self.http_config.get('chassis_db_address') | ||
if self.http_server_ip: | ||
global http_dir | ||
self.start_http_server = self.http_config.get('start_http_server') | ||
http_dir = self.http_config.get('http_server_dir', HTTP_DEFAULT_DIR_PATH) | ||
|
||
# Get Server port from config | ||
self.http_server_port = int(self.http_config.get('http_server_port', HTTP_DEFAULT_BIND_PORT)) | ||
|
||
# Create HTTP home dir path if not present | ||
path = Path(http_dir) | ||
path.mkdir(parents=True, exist_ok=True) | ||
|
||
def run(self): | ||
''' start the http-server if start_http_server is set in config file ''' | ||
|
||
if self.start_http_server == 'yes' or self.start_http_server == '1': | ||
# Start the http-server, generally on the Supervisor card. | ||
syslog.syslog(syslog.LOG_INFO, 'start_http_server is set, starting http-server') | ||
if self.http_config: | ||
with socketserver.TCPServer((self.http_server_ip, self.http_server_port), HTTPRequestHandler) as httpd: | ||
syslog.syslog(syslog.LOG_INFO, 'HTTP Server Port:{} home:{}'.format(self.http_server_port, http_dir)) | ||
httpd.serve_forever() | ||
else: | ||
syslog.syslog(syslog.LOG_INFO, 'start_http_server is not set, exiting...') | ||
|
||
def main(): | ||
parser = argparse.ArgumentParser(description='Python3 HTTP Server') | ||
parser.add_argument('--config-file', help='HTTP Server config yaml file', default=None) | ||
args = parser.parse_args() | ||
|
||
HttpServer(args).run() | ||
|
||
if __name__ == '__main__': | ||
main() |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -56,3 +56,23 @@ stdout_logfile=syslog | |
stderr_logfile=syslog | ||
dependent_startup=true | ||
dependent_startup_wait_for=rsyslogd:running | ||
|
||
{% if INSTANCES %} | ||
{% for redis_inst, redis_items in INSTANCES.items() %} | ||
{%- if redis_inst == 'redis_chassis' %} | ||
[program: http-server] | ||
; http-server should run only on database-chassis which hosts redis_chassis server. | ||
; Using 2 sec sleep in command to give the process enough run time to reach | ||
; RUNNING state when the process needs to exit with status 0 when start_http_server | ||
; config is not set. | ||
command=/bin/bash -c "sleep 2 && python3 /usr/local/bin/http-server" | ||
priority=3 | ||
autostart=true | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The process starts when the database chassis starts but the server will run only if start_http_server flag is set in chassisd.conf file by the vendor. If the flag is not set, the http-server won't run and this service will exit. autorestart is false. |
||
autorestart=unexpected | ||
stdout_logfile=syslog | ||
stderr_logfile=syslog | ||
dependent_startup=true | ||
dependent_startup_wait_for=rsyslogd:running | ||
{%- endif %} | ||
{%- endfor %} | ||
{% endif %} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why put this service in database container? I do not see any dependency
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My initial PR was to run this as host side service as a feature but in the last review in Chassis group, it was decided to use it inside database-chassis for now as it's only for internal image/data hosting for chassis modules. It runs only on Supervisor on Chassis and binds to same internal ip as redis_chassis.server.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@qiluo-msft yes we had discussion on this and if we want to run on host there was comment from Guohan to add this as Feature in Feature Table. But that was little deviation from current model where each feature is like docker with we can enable/disable the feature by doing
container stop/start
. So in chassis subgroup we discussed and thought since that will be some infrastructure change for this scenario to unblock chassis use case we thought to make it part ofdatabase-chassis
docker which is specific to chassis use case. Also in future if this is needed in host we can revisit.