config reload command should check if the system status is ready before execution

[dgsudharsan]

Description

Currently config reload command doesn't check if the system is up before executing. If config reload is given immediately after reboot when services are booting up, it leads to issues with some services.

Example when config reload is given between snmp.timer trigger but before snmp spawns, snmp will never start as the timer will not trigger after 'config reload' as it had triggered once and snmp in order to start after 'config reload' should have already executed once which didn't happen.

Steps to reproduce the issue:

1. reboot the device
2. Perform config reload

Describe the results you received:

config reload proceeds even with the system completely up.

Describe the results you expected:

config reload should fail and only succeed when the system is up.

Output of show version:

show version

SONiC Software Version: SONiC.SONIC.202012.52-50e4cc15_Internal
Distribution: Debian 10.8
Kernel: 4.19.0-12-2-amd64
Build commit: 50e4cc15
Build date: Tue Mar 23 10:25:18 UTC 2021
Built by: sw-r2d2-bot@r-build-sonic-ci03

Platform: x86_64-mlnx_msn3800-r0
HwSKU: ACS-MSN3800
ASIC: mellanox
ASIC Count: 1
Serial Number: MT1937X00537
Uptime: 05:12:35 up 11 min,  2 users,  load average: 0.58, 0.77, 0.60

Docker images:
REPOSITORY                    TAG                                 IMAGE ID            SIZE
docker-teamd                  SONIC.202012.52-50e4cc15_Internal   e404a957f154        412MB
docker-teamd                  latest                              e404a957f154        412MB
docker-nat                    SONIC.202012.52-50e4cc15_Internal   060b170d465d        415MB
docker-nat                    latest                              060b170d465d        415MB
docker-platform-monitor       SONIC.202012.52-50e4cc15_Internal   866984132b08        693MB
docker-platform-monitor       latest                              866984132b08        693MB
docker-orchagent              SONIC.202012.52-50e4cc15_Internal   2da47edc09a5        431MB
docker-orchagent              latest                              2da47edc09a5        431MB
docker-fpm-frr                SONIC.202012.52-50e4cc15_Internal   12a1712803da        431MB
docker-fpm-frr                latest                              12a1712803da        431MB
docker-sflow                  SONIC.202012.52-50e4cc15_Internal   6205a14bb9df        413MB
docker-sflow                  latest                              6205a14bb9df        413MB
docker-wjh                    202012.202012.0-e663b63             bbf69da66de2        441MB
docker-wjh                    latest                              bbf69da66de2        441MB
docker-syncd-mlnx             SONIC.202012.52-50e4cc15_Internal   f5386fd640ba        666MB
docker-syncd-mlnx             latest                              f5386fd640ba        666MB
docker-dhcp-relay             SONIC.202012.52-50e4cc15_Internal   cb5a7323aeb9        409MB
docker-dhcp-relay             latest                              cb5a7323aeb9        409MB
docker-snmp                   SONIC.202012.52-50e4cc15_Internal   8df6c0d14af7        438MB
docker-snmp                   latest                              8df6c0d14af7        438MB
docker-sonic-mgmt-framework   SONIC.202012.52-50e4cc15_Internal   4881ef3e8dc7        616MB
docker-sonic-mgmt-framework   latest                              4881ef3e8dc7        616MB
docker-router-advertiser      SONIC.202012.52-50e4cc15_Internal   7e59764069ca        398MB
docker-router-advertiser      latest                              7e59764069ca        398MB
docker-lldp                   SONIC.202012.52-50e4cc15_Internal   33da46fe8a7f        438MB
docker-lldp                   latest                              33da46fe8a7f        438MB
docker-database               SONIC.202012.52-50e4cc15_Internal   e65ccc177592        398MB
docker-database               latest                              e65ccc177592        398MB
docker-sonic-telemetry        SONIC.202012.52-50e4cc15_Internal   5e079afb02ab        487MB
docker-sonic-telemetry        latest                              5e079afb02ab        487MB

Additional information you deem important (e.g. issue happens only occasionally):

[liat-grozovik]

@qiluo-msft could you please take a look?

[abdosi]

will having this check https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-host-services/scripts/hostcfgd#L612 helps ?

[dgsudharsan]

will having this check https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-host-services/scripts/hostcfgd#L612 helps ?

This doesn't help as the above command returns running even before tasks like snmp are spawned(which have delayed start)
sudo systemctl is-system-running
running
root@sonic:~#