Support OS9 -> SONiC fast-reboot migration #1414

padmanarayana · 2018-02-23T02:54:23Z

- What I did

Add support for OS9 to SONiC migration in fast-reboot mode

- How I did it

Disable dhcp during migration for mgmt. interface – so the initial /etc/network/interfaces will set eth0 to static (note that this is early init before the rc.local). This saves 1 minute of DHCPDISCOVERY retries..
In rc.local, if this is the case of migration:
a. Extract the NOS (OS9) partition from the cmdline (that stores the migration artifacts)
b. Set the mac for eth0 based on OS9 setting
c. Save the arp and fdb.jsons for use by swss
d. Initializes the migration.xml to generate config_db
e. On a non-migration reboot, clean up the jsons.
Swssconfig: Import the migration arp and fdb.jsons if these files exist.

- How to verify it

Perform a customised version of the PTF ftos-fastreboot test (for OS9 -> SONiC migration)

- Description for the changelog
Add support for OS9 to SONiC migration in fast-reboot mode

- A picture of a cute animal (not mandatory but encouraged)

padmanarayana · 2018-02-23T02:55:56Z

files/image_config/platform/rc.local

+        mkdir -p /etc/sonic/migration
+
+        # Copy relevant files
+        nos_migration_import /mnt/nos_migration/minigraph.xml /etc/sonic/migration


We could import the entire bunch of files 1 shot - but this is just to ensure we know what we are importing.

padmanarayana · 2018-02-23T02:59:04Z

files/image_config/platform/rc.local

    migration="TRUE"
    umount /mnt/onie-boot
+else
+    # this is a reboot post NOS migration : remove stale imported files that were used during migration.
+    rm -fr /etc/sonic/migration


Alternatively, we could remove the arp.json & fdb.json after checking that orchagent initialized but removing here localizes the migration changes to rc.local.

lguohan · 2018-02-23T07:25:36Z

build_debian.sh

@@ -315,7 +315,7 @@ sudo tee -a $FILESYSTEM_ROOT/etc/network/interfaces > /dev/null <<EOF

 auto eth0
 allow-hotplug eth0
-iface eth0 inet dhcp
+iface eth0 inet static


I thought we are not going to disable dhcp globally? the dhcp is disabled only from ftos to sonic fast-boot migration?

That was the earlier thought. But I could not disable this during migration since squashfs is readonly... So, changed the approach. Note that with this change, though we disable dhcp globally during the build, if there is no IP configured in the minigraph, the interfaces-config.sh would override the startup /etc/network/interfaces with a config that is set to dhcp.

you do not need to change squash fs, you can change the entry after you have mounted the union fs.

lguohan · 2018-02-23T07:26:40Z

@pavel-shirshov , can you code-review this?

pavel-shirshov

As commented.

pavel-shirshov · 2018-02-23T20:13:22Z

files/image_config/platform/rc.local

+        nos_migration_import /mnt/nos_migration/arp.json /etc/sonic/migration
+        nos_migration_import /mnt/nos_migration/fdb.json /etc/sonic/migration
+        nos_migration_import /mnt/nos_migration/mgmt_interface.mac /etc/sonic/migration
+


Can you please put your files into /host/fast-reboot directory?
See following changes in fast-reboot script: sonic-net/sonic-utilities@461470b

In this case you don't need any changes in swssconfig.sh

pavel-shirshov · 2018-02-23T20:13:52Z

dockers/docker-orchagent/swssconfig.sh

@@ -29,6 +41,9 @@ function fast_reboot {
  esac
 }

+# If we are migrating, import the dumps from the previous NOS
+fast_migration
+


You don't need to change this file. See my comments below

pavel-shirshov · 2018-02-23T20:15:15Z

files/image_config/platform/rc.local

@@ -11,6 +11,16 @@
 #
 # By default this script does nothing.

+# In case the unit is migrating from another NOS, save the logs
+log_migration() {
+    echo $1 >> /etc/migration.log


better to save it to /var/log?
Any other place, not /etc?

pavel-shirshov · 2018-02-23T20:18:21Z

files/image_config/platform/rc.local

+       if [ `echo $eth0_mac | egrep "^([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}$"` ]; then
+           ifconfig eth0 down
+           ip link set eth0 address $eth0_mac
+           ifconfig eth0 up


please use iptools2 commands

pavel-shirshov · 2018-02-23T20:21:21Z

files/image_config/platform/rc.local

+        # Copy relevant files
+        nos_migration_import /mnt/nos_migration/minigraph.xml /etc/sonic/migration
+        nos_migration_import /mnt/nos_migration/arp.json /etc/sonic/migration
+        nos_migration_import /mnt/nos_migration/fdb.json /etc/sonic/migration


Please add support for default routes in default_routes.json.
See here: sonic-net/sonic-utilities@461470b#diff-fcb2d524807024c9b0cd2e2850088f64R258

padmanarayana

Create /host/migration which will contain:
a. migration logs
b. mgmt_interfaces.cfg (previously mgmt_interfaces.mac)
c. minigraph.
Revert changes to swssconfig.sh. the jsons are copied into /host/fast-reboot.
In order to make the eth0 (and switchport MACs) persistent after the migration reboot, write the NOS's mac into the GbE EEPROM so that igb driver can use it post the migration reboot. Added ethtool for writing MAC to EEPROM.
Also, update the 70-persistent-net.rules to fix the eth0 mac after migration.

padmanarayana · 2018-02-27T06:11:42Z

Please review changes.

pavel-shirshov

It looks ok for me

pavel-shirshov · 2018-02-27T20:07:47Z

files/image_config/platform/rc.local

+    [ -f $1 ] && cp $1 $2 || log_migration "ERROR: $1 not found!"
+}
+
+# While migrating form another NOS, we need to preserve the MAC addresses


form -> from

pavel-shirshov · 2018-02-27T20:12:21Z

files/image_config/platform/rc.local

+    ethtool_magic=$(grep "ethtool_magic" $mgmt_config | awk -F'=' '{print $2}') 
+    ethtool_offset=$(grep "ethtool_offset" $mgmt_config | awk -F'=' '{print $2}') 
+    if [ -z "$ethtool_magic" ] || [ -z "$ethtool_offset" ]; then
+        log_migration "Unable to retrieve ethtool params ($ethtool_magic,ethtool_offset)"


missed $ in the error message?

pavel-shirshov · 2018-02-27T20:14:21Z

files/image_config/platform/rc.local

+        rmdir /mnt/nos_migration
+    fi
+
+    update_mgmt_interface_macaddr /host/migration/mgmt_interface.cfg


Are we sure this will work for any platform?

There is an assumption that the platform supports ethtool to modify the eth (GbE in S6100/Z9100) case using igb. If the platform does not support ethtool to change the EEPROM, the alternative (workaround) is to note that the unit has migrated and apply the previous NOS' mac to eth0 (using ip link set) everytime the unit reboots...

padmanarayana · 2018-02-28T13:39:35Z

Takes care of review comments in rc.local & fix a script error (missing fi)
Revert the static change back to dhcp in build_debian.sh : and introduce a initramfs-tools/mgmt-intf-dhcp that will change the setting in /etc/network/interfaces iff the linux boot line has mgmt.-intf-dhcp=off
Update sonic-platform-modules-dell

padmanarayana

Please review !

pavel-shirshov

Where do you use mgmt-intf-dhcp?

Also I suggest to use
https://www.shellcheck.net/ sometime this utility shows hidden issues

pavel-shirshov · 2018-02-28T21:16:27Z

files/initramfs-tools/mgmt-intf-dhcp

@@ -0,0 +1,36 @@
+#!/bin/sh


Better to use /bin/bash here

Thanks for the shellcheck! I want to use sh as is the convention with other initramfs scripts..

pavel-shirshov · 2018-02-28T21:17:32Z

files/initramfs-tools/mgmt-intf-dhcp

+            fi
+
+            if [ -e "${rootmnt}/etc/network/interfaces" ]; then
+                if [ "$val" == "off" ]; then


== is undefined in /bin/sh
Better use =

pavel-shirshov · 2018-02-28T21:17:55Z

files/initramfs-tools/mgmt-intf-dhcp

+            if [ -e "${rootmnt}/etc/network/interfaces" ]; then
+                if [ "$val" == "off" ]; then
+                    sed -i 's/iface eth0 inet dhcp/iface eth0 inet static/g' ${rootmnt}/etc/network/interfaces
+                elif [ "$val" == "on" ]; then


==
The same as previous

padmanarayana

Please review changes!

lguohan · 2018-03-07T08:37:15Z

files/image_config/platform/rc.local

+    elif [ -n "$migration" ] && [ -f /host/migration/minigraph.xml ];  then
+        # Use the minigraph that was imported from the NOS
+        mv /host/migration/minigraph.xml /etc/sonic/
+        sonic-cfggen -m -j /etc/sonic/init_cfg.json --print-data > /etc/sonic/config_db.json


lguohan · 2018-03-07T08:50:55Z

files/image_config/platform/rc.local

+    # Update the 70-persistent-net.rules with the new mac for eth0
+    log_migration "/etc/udev/rules.d/70-persistent-net.rules : replacing $old_mac with $new_mac for eth0"
+    sed -i "/eth0/ s/ATTR{address}==\"$old_mac\"/ATTR{address}==\"$new_mac\"/g" /etc/udev/rules.d/70-persistent-net.rules
+}


for sonic, we are using the same mac for management interface and vlan interface. are you using the same mac for you management interface and vlan interface? The goal here is to make sure your vlan interface has the same mac as our vlan interface.

lguohan

as comments

lguohan

as comments

[portsorch] add buffer drop FC group [bitmap_vnet] Fix VNET route priority issue (#1421) [vnet] Maintain the reference count of the nexthop when creating a vn… (#1414) [intfsorch] Retrieve Port object before setting NAT zone on router interfaces. (#1372)

sonic-net#1414) * [vnet] Maintain the reference count of the nexthop when creating a vnet route

19615e3 Fixing db_migrator for Feature table (#1674) d1c1c61 [tests]: skip some dynamic port breakout unit tests (#1677) 25669c3 [CI] sonic-config-engine now depends on SONiC YANG packages (#1675) 3ff68c4 [neighbor-advertiser] delete the tunnel maps appropriately (#1663) a425ca2 [config] support for configuring muxcable to manual mode of operation (#1642) 25e17de [show platform summary] Add chassis hardware info to platform summary and version (#1624) f5f2a00 [db_migrator] fix old 1911 feature config migration to a new one. (#1635) 56db162 [config] Fix config int add incorrect ip (#1414) 1da879c [db_migrator][Mellanox] Update Mellanox buffer migrator with 2km-cable supported (#1564) c2b760f [sonic_package_manager] flush once finished saving docker image into temporary file (#1638) cd69473 Replace swsssdk.ConfigDBConnector and SonicDBConfig with swsscommon implementation (#1620) 5f20365 Change to use rvtysh when calling the show commands (#1572) 51d6bf5 Fix Aboot breakage in sonic package manager in sonic-installer (#1625) 18bed46 [console][show] Force refresh all lines status during show line (#1641) b616cd9 [TPID CONFIG] Added TPID configuration CLI support (#1618) 01eb4b1 [show] support for show muxcable firmware version of only active banks (#1629) 7744c8d [fdb]cli: fdb entries are cleared according to vlan or port or vlan&&port (#657) e23c5ee Add psu hardware revision to psushow table (#1601) f1726fe Make advance_version_for_expected_database available for other db migrator test cases as well (#1614) 5d1ad05 [show] add support for muxcable metrics (#1615) feeab29 [config] Sort Config Db When Saving (#1623)

Update submodule for sonic-utilities to include the following PRs: Make the soft-reboot available in the SONiC image on master (#1681) [config]: Update environment file during config reload (#1673) Enable pr checker form 202012 (#1649) [config] support for configuring muxcable to manual mode of operation (#1642) [config] Fix config int add incorrect ip (#1414) Signed-off-by: Dror Prital <drorp@nvidia.com>

19615e3 Fixing db_migrator for Feature table (sonic-net#1674) d1c1c61 [tests]: skip some dynamic port breakout unit tests (sonic-net#1677) 25669c3 [CI] sonic-config-engine now depends on SONiC YANG packages (sonic-net#1675) 3ff68c4 [neighbor-advertiser] delete the tunnel maps appropriately (sonic-net#1663) a425ca2 [config] support for configuring muxcable to manual mode of operation (sonic-net#1642) 25e17de [show platform summary] Add chassis hardware info to platform summary and version (sonic-net#1624) f5f2a00 [db_migrator] fix old 1911 feature config migration to a new one. (sonic-net#1635) 56db162 [config] Fix config int add incorrect ip (sonic-net#1414) 1da879c [db_migrator][Mellanox] Update Mellanox buffer migrator with 2km-cable supported (sonic-net#1564) c2b760f [sonic_package_manager] flush once finished saving docker image into temporary file (sonic-net#1638) cd69473 Replace swsssdk.ConfigDBConnector and SonicDBConfig with swsscommon implementation (sonic-net#1620) 5f20365 Change to use rvtysh when calling the show commands (sonic-net#1572) 51d6bf5 Fix Aboot breakage in sonic package manager in sonic-installer (sonic-net#1625) 18bed46 [console][show] Force refresh all lines status during show line (sonic-net#1641) b616cd9 [TPID CONFIG] Added TPID configuration CLI support (sonic-net#1618) 01eb4b1 [show] support for show muxcable firmware version of only active banks (sonic-net#1629) 7744c8d [fdb]cli: fdb entries are cleared according to vlan or port or vlan&&port (sonic-net#657) e23c5ee Add psu hardware revision to psushow table (sonic-net#1601) f1726fe Make advance_version_for_expected_database available for other db migrator test cases as well (sonic-net#1614) 5d1ad05 [show] add support for muxcable metrics (sonic-net#1615) feeab29 [config] Sort Config Db When Saving (sonic-net#1623)

sonic-net#1414) * [vnet] Maintain the reference count of the nexthop when creating a vnet route

* Added ip and mask check to config int Signed-off-by: d-dashkov <Dmytro_Dashkov@Jabil.com>

Update SAI submodule v1.9 with the following fixes 7594e53 (HEAD, origin/v1.9) Skip brcm teardown assertion (sonic-net#1423) (sonic-net#1428) 0c33f4a [FIX]Fix the circular reference issue when build sai header py (sonic-net#1427) 7e0fc24 Add support for building under Doxygen 1.9.1 (sonic-net#1414) (sonic-net#1424) 8ecf3ef [Fix]Correct enum check on branch 1.9 (sonic-net#1418) e2b2f39 Add Thrift 0.14.1 compatibility (sonic-net#1403) (sonic-net#1416)

…tically (#19911) #### Why I did it src/sonic-sairedis ``` * 45ff42c3 - (HEAD -> master, origin/master, origin/HEAD) run VS tests in parallel (#1414) (3 hours ago) [Lawrence Lee] ``` #### How I did it #### How to verify it #### Description for the changelog

Support OS9 -> SONiC fast-reboot migration

2a577b8

padmanarayana commented Feb 23, 2018

View reviewed changes

lguohan reviewed Feb 23, 2018

View reviewed changes

lguohan requested a review from pavel-shirshov February 23, 2018 07:26

pavel-shirshov suggested changes Feb 23, 2018

View reviewed changes

Address review comments. Update NOS mac in EEPROM and net.rules for eth0

5a806b8

padmanarayana commented Feb 26, 2018

View reviewed changes

pavel-shirshov reviewed Feb 27, 2018

View reviewed changes

Address review comments. Update sonic-platform-modules-dell to fac81d...

71572c5

padmanarayana commented Feb 28, 2018

View reviewed changes

pavel-shirshov reviewed Feb 28, 2018

View reviewed changes

Fix script for POSIX compliance

68efae0

padmanarayana commented Mar 1, 2018

View reviewed changes

pavel-shirshov approved these changes Mar 1, 2018

View reviewed changes

lguohan reviewed Mar 7, 2018

View reviewed changes

lguohan approved these changes Mar 9, 2018

View reviewed changes

lguohan merged commit 02b62ac into sonic-net:master Mar 9, 2018

raphaelt-nvidia pushed a commit to raphaelt-nvidia/sonic-buildimage that referenced this pull request Jan 14, 2021

[vnet] Maintain the reference count of the nexthop when creating a vn… (

2d3a575

sonic-net#1414) * [vnet] Maintain the reference count of the nexthop when creating a vnet route

alexrallen mentioned this pull request Jun 14, 2021

[sonic-utilities] submodule update #7799

Merged

dprital mentioned this pull request Jun 25, 2021

[sonic-utilities][202012] submodule update #7981

Merged

4 tasks

theasianpianist pushed a commit to theasianpianist/sonic-buildimage that referenced this pull request Feb 5, 2022

[vnet] Maintain the reference count of the nexthop when creating a vn… (

c5d109f

sonic-net#1414) * [vnet] Maintain the reference count of the nexthop when creating a vnet route

taras-keryk pushed a commit to taras-keryk/sonic-buildimage that referenced this pull request Apr 28, 2022

[config] Fix config int add incorrect ip (sonic-net#1414)

56db162

* Added ip and mask check to config int Signed-off-by: d-dashkov <Dmytro_Dashkov@Jabil.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support OS9 -> SONiC fast-reboot migration #1414

Support OS9 -> SONiC fast-reboot migration #1414

padmanarayana commented Feb 23, 2018

padmanarayana Feb 23, 2018

padmanarayana Feb 23, 2018

lguohan Feb 23, 2018

padmanarayana Feb 23, 2018

lguohan Feb 27, 2018

lguohan commented Feb 23, 2018

pavel-shirshov left a comment

pavel-shirshov Feb 23, 2018

pavel-shirshov Feb 23, 2018

pavel-shirshov Feb 23, 2018

pavel-shirshov Feb 23, 2018

pavel-shirshov Feb 23, 2018

padmanarayana left a comment

padmanarayana commented Feb 27, 2018

pavel-shirshov left a comment

pavel-shirshov Feb 27, 2018

pavel-shirshov Feb 27, 2018

pavel-shirshov Feb 27, 2018

padmanarayana Feb 28, 2018

padmanarayana commented Feb 28, 2018

padmanarayana left a comment

pavel-shirshov left a comment •

edited

Loading

pavel-shirshov Feb 28, 2018

padmanarayana Mar 1, 2018

pavel-shirshov Feb 28, 2018

pavel-shirshov Feb 28, 2018

padmanarayana left a comment

lguohan Mar 7, 2018

lguohan Mar 7, 2018

lguohan left a comment

lguohan left a comment

Support OS9 -> SONiC fast-reboot migration #1414

Support OS9 -> SONiC fast-reboot migration #1414

Conversation

padmanarayana commented Feb 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lguohan commented Feb 23, 2018

pavel-shirshov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

padmanarayana left a comment

Choose a reason for hiding this comment

padmanarayana commented Feb 27, 2018

pavel-shirshov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

padmanarayana commented Feb 28, 2018

padmanarayana left a comment

Choose a reason for hiding this comment

pavel-shirshov left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

padmanarayana left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lguohan left a comment

Choose a reason for hiding this comment

lguohan left a comment

Choose a reason for hiding this comment

pavel-shirshov left a comment •

edited

Loading