[arm] Refactor installer and build to allow arm builds targeted at grub platforms #11341

alexrallen · 2022-07-05T16:27:48Z

Refactors the SONiC Installer to support greater flexibility in building for a given architecture and bootloader.

Why I did it

Currently the SONiC installer assumes that if a platform is ARM based that it uses the uboot bootloader and uses the grub bootloader otherwise. This is not a correct assumption to make as ARM is not strictly tied to uboot and x86 is not strictly tied to grub.

How I did it

To implement this I introduce the following changes:

Remove the different arch folders from the installer/ directory
Merge the generic components of the ARM and x86 installer into installer/installer.sh
Refactor x86 + grub specific functions into installer/default_platform.conf
Modify installer to call default_platform.conf file and also call platform/[platform]/patform.conf file as well to override as needed
Update references to the installer in the build_image.sh script
Add TARGET_BOOTLOADER variable that is by default uboot for ARM devices and grub for x86 unless overridden in platform/[platform]/rules.mk
Update bootloader logic in build_debian.sh to be based on TARGET_BOOTLOADER instead of TARGET_ARCH and to reference the grub package in a generic manner

How to verify it

This has been tested on a ARM test platform as well as on Mellanox amd64 switches as well to ensure there was no impact.

Which release branch to backport (provide reason below if selected)

Description for the changelog

[arm] Refactor installer and build to allow arm builds targeted at grub platforms

Link to config_db schema for YANG module changes

N/A

A picture of a cute animal (not mandatory but encouraged)

…tforms

liat-grozovik · 2022-07-06T09:42:17Z

@stepanblyschak could you please help to review sonic-installer part?
@qiluo-msft could you please help to review or suggest someone to review?

installer/install.sh

installer/default_platform.conf

build_image.sh

qiluo-msft · 2022-07-08T00:01:25Z

installer/install.sh

-if [ -r ./onie-image-arm64.conf ]; then
-    . ./onie-image-arm64.conf
+if [ -r ./onie-image*.conf ]; then
+. ./onie-image*.conf


If there are multiple files matching the wildcard, only one is executed.

$ . a.sh a $ . b.sh b $ . *.sh a ``` #Closed

Seems there is a case of onie-image.conf and onie-image-*.conf both exist for one platform.

Still applicable.

Still applicable.

Not sure the reasoning of this comment @qiluo-msft no changes went in between your two comments, maybe I am misunderstanding.

Anyway, I think this should be resolved now. I have made a couple changes elsewhere (that should have been there in the first place) that make it such that only the needed onie-image files are loaded into an image given the target architecture. Then I load in the generic and the arch-specific version in the installer. This should resolve any concerns.

LGTM

xumia · 2022-07-27T14:27:46Z

@alexrallen looks like one of the template variable (%%EXTRA_CMDLINE_LINUX%%) is not replaced. Is it relative to the change? The grub.cfg in the latest vs image may be not correct.

log:

EXTRA_CMDLINE_LINUX=
ONIE: Unable to find 'Part Number' TLV in EEPROM data.
Success: Support tarball created: /tmp/onie-support-kvm_x86_64.tar.bz2
Installing for i386-pc platform.
Installation finished. No error reported.
Switch CPU vendor is: GenuineIntel
Switch CPU cstates are: disabled
EXTRA_CMDLINE_LINUX=%%EXTRA_CMDLINE_LINUX%%
Installed SONiC base image SONiC-OS successfully
ONIE: NOS install successful: file://dev/vdb/onie-installer.bin

And Grub settings: see %%EXTRA_CMDLINE_LINUX%% in linux.

menuentry 'SONiC-OS-master-10946.127441-ab961b5e3' {
        search --no-floppy --label --set=root SONiC-OS
        echo    'Loading SONiC-OS OS kernel ...'
        insmod gzio
        if [ x = xxen ]; then insmod xzio; insmod lzopio; fi
        insmod part_msdos
        insmod ext2
        linux   /image-master-10946.127441-ab961b5e3/boot/vmlinuz-5.10.0-12-2-amd64 root=UUID=0f1b4b07-9742-426d-bd3a-a1e789401536 rw console=tty0 console=ttyS0,115200n8 quiet intel_idle.max_cstate=0 %%EXTRA_CMDLINE_LINUX%%                  net.ifnames=0 biosdevname=0                 loop=image-master-10946.127441-ab961b5e3/fs.squashfs loopfstype=squashfs                                       systemd.unified_cgroup_hierarchy=0                 apparmor=1 security=apparmor varlog_size=4096 usbcore.autosuspend=-1 
        echo    'Loading SONiC-OS OS initial ramdisk ...'
        initrd  /image-master-10946.127441-ab961b5e3/boot/initrd.img-5.10.0-12-2-amd64
}

…ed at grub platforms (#11341)" This reverts commit 429254c.

qiluo-msft · 2022-08-01T23:16:55Z

installer/default_platform.conf

@@ -1,119 +1,5 @@
 #!/bin/sh


It is not ideal that we have .conf file with so complex bash code. And there are many variables need replacement, like %%SONIC_ROOT%%.

Related work items: sonic-net#284, sonic-net#288, sonic-net#624, sonic-net#630, sonic-net#640, sonic-net#2164, sonic-net#2206, sonic-net#2207, sonic-net#2233, sonic-net#2234, sonic-net#2238, sonic-net#8857, sonic-net#10034, sonic-net#10519, sonic-net#10685, sonic-net#10711, sonic-net#10987, sonic-net#10990, sonic-net#11047, sonic-net#11070, sonic-net#11117, sonic-net#11186, sonic-net#11207, sonic-net#11213, sonic-net#11215, sonic-net#11220, sonic-net#11221, sonic-net#11257, sonic-net#11291, sonic-net#11298, sonic-net#11301, sonic-net#11326, sonic-net#11333, sonic-net#11335, sonic-net#11341, sonic-net#11344, sonic-net#11347, sonic-net#11359, sonic-net#11366, sonic-net#11368, sonic-net#11370, sonic-net#11372, sonic-net#11375, sonic-net#11385, sonic-net#11386, sonic-net#11394, sonic-net#11397, sonic-net#11401, sonic-net#11402, sonic-net#11403, sonic-net#11405, sonic-net#11414

Why I did it Issue was caused by this #11341 *.bin image structure in 202205: vkarri@19d5638dde2d:/sonic$ ls -l /tmp/tmp.9ibWSipeRw/installer/x86_64/ total 12 drwxr-xr-x 2 vkarri dip 12288 Oct 14 13:16 platforms However install.sh which runs on ONiE parition expects the platform specific kernel cmd line conf file under platform/$onie_platform_string file https://github.com/sonic-net/sonic-buildimage/blob/master/installer/install.sh#L102 Thus, any platform which defines and depends on these params might be broken on master label. How I did it Since we are already filtering the conf files based on TARGET_PLATFORM in build_image.sh, i've just updated the location to installer/platforms instead of installer/$arch/platforms Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

alexrallen added 2 commits June 14, 2022 00:55

Refactor installer and build to allow arm builds targeted at grub pla…

30f323f

…tforms

Remove override of serial device and speed

a175d1c

alexrallen requested review from qiluo-msft, xumia and lguohan as code owners July 5, 2022 16:27

alexrallen mentioned this pull request Jul 5, 2022

[Mellanox] Add arm64 architecture support to mellanox platform #11342

Merged

7 tasks

Merge branch 'master' into sonic_arm_grub

ebf6a71

dgsudharsan approved these changes Jul 6, 2022

View reviewed changes

qiluo-msft reviewed Jul 6, 2022

View reviewed changes

installer/install.sh Outdated Show resolved Hide resolved

qiluo-msft requested a review from wen587 July 6, 2022 18:53

wen587 previously requested changes Jul 7, 2022

View reviewed changes

installer/default_platform.conf Show resolved Hide resolved

build_image.sh Show resolved Hide resolved

Make sure all included onie-image.conf files are executed

55b6522

qiluo-msft reviewed Jul 8, 2022

View reviewed changes

Revise calling of onie-image.conf file from installer

590304e

qiluo-msft approved these changes Jul 12, 2022

View reviewed changes

qiluo-msft merged commit 429254c into sonic-net:master Jul 12, 2022

xumia mentioned this pull request Jul 28, 2022

[Bug][Build] Fix the variable patterns not replaced issue #11572

Merged

7 tasks

qiluo-msft added a commit that referenced this pull request Aug 1, 2022

Revert "[arm] Refactor installer and build to allow arm builds target…

596ea5a

…ed at grub platforms (#11341)" This reverts commit 429254c.

qiluo-msft mentioned this pull request Aug 1, 2022

Revert "[arm] Refactor installer and build to allow arm builds targeted at grub platforms" #11599

Closed

qiluo-msft reviewed Aug 1, 2022

View reviewed changes

vivekrnv mentioned this pull request Oct 25, 2022

[Build] Fix the installer conf file location according to the updated onie-mk-demo script #12427

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[arm] Refactor installer and build to allow arm builds targeted at grub platforms #11341

[arm] Refactor installer and build to allow arm builds targeted at grub platforms #11341

alexrallen commented Jul 5, 2022

liat-grozovik commented Jul 6, 2022

qiluo-msft Jul 8, 2022 •

edited

Loading

qiluo-msft Jul 8, 2022

qiluo-msft Jul 12, 2022

alexrallen Jul 12, 2022 •

edited

Loading

xumia commented Jul 27, 2022 •

edited

Loading

qiluo-msft Aug 1, 2022

[arm] Refactor installer and build to allow arm builds targeted at grub platforms #11341

[arm] Refactor installer and build to allow arm builds targeted at grub platforms #11341

Conversation

alexrallen commented Jul 5, 2022

Why I did it

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

liat-grozovik commented Jul 6, 2022

qiluo-msft Jul 8, 2022 • edited Loading

Choose a reason for hiding this comment

qiluo-msft Jul 8, 2022

Choose a reason for hiding this comment

qiluo-msft Jul 12, 2022

Choose a reason for hiding this comment

alexrallen Jul 12, 2022 • edited Loading

Choose a reason for hiding this comment

xumia commented Jul 27, 2022 • edited Loading

qiluo-msft Aug 1, 2022

Choose a reason for hiding this comment

qiluo-msft Jul 8, 2022 •

edited

Loading

alexrallen Jul 12, 2022 •

edited

Loading

xumia commented Jul 27, 2022 •

edited

Loading