Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[202012] CPU Usage Spike Issue Handled #1

Closed
wants to merge 1 commit into from
Closed

Conversation

vivekrnv
Copy link
Owner

@vivekrnv vivekrnv commented Jul 17, 2021

Signed-off-by: Vivek Reddy Karri vkarri@nvidia.com

- What I did

- How I did it

- How to verify it

Verified on the Version which has this commit: sonic-swss/pull/1804

admin@sonic:~$ show ver

SONiC Software Version: SONiC.202012.130-8acb20677_Internal
Distribution: Debian 10.10
Kernel: 4.19.0-12-2-amd64
Build commit: 8acb20677
Build date: Fri Jul 16 12:54:07 UTC 2021
Built by: sw-r2d2-bot@r-build-sonic-ci03

Platform: x86_64-mlnx_msn4700-r0
HwSKU: ACS-MSN4700
ASIC: mellanox
ASIC Count: 1
Serial Number: MT2047X05586
Uptime: 07:27:34 up  7:01,  1 user,  load average: 0.42, 0.25, 0.20

admin@sonic:~$ counterpoll show
Type                        Interval (in ms)    Status
--------------------------  ------------------  --------
QUEUE_STAT                  default (10000)     disable
PORT_STAT                   default (1000)      disable
PORT_BUFFER_DROP            default (60000)     disable
RIF_STAT                    default (1000)      disable
QUEUE_WATERMARK_STAT        default (10000)     disable
PG_WATERMARK_STAT           default (10000)     disable
PG_DROP_STAT                default (10000)     disable
BUFFER_POOL_WATERMARK_STAT  default (10000)     disable

CPU Usage is normal:

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
12db43f74d75        snmp                2.28%               61.24MiB / 15.56GiB   0.38%               0B / 0B             61.4kB / 81.9kB     9
b58649fdaf02        telemetry           1.17%               144.4MiB / 15.56GiB   0.91%               0B / 0B             36.6MB / 86kB       35
abecc451239e        mgmt-framework      0.03%               112.2MiB / 15.56GiB   0.70%               0B / 0B             26.3MB / 69.6kB     19
f39a4a650d43        radv                0.03%               30.09MiB / 15.56GiB   0.19%               0B / 0B             225kB / 77.8kB      6
c58d55ed4e56        lldp                0.04%               54.94MiB / 15.56GiB   0.34%               0B / 0B             6MB / 115kB         11
a7ff6a4e6a51        pmon                37.54%              240.9MiB / 15.56GiB   1.51%               0B / 0B             58.3MB / 6.04MB     17
dc4269cbd26f        dhcp_relay          0.03%               30.65MiB / 15.56GiB   0.19%               0B / 0B             459kB / 73.7kB      6
7d277afdcea1        syncd               0.69%               845MiB / 15.56GiB     5.30%               0B / 0B             39.5MB / 69.6kB     39
b99396746503        teamd               0.06%               34.18MiB / 15.56GiB   0.21%               0B / 0B             3.94MB / 90.1kB     12
632b5e536707        swss                0.15%               47.45MiB / 15.56GiB   0.30%               0B / 0B             15.6MB / 229kB      33
8aa0943e0e64        bgp                 0.05%               71.56MiB / 15.56GiB   0.45%               0B / 0B             16.6MB / 19.4MB     22
12f178550f94        restapi             0.04%               40.02MiB / 15.56GiB   0.25%               0B / 0B             34.8MB / 102kB      8
261eca81d21d        database            1.30%               59.57MiB / 15.56GiB   0.37%               0B / 0B             38.7MB / 65.5kB     11

- Description for the changelog

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>
@liat-grozovik
Copy link

the PR is against 202012 for testing purposes only? we should have the fix is master and then ask to cherry pick right?
BTW in master you probably have additional RFC recently contributed by Raphael so more fixes will be required there as well.

@vivekrnv
Copy link
Owner Author

vivekrnv commented Jul 20, 2021

the PR is against 202012 for testing purposes only? we should have the fix is master and then ask to cherry pick right?
BTW in master you probably have additional RFC recently contributed by Raphael so more fixes will be required there as well.

No, the PR by shlomi which delayed the counters initialization was introduced only in 202012 and thus raised against 202012. This is a fairly simple change and can be easily ported to master, even if cherry picking isn't possible

@liat-grozovik
Copy link

liat-grozovik commented Jul 20, 2021 via email

@vivekrnv
Copy link
Owner Author

The pr by Shlomi is also in master Please double check it with him as well Get Outlook for iOShttps://aka.ms/o0ukef

________________________________ From: Vivek Reddy @.> Sent: Tuesday, July 20, 2021 11:48:52 AM To: vivekreddynv/sonic-snmpagent @.> Cc: Liat Grozovik @.>; Comment @.> Subject: Re: [vivekreddynv/sonic-snmpagent] [202012] CPU Usage Spike Issue Handled (#1) the PR is against 202012 for testing purposes only? we should have the fix is master and then ask to cherry pick right? BTW in master you probably have additional RFC recently contributed by Raphael so more fixes will be required there as well. No, the PR by shlomi which delayed the counters initialization was introduced only in 202012 and thus raised against 202012. This a fairly simple change and can be easily ported to master, even if cheery picking isn't possible — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKTABA6ISKBLNRRSOE4DA5DTYUZ7JANCNFSM5AQZX7RA.

Sorry about that, the PR was also raised to master. I've tried cherry-picking this change to master but there are some conflicts, if this change is good to go, i'll manually port it to master.

@vivekrnv vivekrnv closed this Jul 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants