Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve failure detection and reporting in the Tests job. #14729

Merged
merged 1 commit into from
Feb 3, 2022
Merged

Improve failure detection and reporting in the Tests job. #14729

merged 1 commit into from
Feb 3, 2022

Conversation

bzbarsky-apple
Copy link
Contributor

  • Fix exception formatting when a dependency of our chip-tool process
    has died.

  • Detect startup crashes in the server app without having to wait for
    the entire 10-second timeout.

  • Call apps_register.uninit() before sys.exit(), because otherwise on
    failure we end up sitting there waiting for an unjoined thread until
    the test job times out.

Problem

See above.

Change overview

See above.

Testing

Put exit(2) calls in main() and in a cluster command in chip-all-clusters-app, and observed the harness behavior.

@boring-cyborg boring-cyborg bot added the scripts label Feb 2, 2022
@restyled-io restyled-io bot mentioned this pull request Feb 2, 2022
Closed
@bzbarsky-apple
Improve failure detection and reporting in the Tests job.
81d27cc 
* Fix exception formatting when a dependency of our chip-tool process
  has died.

* Detect startup crashes in the server app without having to wait for
  the entire 10-second timeout.

* Call apps_register.uninit() before sys.exit(), because otherwise on
  failure we end up sitting there waiting for an unjoined thread until
  the test job times out.
@bzbarsky-apple bzbarsky-apple force-pushed the faster-failure-detection branch from b1784ca to 81d27cc Compare February 2, 2022 23:40
@github-actions
Copy link

github-actions bot commented Feb 3, 2022 •
edited
Loading

PR #14729: Size comparison from ae05dfc to 81d27cc

Full report (43 builds for cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, p6, qpg, telink)
platform target config section ae05dfc 81d27cc change % change
cyw30739 light cyw930739m2evb_01 (read/write) 576974 576974 0 0.0
.app_xip_area 484468 484468 0 0.0
.bss 75252 75252 0 0.0
.data 596 596 0 0.0
.rodata 0 0 0 0.0
.text 0 0 0 0.0
lock cyw930739m2evb_01 (read/write) 534734 534734 0 0.0
.app_xip_area 443796 443796 0 0.0
.bss 73724 73724 0 0.0
.data 560 560 0 0.0
.rodata 0 0 0 0.0
.text 0 0 0 0.0
ota-requestor cyw930739m2evb_01 (read/write) 559218 559218 0 0.0
.app_xip_area 458960 458960 0 0.0
.bss 82716 82716 0 0.0
.data 504 504 0 0.0
.rodata 0 0 0 0.0
.text 112 112 0 0.0
efr32 lighting-app BRD4161A (read only) 843736 843736 0 0.0
(read/write) 124792 124792 0 0.0
.bss 122896 122896 0 0.0
.data 1892 1892 0 0.0
.text 843728 843728 0 0.0
BRD4161A+rpc (read only) 831112 831112 0 0.0
(read/write) 141448 141448 0 0.0
.bss 139456 139456 0 0.0
.data 1992 1992 0 0.0
.text 831104 831104 0 0.0
window-app BRD4161A (read only) 815988 815988 0 0.0
(read/write) 123412 123412 0 0.0
.bss 121564 121564 0 0.0
.data 1848 1848 0 0.0
.text 815980 815980 0 0.0
esp32 all-clusters-app c3devkit (read only) 934076 934076 0 0.0
(read/write) 1401002 1401002 0 0.0
.dram0.bss 70048 70048 0 0.0
.dram0.data 14268 14268 0 0.0
.flash.rodata 197792 197792 0 0.0
.flash.text 934076 934076 0 0.0
.iram0.text 62056 62056 0 0.0
m5stack (read only) 982143 982143 0 0.0
(read/write) 465360 465360 0 0.0
.dram0.bss 74808 74808 0 0.0
.dram0.data 34024 34024 0 0.0
.flash.rodata 224400 224400 0 0.0
.flash.text 976759 976759 0 0.0
.iram0.text 123399 123399 0 0.0
k32w light k32w061+release (read/write) 661220 661220 0 0.0
.bss 74948 74948 0 0.0
.data 1852 1852 0 0.0
.text 578620 578620 0 0.0
lock k32w061+release (read/write) 662480 662480 0 0.0
.bss 75268 75268 0 0.0
.data 1892 1892 0 0.0
.text 579520 579520 0 0.0
linux all-clusters-app debug (read only) 2306865 2306865 0 0.0
(read/write) 139744 139744 0 0.0
.bss 54464 54464 0 0.0
.data 1200 1200 0 0.0
.data.rel.ro 78392 78392 0 0.0
.dynamic 592 592 0 0.0
.got 4128 4128 0 0.0
.init 27 27 0 0.0
.init_array 920 920 0 0.0
.rodata 204037 204037 0 0.0
.text 1940850 1940850 0 0.0
bridge-app debug+rpc (read only) 1634421 1634421 0 0.0
(read/write) 80424 80424 0 0.0
.bss 35904 35904 0 0.0
.data 1922 1922 0 0.0
.data.rel.ro 37440 37440 0 0.0
.dynamic 592 592 0 0.0
.got 3968 3968 0 0.0
.init 27 27 0 0.0
.init_array 544 544 0 0.0
.rodata 137620 137620 0 0.0
.text 1381733 1381733 0 0.0
chip-tool debug (read only) 7141165 7141165 0 0.0
(read/write) 180872 180872 0 0.0
.bss 28776 28776 0 0.0
.data 1088 1088 0 0.0
.data.rel.ro 145008 145008 0 0.0
.dynamic 608 608 0 0.0
.got 4760 4760 0 0.0
.init 27 27 0 0.0
.init_array 600 600 0 0.0
.rodata 407573 407573 0 0.0
.text 6366101 6366101 0 0.0
chip-tool-ipv6only arm64 (read only) 7092548 7092548 0 0.0
(read/write) 283521 283521 0 0.0
.bss 47089 47089 0 0.0
.data 1128 1128 0 0.0
.data.rel.ro 186304 186304 0 0.0
.dynamic 560 560 0 0.0
.got 45216 45216 0 0.0
.init 24 24 0 0.0
.init_array 184 184 0 0.0
.rodata 390604 390604 0 0.0
.text 6101476 6101476 0 0.0
door-lock-app debug (read only) 1855657 1855657 0 0.0
(read/write) 110304 110304 0 0.0
.bss 42080 42080 0 0.0
.data 962 962 0 0.0
.data.rel.ro 61856 61856 0 0.0
.dynamic 592 592 0 0.0
.got 4104 4104 0 0.0
.init 27 27 0 0.0
.init_array 664 664 0 0.0
.rodata 170940 170940 0 0.0
.text 1536674 1536674 0 0.0
lighting-app debug+rpc (read only) 1957009 1957009 0 0.0
(read/write) 115424 115424 0 0.0
.bss 43032 43032 0 0.0
.data 1392 1392 0 0.0
.data.rel.ro 65504 65504 0 0.0
.dynamic 608 608 0 0.0
.got 4136 4136 0 0.0
.init 27 27 0 0.0
.init_array 704 704 0 0.0
.rodata 160945 160945 0 0.0
.text 1641666 1641666 0 0.0
ota-provider-app debug (read only) 1638113 1638113 0 0.0
(read/write) 79912 79912 0 0.0
.bss 40960 40960 0 0.0
.data 1064 1064 0 0.0
.data.rel.ro 32376 32376 0 0.0
.dynamic 608 608 0 0.0
.got 4240 4240 0 0.0
.init 27 27 0 0.0
.init_array 608 608 0 0.0
.rodata 148486 148486 0 0.0
.text 1372210 1372210 0 0.0
ota-requestor-app debug (read only) 1643705 1643705 0 0.0
(read/write) 82896 82896 0 0.0
.bss 42272 42272 0 0.0
.data 1096 1096 0 0.0
.data.rel.ro 34248 34248 0 0.0
.dynamic 592 592 0 0.0
.got 4032 4032 0 0.0
.init 27 27 0 0.0
.init_array 616 616 0 0.0
.rodata 143831 143831 0 0.0
.text 1383634 1383634 0 0.0
shell debug (read only) 2281137 2281137 0 0.0
(read/write) 141104 141104 0 0.0
.bss 61760 61760 0 0.0
.data 768 768 0 0.0
.data.rel.ro 72896 72896 0 0.0
.dynamic 592 592 0 0.0
.got 4160 4160 0 0.0
.init 27 27 0 0.0
.init_array 904 904 0 0.0
.rodata 204786 204786 0 0.0
.text 1920306 1920306 0 0.0
thermostat-no-ble arm64 (read only) 2090204 2090204 0 0.0
(read/write) 140273 140273 0 0.0
.bss 57585 57585 0 0.0
.data 952 952 0 0.0
.data.rel.ro 74624 74624 0 0.0
.dynamic 560 560 0 0.0
.got 4136 4136 0 0.0
.init 24 24 0 0.0
.init_array 336 336 0 0.0
.rodata 130764 130764 0 0.0
.text 1741728 1741728 0 0.0
tv-app debug (read only) 2410857 2410857 0 0.0
(read/write) 138640 138640 0 0.0
.bss 57024 57024 0 0.0
.data 3104 3104 0 0.0
.data.rel.ro 72504 72504 0 0.0
.dynamic 592 592 0 0.0
.got 4520 4520 0 0.0
.init 27 27 0 0.0
.init_array 872 872 0 0.0
.rodata 191852 191852 0 0.0
.text 2047522 2047522 0 0.0
mbed all-clusters-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2385672 2385672 0 0.0
.bss 188964 188964 0 0.0
.data 5288 5288 0 0.0
.text 1348272 1348272 0 0.0
lighting-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2338360 2338360 0 0.0
.bss 177888 177888 0 0.0
.data 5568 5568 0 0.0
.text 1300960 1300960 0 0.0
lock-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2302152 2302152 0 0.0
.bss 177776 177776 0 0.0
.data 5544 5544 0 0.0
.text 1264752 1264752 0 0.0
pigweed-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 1139712 1139712 0 0.0
.bss 11756 11756 0 0.0
.data 4368 4368 0 0.0
.text 103096 103096 0 0.0
shell CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2292596 2292596 0 0.0
.bss 175052 175052 0 0.0
.data 5368 5368 0 0.0
.text 1255168 1255168 0 0.0
nrfconnect lighting-app nrf52840dk_nrf52840 (read/write) 980399 980399 0 0.0
bss 118260 118260 0 0.0
rodata 116324 116324 0 0.0
text 668024 668024 0 0.0
nrf52840dk_nrf52840+rpc (read/write) 965959 965959 0 0.0
bss 115304 115304 0 0.0
rodata 107796 107796 0 0.0
text 664464 664464 0 0.0
nrf52840dongle_nrf52840 (read/write) 996463 996463 0 0.0
bss 119432 119432 0 0.0
rodata 115168 115168 0 0.0
text 673400 673400 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 889614 889614 0 0.0
bss 115044 115044 0 0.0
rodata 109616 109616 0 0.0
text 584212 584212 0 0.0
lock-app nrf52840dk_nrf52840 (read/write) 912759 912759 0 0.0
bss 116620 116620 0 0.0
rodata 104596 104596 0 0.0
text 614136 614136 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 822774 822774 0 0.0
bss 113440 113440 0 0.0
rodata 97800 97800 0 0.0
text 531104 531104 0 0.0
pigweed-app nrf52840dk_nrf52840 (read/write) 541779 541779 0 0.0
bss 52588 52588 0 0.0
rodata 50048 50048 0 0.0
text 376940 376940 0 0.0
pump-app nrf52840dk_nrf52840 (read/write) 915411 915411 0 0.0
bss 116380 116380 0 0.0
rodata 105112 105112 0 0.0
text 616476 616476 0 0.0
pump-controller-app nrf52840dk_nrf52840 (read/write) 910635 910635 0 0.0
bss 116388 116388 0 0.0
rodata 104216 104216 0 0.0
text 612536 612536 0 0.0
shell nrf52840dk_nrf52840 (read/write) 798363 798363 0 0.0
bss 109772 109772 0 0.0
rodata 78352 78352 0 0.0
text 533732 533732 0 0.0
p6 all-clusters-app default (read/write) 2446856 2446856 0 0.0
.bss 117116 117116 0 0.0
.data 2576 2576 0 0.0
.text 1405120 1405120 0 0.0
light-app default (read/write) 2340448 2340448 0 0.0
.bss 103084 103084 0 0.0
.data 2400 2400 0 0.0
.text 1298712 1298712 0 0.0
lock-app default (read/write) 2305480 2305480 0 0.0
.bss 102804 102804 0 0.0
.data 2360 2360 0 0.0
.text 1263744 1263744 0 0.0
qpg lighting-app qpg6105+debug (read only) 570844 570844 0 0.0
(read/write) 146940 146940 0 0.0
.bss 87232 87232 0 0.0
.data 1056 1056 0 0.0
.text 565524 565524 0 0.0
lock-app qpg6105+debug (read only) 516684 516684 0 0.0
(read/write) 146940 146940 0 0.0
.bss 86680 86680 0 0.0
.data 992 992 0 0.0
.text 511364 511364 0 0.0
persistent-storage-app qpg6105+debug (read only) 107140 107140 0 0.0
(read/write) 146940 146940 0 0.0
.bss 38504 38504 0 0.0
.data 288 288 0 0.0
.text 101820 101820 0 0.0
telink lighting-app tlsr9518adk80d (read/write) 844938 844938 0 0.0
bss 85288 85288 0 0.0
noinit 37160 37160 0 0.0
text 592178 592178 0 0.0

@andy31415 andy31415 merged commit be3d056 into project-chip:master Feb 3, 2022
@bzbarsky-apple bzbarsky-apple deleted the faster-failure-detection branch February 3, 2022 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andy31415 andy31415 andy31415 approved these changes

@vivien-apple vivien-apple Awaiting requested review from vivien-apple

@anush-apple anush-apple Awaiting requested review from anush-apple

@austinh0 austinh0 Awaiting requested review from austinh0

@balducci-apple balducci-apple Awaiting requested review from balducci-apple

@Byungjoo-Lee Byungjoo-Lee Awaiting requested review from Byungjoo-Lee

@carol-apple carol-apple Awaiting requested review from carol-apple

@cecille cecille Awaiting requested review from cecille

@chrisdecenzo chrisdecenzo Awaiting requested review from chrisdecenzo

@chshu chshu Awaiting requested review from chshu

@chulspro chulspro Awaiting requested review from chulspro

@Damian-Nordic Damian-Nordic Awaiting requested review from Damian-Nordic

@dhrishi dhrishi Awaiting requested review from dhrishi

@electrocucaracha electrocucaracha Awaiting requested review from electrocucaracha

@emargolis emargolis Awaiting requested review from emargolis

@erjiaqing erjiaqing Awaiting requested review from erjiaqing

@franck-apple franck-apple Awaiting requested review from franck-apple

@gjc13 gjc13 Awaiting requested review from gjc13

@harimau-qirex harimau-qirex Awaiting requested review from harimau-qirex

@hawk248 hawk248 Awaiting requested review from hawk248

@isiu-apple isiu-apple Awaiting requested review from isiu-apple

@jelderton jelderton Awaiting requested review from jelderton

@jepenven-silabs jepenven-silabs Awaiting requested review from jepenven-silabs

@jmartinez-silabs jmartinez-silabs Awaiting requested review from jmartinez-silabs

@kpschoedel kpschoedel Awaiting requested review from kpschoedel

@LuDuda LuDuda Awaiting requested review from LuDuda

@msandstedt msandstedt Awaiting requested review from msandstedt

@sagar-apple sagar-apple Awaiting requested review from sagar-apple

@saurabhst saurabhst Awaiting requested review from saurabhst

@selissia selissia Awaiting requested review from selissia

@tecimovic tecimovic Awaiting requested review from tecimovic

@turon turon Awaiting requested review from turon

@vijs vijs Awaiting requested review from vijs

@wbschiller wbschiller Awaiting requested review from wbschiller

@woody-apple woody-apple Awaiting requested review from woody-apple

@xylophone21 xylophone21 Awaiting requested review from xylophone21

@yufengwangca yufengwangca Awaiting requested review from yufengwangca

@yunhanw-google yunhanw-google Awaiting requested review from yunhanw-google

@harsha-rajendran harsha-rajendran Awaiting requested review from harsha-rajendran

Assignees
No one assigned
Labels
fast track review - pending scripts
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bzbarsky-apple @andy31415