Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YAML] Add a mechanism to specify the test timeout for a given YAML file #11861

Merged
merged 1 commit into from
Nov 30, 2021

Conversation

vivien-apple
Copy link
Contributor

Problem

Some YAML tests may requires more than 30 seconds to finalise.

Change overview

  • Add a mechanism to specify the timeout for a given YAML file.

Testing

I locally checked that it works by getting Test_SubscribeOnOff to timeout

@github-actions
Copy link

PR #11861: Size comparison from 0c88b57 to d0ca314

Increases (1 build for linux)
platform target config section 0c88b57 d0ca314 change % change
linux chip-tool debug (read only) 4892293 4892405 112 0.0
.text 4317765 4317877 112 0.0
Full report (38 builds for efr32, esp32, k32w, linux, mbed, nrfconnect, p6, qpg, telink)
platform target config section 0c88b57 d0ca314 change % change
efr32 lighting-app BRD4161A (read only) 745208 745208 0 0.0
(read/write) 115364 115364 0 0.0
.bss 113564 113564 0 0.0
.data 1800 1800 0 0.0
.text 745200 745200 0 0.0
lock-app BRD4161A (read only) 722296 722296 0 0.0
(read/write) 113148 113148 0 0.0
.bss 111388 111388 0 0.0
.data 1756 1756 0 0.0
.text 722288 722288 0 0.0
window-app BRD4161A (read only) 723272 723272 0 0.0
(read/write) 113468 113468 0 0.0
.bss 111708 111708 0 0.0
.data 1760 1760 0 0.0
.text 723264 723264 0 0.0
lighting-app BRD4161A+rpc (read only) 732724 732724 0 0.0
(read/write) 131992 131992 0 0.0
.bss 130068 130068 0 0.0
.data 1924 1924 0 0.0
.text 732716 732716 0 0.0
esp32 all-clusters-app c3devkit (read only) 824912 824912 0 0.0
(read/write) 1219650 1219650 0 0.0
.dram0.bss 56512 56512 0 0.0
.dram0.data 14084 14084 0 0.0
.flash.rodata 163800 163800 0 0.0
.flash.text 824912 824912 0 0.0
.iram0.text 61394 61394 0 0.0
m5stack (read only) 895819 895819 0 0.0
(read/write) 418980 418980 0 0.0
.dram0.bss 61864 61864 0 0.0
.dram0.data 34056 34056 0 0.0
.flash.rodata 191780 191780 0 0.0
.flash.text 890435 890435 0 0.0
.iram0.text 122943 122943 0 0.0
k32w lock-app k32w061+debug (read/write) 595020 595020 0 0.0
.bss 67500 67500 0 0.0
.data 1880 1880 0 0.0
.text 519840 519840 0 0.0
shell k32w061+debug (read/write) 660252 660252 0 0.0
.bss 78564 78564 0 0.0
.data 1848 1848 0 0.0
.text 574040 574040 0 0.0
lighting-app k32w061+se05x+release (read/write) 703576 703576 0 0.0
.bss 76988 76988 0 0.0
.data 1912 1912 0 0.0
.text 618876 618876 0 0.0
linux all-clusters-app debug (read only) 1701841 1701841 0 0.0
(read/write) 126624 126624 0 0.0
.bss 57520 57520 0 0.0
.data 1042 1042 0 0.0
.data.rel.ro 62784 62784 0 0.0
.dynamic 592 592 0 0.0
.got 4112 4112 0 0.0
.init 27 27 0 0.0
.init_array 552 552 0 0.0
.rodata 135253 135253 0 0.0
.text 1433138 1433138 0 0.0
chip-tool debug (read only) 4892293 4892405 112 0.0
(read/write) 166280 166280 0 0.0
.bss 40584 40584 0 0.0
.data 2272 2272 0 0.0
.data.rel.ro 117936 117936 0 0.0
.dynamic 592 592 0 0.0
.got 4416 4416 0 0.0
.init 27 27 0 0.0
.init_array 472 472 0 0.0
.rodata 265554 265554 0 0.0
.text 4317765 4317877 112 0.0
ota-provider-app debug (read only) 1252473 1252473 0 0.0
(read/write) 75560 75560 0 0.0
.bss 44832 44832 0 0.0
.data 784 784 0 0.0
.data.rel.ro 24840 24840 0 0.0
.dynamic 592 592 0 0.0
.got 4016 4016 0 0.0
.init 27 27 0 0.0
.init_array 448 448 0 0.0
.rodata 112999 112999 0 0.0
.text 1042386 1042386 0 0.0
ota-requestor-app debug (read only) 1335393 1335393 0 0.0
(read/write) 79264 79264 0 0.0
.bss 47296 47296 0 0.0
.data 848 848 0 0.0
.data.rel.ro 26024 26024 0 0.0
.dynamic 592 592 0 0.0
.got 3992 3992 0 0.0
.init 27 27 0 0.0
.init_array 472 472 0 0.0
.rodata 123792 123792 0 0.0
.text 1112178 1112178 0 0.0
shell debug (read only) 794665 794665 0 0.0
(read/write) 66088 66088 0 0.0
.bss 23816 23816 0 0.0
.data 242 242 0 0.0
.data.rel.ro 37520 37520 0 0.0
.dynamic 592 592 0 0.0
.got 3528 3528 0 0.0
.init 27 27 0 0.0
.init_array 344 344 0 0.0
.rodata 78479 78479 0 0.0
.text 612674 612674 0 0.0
tv-app debug (read only) 1838569 1838569 0 0.0
(read/write) 405016 405016 0 0.0
.bss 336968 336968 0 0.0
.data 2768 2768 0 0.0
.data.rel.ro 59600 59600 0 0.0
.dynamic 592 592 0 0.0
.got 4432 4432 0 0.0
.init 27 27 0 0.0
.init_array 616 616 0 0.0
.rodata 155149 155149 0 0.0
.text 1539154 1539154 0 0.0
bridge-app debug+rpc (read only) 1297013 1297013 0 0.0
(read/write) 76816 76816 0 0.0
.bss 42224 42224 0 0.0
.data 1568 1568 0 0.0
.data.rel.ro 28024 28024 0 0.0
.dynamic 592 592 0 0.0
.got 3984 3984 0 0.0
.init 27 27 0 0.0
.init_array 408 408 0 0.0
.rodata 110668 110668 0 0.0
.text 1089445 1089445 0 0.0
lighting-app debug+rpc (read only) 1566073 1566073 0 0.0
(read/write) 110064 110064 0 0.0
.bss 47888 47888 0 0.0
.data 1234 1234 0 0.0
.data.rel.ro 55616 55616 0 0.0
.dynamic 608 608 0 0.0
.got 4136 4136 0 0.0
.init 27 27 0 0.0
.init_array 536 536 0 0.0
.rodata 128273 128273 0 0.0
.text 1303298 1303298 0 0.0
mbed all-clusters-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2283728 2283728 0 0.0
.bss 179108 179108 0 0.0
.data 5216 5216 0 0.0
.heap 852120 852120 0 0.0
.text 1246328 1246328 0 0.0
lighting-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2269720 2269720 0 0.0
.bss 171972 171972 0 0.0
.data 5576 5576 0 0.0
.heap 858896 858896 0 0.0
.text 1232320 1232320 0 0.0
lock-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2245560 2245560 0 0.0
.bss 170860 170860 0 0.0
.data 5568 5568 0 0.0
.heap 860016 860016 0 0.0
.text 1208160 1208160 0 0.0
pigweed-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 1139744 1139744 0 0.0
.bss 11752 11752 0 0.0
.data 4368 4368 0 0.0
.heap 1020328 1020328 0 0.0
.text 103128 103128 0 0.0
shell CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2049520 2049520 0 0.0
.bss 156160 156160 0 0.0
.data 4968 4968 0 0.0
.heap 875320 875320 0 0.0
.text 1012120 1012120 0 0.0
nrfconnect lighting-app nrf52840dk_nrf52840 (read/write) 860183 860183 0 0.0
bss 110384 110384 0 0.0
rodata 95032 95032 0 0.0
text 579140 579140 0 0.0
lock-app nrf52840dk_nrf52840 (read/write) 835579 835579 0 0.0
bss 109408 109408 0 0.0
rodata 91500 91500 0 0.0
text 559236 559236 0 0.0
pigweed-app nrf52840dk_nrf52840 (read/write) 497327 497327 0 0.0
bss 51824 51824 0 0.0
rodata 45780 45780 0 0.0
text 339436 339436 0 0.0
pump-app nrf52840dk_nrf52840 (read/write) 841723 841723 0 0.0
bss 109548 109548 0 0.0
rodata 93212 93212 0 0.0
text 563420 563420 0 0.0
pump-controller-app nrf52840dk_nrf52840 (read/write) 835315 835315 0 0.0
bss 109444 109444 0 0.0
rodata 91460 91460 0 0.0
text 558856 558856 0 0.0
shell nrf52840dk_nrf52840 (read/write) 776239 776239 0 0.0
bss 109004 109004 0 0.0
rodata 72516 72516 0 0.0
text 520124 520124 0 0.0
lighting-app nrf52840dk_nrf52840+rpc (read/write) 822551 822551 0 0.0
bss 106740 106740 0 0.0
rodata 86216 86216 0 0.0
text 553312 553312 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 785222 785222 0 0.0
bss 111760 111760 0 0.0
rodata 90288 90288 0 0.0
text 508608 508608 0 0.0
lock-app nrf5340dk_nrf5340_cpuapp (read/write) 760902 760902 0 0.0
bss 110820 110820 0 0.0
rodata 86808 86808 0 0.0
text 488796 488796 0 0.0
shell nrf5340dk_nrf5340_cpuapp (read/write) 691294 691294 0 0.0
bss 109988 109988 0 0.0
rodata 67160 67160 0 0.0
text 440732 440732 0 0.0
p6 all-clusters-app default (read/write) 2292608 2292608 0 0.0
.bss 112096 112096 0 0.0
.data 2520 2520 0 0.0
.heap 918728 918728 0 0.0
.text 1250872 1250872 0 0.0
lock-app default (read/write) 2210080 2210080 0 0.0
.bss 100720 100720 0 0.0
.data 2400 2400 0 0.0
.heap 930224 930224 0 0.0
.text 1168344 1168344 0 0.0
qpg lighting-app qpg6100+debug (read only) 489736 489736 0 0.0
(read/write) 114144 114144 0 0.0
.bss 50056 50056 0 0.0
.data 1008 1008 0 0.0
.text 484416 484416 0 0.0
lock-app qpg6100+debug (read only) 464692 464692 0 0.0
(read/write) 114140 114140 0 0.0
.bss 49000 49000 0 0.0
.data 964 964 0 0.0
.text 459372 459372 0 0.0
persistent-storage-app qpg6100+debug (read only) 105408 105408 0 0.0
(read/write) 114142 114142 0 0.0
.bss 8978 8978 0 0.0
.data 272 272 0 0.0
.text 100088 100088 0 0.0
telink lighting-app tlsr9518adk80d (read/write) 763014 763014 0 0.0
bss 78896 78896 0 0.0
noinit 37160 37160 0 0.0
text 529210 529210 0 0.0

@jmeg-sfy
Copy link
Contributor

jmeg-sfy commented Nov 22, 2021

can you give us a review here ? @bzbarsky-apple i cannot see your checks mark for some reason

@vivien-apple vivien-apple self-assigned this Nov 25, 2021
@github-actions
Copy link

github-actions bot commented Nov 25, 2021

PR #11861: Size comparison from c79b67b to f431b29

Increases (1 build for linux)
platform target config section c79b67b f431b29 change % change
linux chip-tool debug (read only) 6005893 6006005 112 0.0
.text 5333397 5333509 112 0.0
Full report (38 builds for efr32, esp32, k32w, linux, mbed, nrfconnect, p6, qpg, telink)
platform target config section c79b67b f431b29 change % change
efr32 lighting-app BRD4161A (read only) 759184 759184 0 0.0
(read/write) 119812 119812 0 0.0
.bss 117988 117988 0 0.0
.data 1820 1820 0 0.0
.text 759176 759176 0 0.0
BRD4161A+rpc (read only) 746964 746964 0 0.0
(read/write) 136436 136436 0 0.0
.bss 134492 134492 0 0.0
.data 1944 1944 0 0.0
.text 746956 746956 0 0.0
lock-app BRD4161A (read only) 735120 735120 0 0.0
(read/write) 117524 117524 0 0.0
.bss 115748 115748 0 0.0
.data 1776 1776 0 0.0
.text 735112 735112 0 0.0
window-app BRD4161A (read only) 738560 738560 0 0.0
(read/write) 117884 117884 0 0.0
.bss 116100 116100 0 0.0
.data 1784 1784 0 0.0
.text 738552 738552 0 0.0
esp32 all-clusters-app c3devkit (read only) 835642 835642 0 0.0
(read/write) 1224282 1224282 0 0.0
.dram0.bss 59240 59240 0 0.0
.dram0.data 14100 14100 0 0.0
.flash.rodata 165680 165680 0 0.0
.flash.text 835642 835642 0 0.0
.iram0.text 61394 61394 0 0.0
m5stack (read only) 906987 906987 0 0.0
(read/write) 423532 423532 0 0.0
.dram0.bss 64632 64632 0 0.0
.dram0.data 34072 34072 0 0.0
.flash.rodata 193548 193548 0 0.0
.flash.text 901603 901603 0 0.0
.iram0.text 122943 122943 0 0.0
k32w lighting-app k32w061+se05x+release (read/write) 720916 720916 0 0.0
.bss 78276 78276 0 0.0
.data 1956 1956 0 0.0
.text 634884 634884 0 0.0
lock-app k32w061+debug (read/write) 611236 611236 0 0.0
.bss 68716 68716 0 0.0
.data 1920 1920 0 0.0
.text 534800 534800 0 0.0
shell k32w061+debug (read/write) 676568 676568 0 0.0
.bss 79868 79868 0 0.0
.data 1892 1892 0 0.0
.text 589008 589008 0 0.0
linux all-clusters-app debug (read only) 1769433 1769433 0 0.0
(read/write) 131480 131480 0 0.0
.bss 60080 60080 0 0.0
.data 1170 1170 0 0.0
.data.rel.ro 64928 64928 0 0.0
.dynamic 592 592 0 0.0
.got 4112 4112 0 0.0
.init 27 27 0 0.0
.init_array 576 576 0 0.0
.rodata 138325 138325 0 0.0
.text 1492738 1492738 0 0.0
bridge-app debug+rpc (read only) 1345293 1345293 0 0.0
(read/write) 77824 77824 0 0.0
.bss 41712 41712 0 0.0
.data 1680 1680 0 0.0
.data.rel.ro 29384 29384 0 0.0
.dynamic 592 592 0 0.0
.got 3984 3984 0 0.0
.init 27 27 0 0.0
.init_array 424 424 0 0.0
.rodata 112988 112988 0 0.0
.text 1132213 1132213 0 0.0
chip-tool debug (read only) 6005893 6006005 112 0.0
(read/write) 199664 199664 0 0.0
.bss 40064 40064 0 0.0
.data 2384 2384 0 0.0
.data.rel.ro 151632 151632 0 0.0
.dynamic 592 592 0 0.0
.got 4456 4456 0 0.0
.init 27 27 0 0.0
.init_array 488 488 0 0.0
.rodata 283944 283944 0 0.0
.text 5333397 5333509 112 0.0
lighting-app debug+rpc (read only) 1618217 1618217 0 0.0
(read/write) 111072 111072 0 0.0
.bss 47408 47408 0 0.0
.data 1362 1362 0 0.0
.data.rel.ro 56976 56976 0 0.0
.dynamic 608 608 0 0.0
.got 4136 4136 0 0.0
.init 27 27 0 0.0
.init_array 552 552 0 0.0
.rodata 131249 131249 0 0.0
.text 1349250 1349250 0 0.0
ota-provider-app debug (read only) 1309361 1309361 0 0.0
(read/write) 76312 76312 0 0.0
.bss 44320 44320 0 0.0
.data 912 912 0 0.0
.data.rel.ro 25944 25944 0 0.0
.dynamic 592 592 0 0.0
.got 4048 4048 0 0.0
.init 27 27 0 0.0
.init_array 464 464 0 0.0
.rodata 114736 114736 0 0.0
.text 1094274 1094274 0 0.0
ota-requestor-app debug (read only) 1405849 1405849 0 0.0
(read/write) 80144 80144 0 0.0
.bss 46752 46752 0 0.0
.data 976 976 0 0.0
.data.rel.ro 27272 27272 0 0.0
.dynamic 592 592 0 0.0
.got 4032 4032 0 0.0
.init 27 27 0 0.0
.init_array 488 488 0 0.0
.rodata 126176 126176 0 0.0
.text 1176530 1176530 0 0.0
shell debug (read only) 820129 820129 0 0.0
(read/write) 66936 66936 0 0.0
.bss 23496 23496 0 0.0
.data 338 338 0 0.0
.data.rel.ro 38560 38560 0 0.0
.dynamic 592 592 0 0.0
.got 3560 3560 0 0.0
.init 27 27 0 0.0
.init_array 360 360 0 0.0
.rodata 78927 78927 0 0.0
.text 634514 634514 0 0.0
tv-app debug (read only) 1915025 1915025 0 0.0
(read/write) 320088 320088 0 0.0
.bss 250168 250168 0 0.0
.data 2880 2880 0 0.0
.data.rel.ro 61368 61368 0 0.0
.dynamic 592 592 0 0.0
.got 4432 4432 0 0.0
.init 27 27 0 0.0
.init_array 632 632 0 0.0
.rodata 159240 159240 0 0.0
.text 1607586 1607586 0 0.0
mbed all-clusters-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2293520 2293520 0 0.0
.bss 181788 181788 0 0.0
.data 5240 5240 0 0.0
.heap 849416 849416 0 0.0
.text 1256120 1256120 0 0.0
lighting-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2277552 2277552 0 0.0
.bss 172404 172404 0 0.0
.data 5600 5600 0 0.0
.heap 858440 858440 0 0.0
.text 1240152 1240152 0 0.0
lock-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2251936 2251936 0 0.0
.bss 171220 171220 0 0.0
.data 5584 5584 0 0.0
.heap 859640 859640 0 0.0
.text 1214536 1214536 0 0.0
pigweed-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 1139744 1139744 0 0.0
.bss 11752 11752 0 0.0
.data 4368 4368 0 0.0
.heap 1020328 1020328 0 0.0
.text 103128 103128 0 0.0
shell CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2050800 2050800 0 0.0
.bss 156424 156424 0 0.0
.data 4984 4984 0 0.0
.heap 875040 875040 0 0.0
.text 1013400 1013400 0 0.0
nrfconnect lighting-app nrf52840dk_nrf52840 (read/write) 873291 873291 0 0.0
bss 112644 112644 0 0.0
rodata 96676 96676 0 0.0
text 588304 588304 0 0.0
nrf52840dk_nrf52840+rpc (read/write) 835819 835819 0 0.0
bss 108996 108996 0 0.0
rodata 87924 87924 0 0.0
text 562576 562576 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 798318 798318 0 0.0
bss 114020 114020 0 0.0
rodata 91936 91936 0 0.0
text 517764 517764 0 0.0
lock-app nrf52840dk_nrf52840 (read/write) 845447 845447 0 0.0
bss 109680 109680 0 0.0
rodata 92680 92680 0 0.0
text 567616 567616 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 770718 770718 0 0.0
bss 111088 111088 0 0.0
rodata 87968 87968 0 0.0
text 497172 497172 0 0.0
pigweed-app nrf52840dk_nrf52840 (read/write) 497327 497327 0 0.0
bss 51824 51824 0 0.0
rodata 45780 45780 0 0.0
text 339436 339436 0 0.0
pump-app nrf52840dk_nrf52840 (read/write) 851547 851547 0 0.0
bss 109816 109816 0 0.0
rodata 94388 94388 0 0.0
text 571800 571800 0 0.0
pump-controller-app nrf52840dk_nrf52840 (read/write) 845139 845139 0 0.0
bss 109712 109712 0 0.0
rodata 92636 92636 0 0.0
text 567236 567236 0 0.0
shell nrf52840dk_nrf52840 (read/write) 778327 778327 0 0.0
bss 109180 109180 0 0.0
rodata 72996 72996 0 0.0
text 521552 521552 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 693382 693382 0 0.0
bss 110164 110164 0 0.0
rodata 67640 67640 0 0.0
text 442160 442160 0 0.0
p6 all-clusters-app default (read/write) 2310328 2310328 0 0.0
.bss 114784 114784 0 0.0
.data 2544 2544 0 0.0
.heap 916016 916016 0 0.0
.text 1268592 1268592 0 0.0
lock-app default (read/write) 2222144 2222144 0 0.0
.bss 101080 101080 0 0.0
.data 2416 2416 0 0.0
.heap 929848 929848 0 0.0
.text 1180408 1180408 0 0.0
qpg lighting-app qpg6100+debug (read only) 500992 500992 0 0.0
(read/write) 114144 114144 0 0.0
.bss 50384 50384 0 0.0
.data 1024 1024 0 0.0
.text 495672 495672 0 0.0
lock-app qpg6100+debug (read only) 474960 474960 0 0.0
(read/write) 114140 114140 0 0.0
.bss 49256 49256 0 0.0
.data 980 980 0 0.0
.text 469640 469640 0 0.0
persistent-storage-app qpg6100+debug (read only) 105424 105424 0 0.0
(read/write) 114138 114138 0 0.0
.bss 12002 12002 0 0.0
.data 276 276 0 0.0
.text 100104 100104 0 0.0
telink lighting-app tlsr9518adk80d (read/write) 774870 774870 0 0.0
bss 79216 79216 0 0.0
noinit 37160 37160 0 0.0
text 538758 538758 0 0.0

@woody-apple
Copy link
Contributor

Fast tracking given this is adding tests, and has been in review for a while.

@woody-apple woody-apple merged commit 91cc989 into project-chip:master Nov 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants