-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Address various DEM testing issues #35067
Conversation
Review changes with SemanticDiff. Analyzed 10 of 16 files. Overall, the semantic diff is 37% smaller than the GitHub diff.
|
PR #35067: Size comparison from b823a29 to a71bb17 Full report (10 builds for nrfconnect, nxp, qpg, stm32, tizen)
|
… as CI tests can run slower
PR #35067: Size comparison from b823a29 to 0c66ca8 Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
…sary and renumber test cases
PR #35067: Size comparison from b823a29 to 5bdddd1 Full report (22 builds for bl602, bl702, bl702l, nrfconnect, nxp, qpg, stm32, tizen)
|
PR #35067: Size comparison from b823a29 to 1f89e7c Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
PR #35067: Size comparison from b823a29 to 8f63c49 Increases above 0.2%:
Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
…eature_map() call
PR #35067: Size comparison from 1d1319b to 65af1b3 Full report (41 builds for bl602, bl702, bl702l, cyw30739, linux, nrfconnect, nxp, psoc6, qpg, stm32, tizen)
|
PR #35067: Size comparison from 1d1319b to 17edc0c Increases above 0.2%:
Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
I've reviewed Test cases 2.2 - 2.9 and ensured that they align with the test plan PR (https://github.com/CHIP-Specifications/chip-test-plans/pull/4543) fixing up minor issues. I think they all look good now - Thanks P! I just need to look at 2.10 (Q Quality tests) to tweak this test case so it covers both Forecast and PowerAdjustment attributes. |
… tidying up and fails due to powerAdjustmentCapability not being updated as expected.
PR #35067: Size comparison from 18a1863 to 58e6bf3 Full report (7 builds for nxp, qpg, stm32, tizen)
|
…buteChangeCallback when reason was changing.
PR #35067: Size comparison from 18a1863 to 81175d3 Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
PR #35067: Size comparison from 18a1863 to 50085be Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
PR #35067: Size comparison from 18a1863 to ce885c7 Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me now
PR #35067: Size comparison from cfdbba9 to cd820e7 Full report (77 builds for bl602, bl702, bl702l, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
* Add checks for correct DEM feature map for test * Add checks for a nominalPower having a value and get slotIndex check correct * Allow a little tolerance checking the duration returned in the event as CI tests can run slower * Restyled by clang-format * Restyled by autopep8 * Add check DEM feature map step + add event registration step if necessary and renumber test cases * Fix issue project-chip#4415 Fails randomly on CI when checking pause duration * Test an illegal slot index == number of slots in a forecast * Add a test in PFR when ModifyForecastRequest does not specify NominalPower * Set ClusterRevision to 4 * First versionof DEM 1-1 * Regenerate files * Get DEM 1.1 test passing * Update test step definitions * Get featureSet option correct * Restyled by autopep8 * First cut of a Q quality test * Regenerate files following a re-bootstrap * Fix typo in comment * Cancel subscriptions at the end of the test * Add a cancel subscription method * Ensure MatterReportingAttributeChangeCallback is called after forecast is changed * Restyled by autopep8 * Restyled by isort * Fix ruff detected errors * Restyled by autopep8 * Fix the cancelling of subscriptions * Used .name instead of needing a feature string mapping. * Removed TC_DEM_1_1 now that the test plan has removed this test case. * Corrected Featuremap -> FeatureMap in test steps * Aligned verification step 2 on TC_DEM_2.3 to actual code and syncd test plan. * Aligned verification step 2 on TC_DEM_2.4 to actual code and syncd test plan. * Added in missing text in step 18 in TC_DEM_2.4 * TC_DEM_2.5 minor update to step 2 and aligning to updated test plan. * TC_DEM_2.5 - slight change to call to name args in validate_feature_map() call * TC_DEM_2.6,2.7,2.8 step 2 alignment with tweaked test plan * TC_DEM_2.6/2.7/2.8 - slight change to call to name args in validate_feature_map() call * Corrected log message in TC_DEM_2.9 step 4b * Updated TC_DEM_2.10 (Q Quality) Basic structure is there - steps need tidying up and fails due to powerAdjustmentCapability not being updated as expected. * Fixed bug where PowerAdjustReason wasn't calling MatterReportingAttributeChangeCallback when reason was changing. * Restyled by clang-format * Restyled by autopep8 * Restyled by isort * Autogen'd the TestSteps from latest test plan * Changed featureSet to 0x7b to enable PA, PFR, FA (plus other) features --------- Co-authored-by: Restyled.io <commits@restyled.io> Co-authored-by: James Harrow <james.harrow@gmail.com> Co-authored-by: Hasty Granbery <hasty@granbery.org>
Addresses various DEM testing issues:
Fixes [Test Failed] DEM_TC_2_6 Device crashes. #34954
Fixes [Test Failed] [TC_DEM_2_7] fails on ESP32 #34960
Fixes [Test Failed] [TC_DEM_2.8] Step 5 - Invalidcommand returned instead on ConstraintError #34962
Fixes [CERT-TEST-FAILURE] - TC-DEM-1.1 - ClusterRevision value has to be updated for Device Energy Management Cluster #34297
https://github.com/CHIP-Specifications/chip-test-plans/issues/4415
https://github.com/CHIP-Specifications/chip-test-plans/issues/4482
Adds a new TC_DEM_2.10.py script for checking Q Quality and that MatterReporting callback is called when expected (found and fixed some issues here!)
This should be cross-reviewed against the DEM Test Plan change PR:
https://github.com/CHIP-Specifications/chip-test-plans/pull/4543