[Test] Creating TC_CADMIN_1_3 and TC_CADMIN_1_4 python test modules #35818
+369
−920
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
j-ororke
commented
Sep 27, 2024
•
j-ororke
commented
- Current WIP python script creation for TC_CADMIN_1_3 following test steps in PR: 4689
- Added WIP python script for TC_CADMIN_1_4 following test steps in PR: 4730
- Current WIP script creation following test steps in PR: CHIP-Specifications/chip-test-plans#4689
|
Review changes with SemanticDiff. Analyzed 2 of 4 files.
|
github-actions
bot
added
tests
matter-1.4-te2-script-change
|
PR #35818: Size comparison from 1b4fa25 to 5a48f12 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
arkq
reviewed
Sep 27, 2024
- Updated TC_CADMIN_1_3 to latest CI format
|
PR #35818: Size comparison from 1b4fa25 to 28ba101 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
|
PR #35818: Size comparison from 941c1d2 to b2d9a67 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Resolving linting errors
|
PR #35818: Size comparison from 941c1d2 to 75e368c Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
|
PR #35818: Size comparison from 9ae53c5 to 45c711e Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Appended WIP CADMIN_1_4 test script to TC_CADMIN_1_3 test module - Renamed TC_CADMIN_1_3 to TC_CADMIN_1_3_4 - Added issue in a TODO comment since current workaround in place to make CADMIN_1_3 pass
- Updated test step 14 for TC_CADMIN_1_3 to contain correct information
- Minor change to attempt resolving restylizer issue
|
PR #35818: Size comparison from af3727b to 6d59815 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Updating to resolve some linting issues
|
PR #35818: Size comparison from af3727b to 7a74ef3 Full report (20 builds for cc13x4_26x4, cc32xx, nrfconnect, nxp, qpg, stm32, tizen)
|
arkq
reviewed
Oct 9, 2024
Updating to reflect change in master for "factory-reset", thank you @arkq Co-authored-by: Arkadiusz Bokowy <arkadiusz.bokowy@gmail.com>
|
PR #35818: Size comparison from af3727b to dabe8b7 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Now using get_setup_payload_info() to gather discriminator and passcode for BCM commissioning in TC_CADMIN_1_4
- Reverting back to setting dependency in test function as not able to get isort to not sort it out of position
- Added TC_CADMIN_1_4_noreset test module to attempt validation of script with no factory reset between tests - Updated test_TC_CADMIN_1_4 unit-test module to exit if credentials not found.
- Resolving lint issues
|
PR #35818: Size comparison from 1ae2e6e to 2a6a8d1 Full report (79 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- reorganizing CI args to attempt testing with factory-reset disabled
|
PR #35818: Size comparison from 1ae2e6e to 655e699 Full report (79 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Updating to attempt to get TC_CADMIN_1_4_noreset to get it to run in CI
|
PR #35818: Size comparison from 1ae2e6e to e2dcbed Full report (79 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Removed commissioning-method CI argument to not run commissioning before start of this test
|
PR #35818: Size comparison from 1ae2e6e to 1f6cf68 Full report (79 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
Removing unneeded small unit-test due to not needed
Removed prior unit-test test_TC_CADMIN_1_4.py as it was deleted
- These 2 yaml scripts are no longer needed as they are being replaced by the python test modules.
|
PR #35818: Size comparison from 1ae2e6e to 854da17 Full report (14 builds for cc13x4_26x4, cc32xx, nrfconnect, qpg, stm32, tizen)
|
- Disabled verbose output in CI arguments (quiet mode)
|
PR #35818: Size comparison from 1ae2e6e to ab6f907 Full report (67 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
cecille
reviewed
Oct 21, 2024
| @@ -0,0 +1,198 @@ | |||
| #!/usr/bin/env -S python3 -B | |||
There was a problem hiding this comment.
Is this file here just so you can re-run the 1.4 test? If so, this can be deleted and you can just add a second run to the TC_CADMIN_1_3_4.py file - just use run2: etc.
There was a problem hiding this comment.
Thank you,
I was not aware that I could do that in this manner, so I have changed the CI arguments as you have mentioned and removed the prior TC_CADMIN_1_4_noreset test module as it will no longer be needed.
| nonce = random.randbytes(32) | ||
|
|
||
|
|
||
| class TC_CADMIN_1_3_4(MatterBaseTest): |
There was a problem hiding this comment.
I think you may end up wanting to pull more of these tests into here at some point - maybe just use TC_CADMIN as the class name? Ditto for the file.
There was a problem hiding this comment.
Hi Cecille,
That sounds great, I have changed the class name as you have mentoned above to being just TC_CADMIN instead of TC_CADMIN_1_3_4!
| async def get_fabrics(self, th: ChipDeviceCtrl) -> int: | ||
| OC_cluster = Clusters.OperationalCredentials | ||
| if th == self.th2: | ||
| th2_fabric_info = await th.ReadAttribute(nodeid=self.dut_node_id, fabricFiltered=True, attributes=[(0, OC_cluster.Attributes.Fabrics)]) |
There was a problem hiding this comment.
why is this different for th2? can you not just use read_single_attribute_check_success and pass in the th?
There was a problem hiding this comment.
Hi Cecille,
Sorry, I believe there was a reason I did this previously.
However, it appears that it does now work the same as TH1's method for getting the fabric information.
So, I have changed TH2's method to now match with TH1's method for getting the fabric information.
| current_fabric_index = await self.read_single_attribute_check_success(dev_ctrl=th, endpoint=0, cluster=cluster, attribute=attribute) | ||
| return current_fabric_index | ||
|
|
||
| def pics_TC_CADMIN_1_3(self) -> list[str]: |
There was a problem hiding this comment.
can these two tests be combined into a single parameterized function?
| fabric_info = await self.read_single_attribute_check_success(dev_ctrl=th, fabric_filtered=True, cluster=OC_cluster, attribute=OC_cluster.Attributes.Fabrics) | ||
| return fabric_info | ||
|
|
||
| async def get_rcac_decoded(self, th: str) -> int: |
There was a problem hiding this comment.
is sending the csr request here just so you can issue a noc chain so you can get the rcac? If so, you shouldn't need to jump through so many hoops - the CA manager is
self.certificate_authority_manager and it maintains a list of the active roots: activeCaList
cecille
removed
the
matter-1.4-te2-script-change
- Removed calls in tests.yaml for TC_CADMIN_1_3_4 as not needed individual calls for tests now that we have changed the CI arguments - Removed TC_CADMIN_1_4_noreset standalone test module as we are now using the CI arguments - Removed the line from execute_python_tests test runner module to remove not call for TC_CADMIN_1_4_noreset test module, as no longer needed - Renamed class for tests to TC_CADMIN from TC_CADMIN_1_3_4 to possibly encompaculate all CADMIN tests eventually. - Changed method to get th2_fabric_info to match with the method used to get th1_fabric_info in TC_CADMIN_1_3_4 test module
github-actions
bot
added
the
matter-1.4-te2-script-change
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.