-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Test] Creating TC_CADMIN_1_3 and TC_CADMIN_1_4 python test modules #35818
base: master
Are you sure you want to change the base?
Conversation
j-ororke
commented
Sep 27, 2024
•
edited
Loading
edited
- Current WIP python script creation for TC_CADMIN_1_3 following test steps in PR: 4689
- Added WIP python script for TC_CADMIN_1_4 following test steps in PR: 4730
- Current WIP script creation following test steps in PR: CHIP-Specifications/chip-test-plans#4689
Review changes with SemanticDiff. Analyzed 2 of 4 files.
|
PR #35818: Size comparison from 1b4fa25 to 5a48f12 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Updated TC_CADMIN_1_3 to latest CI format
PR #35818: Size comparison from 1b4fa25 to 28ba101 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
PR #35818: Size comparison from 941c1d2 to b2d9a67 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Resolving linting errors
PR #35818: Size comparison from 941c1d2 to 75e368c Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
PR #35818: Size comparison from 9ae53c5 to 45c711e Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Appended WIP CADMIN_1_4 test script to TC_CADMIN_1_3 test module - Renamed TC_CADMIN_1_3 to TC_CADMIN_1_3_4 - Added issue in a TODO comment since current workaround in place to make CADMIN_1_3 pass
- Updated test step 14 for TC_CADMIN_1_3 to contain correct information
- Minor change to attempt resolving restylizer issue
PR #35818: Size comparison from af3727b to 6d59815 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Updating to resolve some linting issues
PR #35818: Size comparison from af3727b to 7a74ef3 Full report (20 builds for cc13x4_26x4, cc32xx, nrfconnect, nxp, qpg, stm32, tizen)
|
Updating to reflect change in master for "factory-reset", thank you @arkq Co-authored-by: Arkadiusz Bokowy <arkadiusz.bokowy@gmail.com>
PR #35818: Size comparison from af3727b to dabe8b7 Full report (88 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Now using get_setup_payload_info() to gather discriminator and passcode for BCM commissioning in TC_CADMIN_1_4
- Reverting back to setting dependency in test function as not able to get isort to not sort it out of position
- Added TC_CADMIN_1_4_noreset test module to attempt validation of script with no factory reset between tests - Updated test_TC_CADMIN_1_4 unit-test module to exit if credentials not found.
- Resolving lint issues
PR #35818: Size comparison from 1ae2e6e to 2a6a8d1 Full report (79 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- reorganizing CI args to attempt testing with factory-reset disabled
PR #35818: Size comparison from 1ae2e6e to 655e699 Full report (79 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Updating to attempt to get TC_CADMIN_1_4_noreset to get it to run in CI
PR #35818: Size comparison from 1ae2e6e to e2dcbed Full report (79 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
- Removed commissioning-method CI argument to not run commissioning before start of this test
PR #35818: Size comparison from 1ae2e6e to 1f6cf68 Full report (79 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
Removing unneeded small unit-test due to not needed
Removed prior unit-test test_TC_CADMIN_1_4.py as it was deleted
- These 2 yaml scripts are no longer needed as they are being replaced by the python test modules.
PR #35818: Size comparison from 1ae2e6e to 854da17 Full report (14 builds for cc13x4_26x4, cc32xx, nrfconnect, qpg, stm32, tizen)
|
- Disabled verbose output in CI arguments (quiet mode)
PR #35818: Size comparison from 1ae2e6e to ab6f907 Full report (67 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
@@ -0,0 +1,198 @@ | |||
#!/usr/bin/env -S python3 -B |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this file here just so you can re-run the 1.4 test? If so, this can be deleted and you can just add a second run to the TC_CADMIN_1_3_4.py file - just use run2: etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you,
I was not aware that I could do that in this manner, so I have changed the CI arguments as you have mentioned and removed the prior TC_CADMIN_1_4_noreset test module as it will no longer be needed.
nonce = random.randbytes(32) | ||
|
||
|
||
class TC_CADMIN_1_3_4(MatterBaseTest): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you may end up wanting to pull more of these tests into here at some point - maybe just use TC_CADMIN as the class name? Ditto for the file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi Cecille,
That sounds great, I have changed the class name as you have mentoned above to being just TC_CADMIN instead of TC_CADMIN_1_3_4!
async def get_fabrics(self, th: ChipDeviceCtrl) -> int: | ||
OC_cluster = Clusters.OperationalCredentials | ||
if th == self.th2: | ||
th2_fabric_info = await th.ReadAttribute(nodeid=self.dut_node_id, fabricFiltered=True, attributes=[(0, OC_cluster.Attributes.Fabrics)]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is this different for th2? can you not just use read_single_attribute_check_success and pass in the th?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi Cecille,
Sorry, I believe there was a reason I did this previously.
However, it appears that it does now work the same as TH1's method for getting the fabric information.
So, I have changed TH2's method to now match with TH1's method for getting the fabric information.
current_fabric_index = await self.read_single_attribute_check_success(dev_ctrl=th, endpoint=0, cluster=cluster, attribute=attribute) | ||
return current_fabric_index | ||
|
||
def pics_TC_CADMIN_1_3(self) -> list[str]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can these two tests be combined into a single parameterized function?
fabric_info = await self.read_single_attribute_check_success(dev_ctrl=th, fabric_filtered=True, cluster=OC_cluster, attribute=OC_cluster.Attributes.Fabrics) | ||
return fabric_info | ||
|
||
async def get_rcac_decoded(self, th: str) -> int: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is sending the csr request here just so you can issue a noc chain so you can get the rcac? If so, you shouldn't need to jump through so many hoops - the CA manager is
self.certificate_authority_manager and it maintains a list of the active roots: activeCaList
- Removed calls in tests.yaml for TC_CADMIN_1_3_4 as not needed individual calls for tests now that we have changed the CI arguments - Removed TC_CADMIN_1_4_noreset standalone test module as we are now using the CI arguments - Removed the line from execute_python_tests test runner module to remove not call for TC_CADMIN_1_4_noreset test module, as no longer needed - Renamed class for tests to TC_CADMIN from TC_CADMIN_1_3_4 to possibly encompaculate all CADMIN tests eventually. - Changed method to get th2_fabric_info to match with the method used to get th1_fabric_info in TC_CADMIN_1_3_4 test module