Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CERT-TEST-FAILURE] Test_TC_IDM_5_2.yaml test cases 3 and 4 should expect TIMEOUT? #33936

Closed
Apollon77 opened this issue Jun 14, 2024 · 4 comments
Closed

[CERT-TEST-FAILURE] Test_TC_IDM_5_2.yaml test cases 3 and 4 should expect TIMEOUT? #33936

Apollon77 opened this issue Jun 14, 2024 · 4 comments
Labels
bug Something isn't working cert blocker needs triage

Comments

@Apollon77
Copy link
Contributor

Feature Area

Other

Test Case

Test_TC_IDM_5_2

Reproduction steps

The test cases 3 and 4 are frommmy perspective testing a Invoke/Write interaction with an expired TimedInteraction before.

connectedhomeip/src/app/tests/suites/certification/Test_TC_IDM_5_2.yaml

Line 90 in a12a0c0

"Step 3: TH sends a Timed Request Message(Timed Invoke Transaction)

When I check specs (1.1+) it tells e.g.

Incoming Write Request Action
If this action is part of a Timed Write transaction, and the Timeout has expired from the preced­ ing Timed Request action, then a Status Response action with the TIMEOUT Status Code SHALL be submitted to the message layer and this interaction SHALL terminate.

and for read:

If this action is part of a Timed Invoke transaction, and the Timeout has expired from the pre­ ceding Timed Request action, then a Status Response action with the TIMEOUT Status Code SHALL be submitted to the message layer and this interaction SHALL terminate.

This test here expects UnsupportedAccess as error.

Do I miss anything or is the test defined wrong or the specticification?

Bug prevalence

came back as error from a current 1.1 certification

GitHub hash of the SDK that was being used

master

Platform

core

Anything else?

No response
@Apollon77 Apollon77 added bug Something isn't working cert blocker needs triage labels Jun 14, 2024
@github-project-automation github-project-automation bot moved this to Open Cert Blockers in [Certification] Blockers Jun 14, 2024
@cecille
Copy link
Contributor

cecille commented Jun 15, 2024

This appears to be quite deliberate in the code - see https://github.com/project-chip/connectedhomeip/blob/master/src/app/TimedHandler.cpp#L69.

@bzbarsky-apple - I know this was three years ago, but do you recall what part of the spec was being referenced here? Do we have a spec conflict?

@cecille
Copy link
Contributor

cecille commented Jun 15, 2024

Found it - please see https://github.com/CHIP-Specifications/connectedhomeip-spec/commit/0bd0e4f937ebaf84d6ebe7dc22e79c5af605bc2a. Error was changed, it would appear that the SDK and tests were not updated.

cecille added a commit to cecille/connectedhomeip that referenced this issue Jun 17, 2024
@cecille
TC-IDM-5.2: Fix status code in test and SDK
8482015 
Please see:
project-chip#33936
@cecille cecille mentioned this issue Jun 17, 2024
Merged
@bzbarsky-apple
Copy link
Contributor

Yeah, code was written 3 years ago, spec stealth-changed 2 years ago.... :(

mergify bot pushed a commit that referenced this issue Jul 4, 2024
@cecille
TC-IDM-5.2: Fix status code in test and SDK (#33952)
896d771 
* TC-IDM-5.2: Fix status code in test and SDK

Please see:
#33936

* Fix unit tests

We have unit tests! They didn't match the spec, but they exist!

One for two is still better than 0 for 2.

* Fix unit test

* Fix error on write request with incorrect timed request flag

Please see
#33952 (comment)

This was changed in the spec, but not in the SDK or unit tests.

The unit test here covers this, but we should consider adding it to
the cert testing as well. Please see
CHIP-Specifications/chip-test-plans#4256

* fix write in test as well
austina-csa pushed a commit to austina-csa/connectedhomeip that referenced this issue Jul 10, 2024
@cecille @austina-csa
TC-IDM-5.2: Fix status code in test and SDK (project-chip#33952)
4e230c9 
* TC-IDM-5.2: Fix status code in test and SDK

Please see:
project-chip#33936

* Fix unit tests

We have unit tests! They didn't match the spec, but they exist!

One for two is still better than 0 for 2.

* Fix unit test

* Fix error on write request with incorrect timed request flag

Please see
project-chip#33952 (comment)

This was changed in the spec, but not in the SDK or unit tests.

The unit test here covers this, but we should consider adding it to
the cert testing as well. Please see
CHIP-Specifications/chip-test-plans#4256

* fix write in test as well
@cjandhyala
Copy link
Contributor

@Apollon77 closing the issue given SDK and test plan issues are closed. Feel free to reopen if you still think this issue needs to be addressed.

@github-project-automation github-project-automation bot moved this from Open Cert Blockers to Complete in [Certification] Blockers Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working cert blocker needs triage
Projects
[Certification] Blockers
Archived in project
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Apollon77 @cecille @bzbarsky-apple @cjandhyala