Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add no-notification F3548 automated test #711

Open
BenjaminPelletier opened this issue Jun 6, 2024 · 0 comments
Open

Add no-notification F3548 automated test #711

BenjaminPelletier opened this issue Jun 6, 2024 · 0 comments
Labels
automated-testing Related to automated testing tools enhancement New feature or request P2 Normal priority

Comments

@BenjaminPelletier
Copy link
Member

BenjaminPelletier commented Jun 6, 2024

Is your feature request related to a problem? Please describe.
Currently, the behavior of a USS is not verified in the case where an expected notification is not sent.

Describe the solution you'd like
Create a test scenario that leverages mock_uss to create/modify/delete an operational intent where notifications are intentionally not sent and/or substantially delayed. This will require the addition of a new mock_uss behavior that tells mock_uss not to send notifications, and then to plan on mock_uss with this behavior.

Describe alternatives you've considered
There are a number of possible variants (e.g., don't send create notification but do send others, don't send any notifications, send create notification but don't send modify notification, send notification but wait 10 seconds to do so, etc), but I think we will get a large portion of the value of this test with a single, simple version (e.g., don't send any notifications).

One interesting consideration will be what behavior the new test scenario will exhibit if the USS under test rejects a flight planning request (presumably because the USS is relying solely on notifications for their airspace picture). F3548 generally does not prohibit rejecting planning attempts for any reason, so I don't think this behavior would violate any F3548 requirement. The question is whether InterUSS should necessarily expect the USS to be able to plan successfully under ExpectedBehavior ("these actions are generally expected to succeed (allowing the user to fly) when a UTM rule does not prohibit them"), effectively requiring every USS wanting a successful InterUSS automated testing outcome to implement missed-notification mitigations. My opinion is that this is a fair expectation and InterUSS would be justified indicating failure for ExpectedBehavior if a USS failed to plan a flight merely because mock_uss did not send a timely notification (but the operational intent details were valid and retrievable via GET), but I acknowledge this may be a good topic for discussion in the future.

Additional context
This scenario attempts to force a USS to perform a GET call to retrieve operational intent details but actually fails to achieve that goal because the subscription strategy of the USS cannot be controlled. This test will actually achieve that goal if the USS under test is able to plan successfully since the only way to get the operational intent details from mock_uss will be via GET. After this scenario is implemented, we should adjust the existing scenario linked here to not attempt to differentiate how the operational intent details are obtained (instead focusing on the fact that the operational intent details are invalid regardless of acquisition method).

@BenjaminPelletier BenjaminPelletier added enhancement New feature or request automated-testing Related to automated testing tools P2 Normal priority labels Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automated-testing Related to automated testing tools enhancement New feature or request P2 Normal priority
Projects
None yet
Development

No branches or pull requests

1 participant