Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pydantic for Test Scenario #205

Merged
merged 28 commits into from
Oct 7, 2024
Merged

Conversation

amaslenn
Copy link
Contributor

@amaslenn amaslenn commented Sep 26, 2024

Summary

Introduce Pydantic schema validation for Test Scenarios.

Test Plan

  1. CI

Additional Notes

Highlights for Release notes

We are working on schema improvements to simplify configs management and make them verifiable. This will help ensure that configs are correct before expensive runs on real hardware. Today we are enabling it for Test Scenario configs. This is a continuation of #145.

  1. Tests becomes and array. This helps making case names more expressive:
    before:
    [Tests.1]
    # ...
    now:
    [[Tests]]
    id = "any-name.you_want" # before it was just "1"
  2. id field is mandatory and must be unique and is used to specify dependencies:
    [[Tests]]
    id = "Tests.1"
    # ...
    
    [[Tests]]
    id = "Tests.2"
    # ...
      [[Tests.dependencies]]
      id = "Tests.1"
      # ...
  3. name (under the list of tests) renamed to test_name to better reflect its meaning. It still references a test defined in a separate TOML file.
  4. Dependencies converted to a list to support multiple dependencies of the same type.
    before
    # ...
    
    [Tests.2]
    name = "ucc_test_alltoall"
      [Tests.2.dependencies]
      start_post_comp = { name = "Tests.1", time = 0 }  # only one dependency of this type is allowed
    now
    # ...
    
    [[Tests]]
    id = "Tests.3"
    test_name = "ucc_test_alltoall"
    # ...
      [[Tests.dependencies]]
      type = "start_post_comp"
      id = "Tests.1"
    
      [[Tests.dependencies]]
      type = "start_post_comp"
      id = "Tests.2"

@TaekyungHeo
Copy link
Member

We should also have a corresponding PR in CloudAIX. The CloudAIX TOML files and code should be updated accordingly.

@TaekyungHeo TaekyungHeo added Oct24 Oct'24 release feature feature labels Oct 4, 2024
Copy link
Contributor

@srivatsankrishnan srivatsankrishnan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

TaekyungHeo
TaekyungHeo previously approved these changes Oct 7, 2024
@srivatsankrishnan srivatsankrishnan merged commit c40e92c into main Oct 7, 2024
2 checks passed
@srivatsankrishnan srivatsankrishnan deleted the am/pydantic-test-scenario branch October 7, 2024 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Oct24 Oct'24 release feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants