Skip to content

Commit

Permalink
Add hazard info to journal (#824)
Browse files Browse the repository at this point in the history
* Add 'hazard info' entries to journal

* Update doc
  • Loading branch information
bkorycki authored Jan 30, 2025
1 parent 8fa2f93 commit 9a22a3d
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 0 deletions.
5 changes: 5 additions & 0 deletions docs/run-journal.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ In order of occurrence:

- starting journal - marks the beginning of the journal
- starting run - gives basic information on the run
- hazard info - gives information on each Hazard in the run
- test info - gives information on each Test in the run
- running pipeline - marks the start of the item pipeline
- using test items - for each Test, the counts of Items available and actually used
Expand Down Expand Up @@ -73,6 +74,10 @@ Listed in run order. If a message isn't listed here, its fields are covered abov
- suts - list of uids for the SUTs in the run
- max_items - maximum number of Items per test to be run
- thread_count - maximum number of threads per stage of pipeline
- hazard info
- hazard - uid of the Hazard
- benchmark - uid of the Benchmark the Hazard is part of
- tests - list of uids of the Tests in the Hazard
- test info
- test - uid of Test
- initialization - initialization record for Test
Expand Down
9 changes: 9 additions & 0 deletions src/modelbench/benchmark_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -589,6 +589,15 @@ def run(self) -> BenchmarkRun:
max_items=benchmark_run.max_items,
thread_count=self.thread_count,
)
for benchmark in benchmark_run.benchmarks:
for hazard in benchmark.hazards():
benchmark_run.journal.raw_entry(
"hazard info",
hazard=hazard.uid,
benchmark=benchmark.uid,
tests=hazard.test_uids(),
)

for test in benchmark_run.tests:
benchmark_run.journal.raw_entry(
"test info",
Expand Down
1 change: 1 addition & 0 deletions tests/modelbench_tests/test_benchmark_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -623,6 +623,7 @@ def test_basic_benchmark_run(self, tmp_path, a_sut, fake_secrets, benchmark):
assert messages == [
"starting journal",
"starting run",
"hazard info",
"test info",
"running pipeline",
"using test items",
Expand Down

0 comments on commit 9a22a3d

Please sign in to comment.