Skip to content

(Single-card) Model perf tests #7129

(Single-card) Model perf tests

(Single-card) Model perf tests #7129

Triggered via schedule January 12, 2025 14:00
Status Failure
Total duration 30m 45s
Artifacts 6

perf-models.yaml

on: schedule
build-artifact  /  ...  /  build-docker-image
47s
build-artifact / build-docker-image / build-docker-image
Matrix: build-artifact / build-artifact
Matrix: models-perf / models-perf
Fit to window
Zoom out
Zoom in

Annotations

7 errors, 10 warnings, and 30 notices
models-perf / other N300 WH B0
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
fail-to-clear-disk-startup
Disk usage is still high. Will reboot. Please let the infra team know.
models-perf / other N300 WH B0
The operation was canceled.
models-perf / cnn_javelin N300 WH B0
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
fail-to-clear-disk-startup
Disk usage is still high. Will reboot. Please let the infra team know.
models-perf / cnn_javelin N300 WH B0
The operation was canceled.
models-perf / other GS
Process completed with exit code 1.
models-perf / llm_javelin N300 WH B0
Your workflow is using a version of actions/cache that is scheduled for deprecation, actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2. Please update your workflow to use either v3 or v4 of actions/cache to avoid interruptions. Learn more: https://github.blog/changelog/2024-12-05-notice-of-upcoming-releases-and-breaking-changes-for-github-actions/#actions-cache-v1-v2-and-actions-toolkit-cache-package-closing-down
models-perf / cnn_javelin GS
Your workflow is using a version of actions/cache that is scheduled for deprecation, actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2. Please update your workflow to use either v3 or v4 of actions/cache to avoid interruptions. Learn more: https://github.blog/changelog/2024-12-05-notice-of-upcoming-releases-and-breaking-changes-for-github-actions/#actions-cache-v1-v2-and-actions-toolkit-cache-package-closing-down
weka-mount-hugepages-service-not-found
Hugepages service not found. Using old rc.local method
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
models-perf / llm_javelin GS
Your workflow is using a version of actions/cache that is scheduled for deprecation, actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2. Please update your workflow to use either v3 or v4 of actions/cache to avoid interruptions. Learn more: https://github.blog/changelog/2024-12-05-notice-of-upcoming-releases-and-breaking-changes-for-github-actions/#actions-cache-v1-v2-and-actions-toolkit-cache-package-closing-down
weka-mount-hugepages-service-not-found
Hugepages service not found. Using old rc.local method
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
models-perf / other GS
Your workflow is using a version of actions/cache that is scheduled for deprecation, actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2. Please update your workflow to use either v3 or v4 of actions/cache to avoid interruptions. Learn more: https://github.blog/changelog/2024-12-05-notice-of-upcoming-releases-and-breaking-changes-for-github-actions/#actions-cache-v1-v2-and-actions-toolkit-cache-package-closing-down
weka-mount-hugepages-service-not-found
Hugepages service not found. Using old rc.local method
hugepages-service-not-found-startup
Hugepages service not found. Using old rc.local method
printing-out-smi-info-cleanup
Touching and printing out SMI info
printing-out-smi-info-cleanup
Touching and printing out SMI info
disk-usage-before-startup
Disk usage is 95 %
disk-usage-after-startup
Disk usage is 95 %
printing-smi-info-startup
Touching and printing out SMI info
disk-usage-before-startup
Disk usage is 40 %
disk-usage-after-startup
Disk usage is 40 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
hugepages-setup-success-startup
Hugepages is now setup.
disk-usage-before-startup
Disk usage is 95 %
disk-usage-after-startup
Disk usage is 95 %
printing-smi-info-startup
Touching and printing out SMI info
disk-usage-before-startup
Disk usage is 40 %
disk-usage-after-startup
Disk usage is 40 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
hugepages-setup-success-startup
Hugepages is now setup.
weka-mount-hugepages-service-found
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
disk-usage-before-startup
Disk usage is 63 %
disk-usage-after-startup
Disk usage is 63 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
hugepages-service-found-startup
Hugepages service found. Command returned with exit code 3. Restarting it so we can ensure hugepages are available
hugepages-setup-success-startup
Hugepages is now setup.
disk-usage-before-startup
Disk usage is 40 %
disk-usage-after-startup
Disk usage is 40 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
hugepages-setup-success-startup
Hugepages is now setup.

Artifacts

Produced during runtime
Name Size
TTMetal_build_grayskull
319 MB
TTMetal_build_wormhole_b0
319 MB
perf-report-csv-cnn_javelin-grayskull-bare_metal
332 Bytes
perf-report-csv-llm_javelin-grayskull-bare_metal
734 Bytes
perf-report-csv-llm_javelin-wormhole_b0-bare_metal
1.18 KB
perf-report-csv-other-grayskull-bare_metal
1.25 KB