You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently only the mean of the metric is displayed in the UI when repetitions > 1. It would be nice to have variance/std.
Motivation
Many LLM APIs (including OpenAI) can struggle to give deterministic completions. It would be nice to see the stability of metrics without looking at the individual repetitions.
The text was updated successfully, but these errors were encountered:
Feature request
Currently only the mean of the metric is displayed in the UI when repetitions > 1. It would be nice to have variance/std.
Motivation
Many LLM APIs (including OpenAI) can struggle to give deterministic completions. It would be nice to see the stability of metrics without looking at the individual repetitions.
The text was updated successfully, but these errors were encountered: