You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'd like to be able to measure the following for every trace:
Total number of tokens generated
The time from when the model started computation to when the first token was generated
The time from when the first token was generated to the time when the last token was generated
The time from when the model started computation to when the last token was generated
Average speed of computation: (end_time - start_time) / num_tokens
I believe item 1 is currently available in the product. I'm not sure about 2. I am pretty sure 3 - 5 are unavailable. I'm most interested in measuring items 2 and 3. I can get item 2 directly from the LLM service I'm using, but its hard to find out 3 without doing some data munging myself.
I'd also like to see averages and histograms of those charts computed over a collection, e.g. all traces between Oct 1 2024 and Nov 1 2024
Finally, I'd like to be able to see charts which display how the average has changed over time, daily/weekly/monthly as well as over an arbitrary period (such as by 14-day chunks). I'm most interested in this analysis for items 2 and 3 from the top of this idea.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Describe the feature or potential improvement
I'd like to be able to measure the following for every trace:
(end_time - start_time) / num_tokens
I believe item 1 is currently available in the product. I'm not sure about 2. I am pretty sure 3 - 5 are unavailable. I'm most interested in measuring items 2 and 3. I can get item 2 directly from the LLM service I'm using, but its hard to find out 3 without doing some data munging myself.
I'd also like to see averages and histograms of those charts computed over a collection, e.g. all traces between Oct 1 2024 and Nov 1 2024
Finally, I'd like to be able to see charts which display how the average has changed over time, daily/weekly/monthly as well as over an arbitrary period (such as by 14-day chunks). I'm most interested in this analysis for items 2 and 3 from the top of this idea.
Additional information
No response
Beta Was this translation helpful? Give feedback.
All reactions