Skip to content
This repository has been archived by the owner on Apr 8, 2024. It is now read-only.

[hold] Add batch latency measurement and percentiles to treelite_python #177

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open

[hold] Add batch latency measurement and percentiles to treelite_python #177

wants to merge 19 commits into from

Conversation

jfomhover
Copy link
Contributor

@jfomhover jfomhover commented Nov 24, 2021 •
edited
Loading

This PR introduces num_threads and batch_size as parameters for inferencing benchmark.

Additions:

  • code to process batches of data in the treelite_python module,
  • unified code to record latency metrics and percentiles.
  • num_threads and batch_size in inferencing variant parameters
  • num_threads and batch_size in the inferencing pipeline (depending on support from each component)

@jfomhover jfomhover added the enhancement New feature or request label Nov 24, 2021
@jfomhover jfomhover added this to the Standardization milestone milestone Nov 24, 2021
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 07:08 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 07:11 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 07:14 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 17:07 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:22 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:23 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:31 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:44 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:46 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:49 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:59 Inactive
@jfomhover jfomhover mentioned this pull request Nov 24, 2021
Closed
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 19:03 Inactive
@github-actions
Copy link

github-actions bot commented Nov 24, 2021 •
edited
Loading

Unit Test Results for Build

  1 files  ±0    1 suites  ±0   7s ⏱️ ±0s
76 tests ±0  76 ✔️ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 51c375b. ± Comparison against base commit 7b8a339.

♻️ This comment has been updated with latest results.

@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 20:31 Inactive
@jfomhover jfomhover mentioned this pull request Nov 24, 2021
Merged
@jfomhover jfomhover temporarily deployed to mlops November 25, 2021 05:12 Inactive
@jfomhover jfomhover changed the title Add batch latency measurement and percentiles to treelite_python [hold] Add batch latency measurement and percentiles to treelite_python Nov 29, 2021
@jfomhover jfomhover added the hold This PR/Issue should be put on hold. label Nov 29, 2021
@jfomhover
Copy link
Contributor Author

Split changes into a PR to unify metrics first, waiting for #180 merged

@jfomhover jfomhover temporarily deployed to mlops December 3, 2021 19:00 Inactive
@jfomhover jfomhover temporarily deployed to mlops December 3, 2021 19:02 Inactive
@jfomhover jfomhover removed this from the Standardization milestone milestone Feb 10, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
enhancement New feature or request hold This PR/Issue should be put on hold.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@jfomhover