community: Fix ChatLiteLLMRouter runtime issues #28163
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
dosubot
bot
added
size:L
|
cc: @StephenKThung |
|
@baskaryan This PR is ready to go in. Can you review and merge it? Thanks. |
| @@ -43,6 +56,39 @@ | |||
| ] | |||
|
|
|||
|
|
|||
| # from https://stackoverflow.com/a/78573267 | |||
| def aiter_to_iter(it: AsyncIterator) -> Generator: | |||
There was a problem hiding this comment.
Can we implement a synchronous _get_fake_results_generator instead of implementing this conversion?
There was a problem hiding this comment.
The aiter_to_iter() function allows reuse of the async fake results generator function for both the async and sync cases. I think having a separate synchronous function would lead to duplication of the output and output logic.
|
|
||
|
|
||
| @pytest.mark.scheduled | ||
| @pytest.mark.enable_socket |
There was a problem hiding this comment.
We don't run community integration tests in CI. is this needed?
There was a problem hiding this comment.
pytest.mark.enable_socket is needed to run the tests manually on a dev machine, since later versions of pytest disable sockets by default, and we need sockets enabled to run these tests.
dosubot
bot
added
the
lgtm
Description: Fix ChatLiteLLMRouter ctor validation and model_name parameter
Issue: #19356, #27455, #28077
Twitter handle: @bburgin_0