Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

model_info for llm router config #620

Merged
merged 1 commit into from
Jul 18, 2024
Merged

Conversation

ykeremy
Copy link
Contributor

@ykeremy ykeremy commented Jul 18, 2024

🚀 This description was created by Ellipsis for commit 2141b19

Summary:

Cleaned up GPT-4 routers and added model_info field to Azure GPT router configurations.

Key points:

  • Removed get_llm_router_config_for_cloud from cloud/__init__.py and cloud/llm/router.py.
  • Added model_info field to LLMRouterModelConfig in skyvern/forge/sdk/api/llm/models.py.
  • Updated get_gpt_turbo_router_config, get_llm_router_config_for_gpt4o, and get_massive_llm_router_config_for_gpt4o to include model_info.
  • Removed ENABLE_AZURE_GPT4V_ROUTER from cloud/config.py.
  • Added MAX_PARALLEL_REQUESTS to massive Azure GPT-4O configurations in cloud/config.py.
  • Updated scripts/run_task.py to use get_llm_api_handler_with_router for massive-gpt4o.

Generated with ❤️ by ellipsis.dev

<!-- ELLIPSIS_HIDDEN -->

| 🚀 | This description was created by [Ellipsis](https://www.ellipsis.dev) for commit afb421b1830123c03fc5e5c5ee8ecb0ac24826e5  |
|--------|--------|

### Summary:
Cleaned up GPT-4 routers and added `model_info` field to Azure GPT router configurations.

**Key points**:
- Removed `get_llm_router_config_for_cloud` function and related configurations from `cloud/__init__.py` and `cloud/llm/router.py`.
- Added `model_info` field to `LLMRouterModelConfig` in `skyvern/forge/sdk/api/llm/models.py`.
- Updated `get_gpt_turbo_router_config`, `get_llm_router_config_for_gpt4o`, and `get_massive_llm_router_config_for_gpt4o` functions to include `model_info` field.
- Removed `ENABLE_AZURE_GPT4V_ROUTER` setting from `cloud/config.py`.
- Added `MAX_PARALLEL_REQUESTS` to massive Azure GPT-4O configurations in `cloud/config.py`.
- Updated `scripts/run_task.py` to use `get_llm_api_handler_with_router` for `massive-gpt4o` model.

----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)

<!-- ELLIPSIS_HIDDEN -->
@ykeremy ykeremy added the sync label Jul 18, 2024
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to 2141b19 in 23 seconds

More details
  • Looked at 19 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. skyvern/forge/sdk/api/llm/models.py:30
  • Draft comment:
    The implementation of model_info with field(default_factory=dict) is correct and follows best practices for handling mutable default arguments in dataclasses.
  • Reason this comment was not posted:
    Confidence changes required: 0%
    The PR adds a new field model_info to the LLMRouterModelConfig class. This field is initialized with a default empty dictionary using field(default_factory=dict). This is a standard practice for mutable default arguments in dataclasses to avoid shared mutable defaults across instances. The implementation here is correct and follows Python best practices for dataclasses.

Workflow ID: wflow_tJbZj32Aay4plCIj


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 2141b19 in 28 seconds

More details
  • Looked at 18 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. skyvern/forge/sdk/api/llm/models.py:30
  • Draft comment:
    The implementation of model_info with field(default_factory=dict) is correct and follows best practices for handling mutable default arguments in dataclasses.
  • Reason this comment was not posted:
    Confidence changes required: 0%
    The PR adds a new field model_info to the LLMRouterModelConfig class. This field is initialized with a default empty dictionary using field(default_factory=dict). This is a standard practice for mutable default arguments in dataclasses to avoid shared mutable defaults across instances. The implementation here is correct and follows Python best practices for dataclasses.

Workflow ID: wflow_NSDBpwzgDJAnCAdp


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@wintonzheng wintonzheng merged commit fd9419b into main Jul 18, 2024
2 checks passed
@wintonzheng wintonzheng deleted the shu/cleanup_gpt4_routers branch July 18, 2024 22:37
@wintonzheng wintonzheng changed the title clean up gpt4 routers. add model_info base_model for azure gpt router model_info for llm router config Jul 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants