Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor deserialize_chat_template #354

Merged
merged 2 commits into from
May 28, 2024
Merged

Conversation

Jeadie
Copy link
Contributor

@Jeadie Jeadie commented May 28, 2024

Changes

  • The macro deserialize_chat_template returns two objects ChatTemplate and GenerationConfig.
  • These two objects are independent when instantiating from a impl ModelPaths.
  • Create new function to create ChatTemplate
  • Creating a GenerationConfig is a one liner, no function needed.
  • Also, fixed some documentation.

Copy link

Code Metrics Report
  ===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Dockerfile              1           34           25            0            9
 Happy                   1          442          369            0           73
 JSON                    5            9            9            0            0
 Python                 21          741          622           21           98
 TOML                   15          388          351            1           36
-------------------------------------------------------------------------------
 Jupyter Notebooks       1            0            0            0            0
 |- Markdown             1           60           30           22            8
 |- Python               1           96           87            1            8
 (Total)                            156          117           23           16
-------------------------------------------------------------------------------
 Markdown               15         1028            0          761          267
 |- BASH                 6          205          192            0           13
 |- Python               6          121          110            0           11
 |- Rust                 3          185          172            9            4
 (Total)                           1539          474          770          295
-------------------------------------------------------------------------------
 Rust                   84        28002        25642          365         1995
 |- Markdown            40          422            0          410           12
 (Total)                          28424        25642          775         2007
===============================================================================
 Total                 144        30644        27018         1148         2478
===============================================================================
  

@Jeadie
Copy link
Contributor Author

Jeadie commented May 28, 2024

The current implementation of load_model_from_path uses the chat_template from the get_template_filename regardless if the Loader has been instantiated with a chat_template: Option<String> override (e.g. GGUFLoader.

I think the correct behaviour should be that if the use has provided a specific chat_template, use it instead of the one derived from get_template_filename.

Copy link
Owner

@EricLBuehler EricLBuehler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks!

@EricLBuehler EricLBuehler merged commit 511a616 into EricLBuehler:master May 28, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants