Update the server for AIdventure 2.0.0 #29
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
config.py: - Removed TOKENIZERS_PATH. model.py: - __init__ now expect a fourth parameter, model_path. - _model_path is now set to p_model_path. - Removed _tokenizers_path. A model's files are now all in the same folder. - Extracted _load_translator to the Translator class to use polymorphism. - Updated _load to always call _load_translator no matter the type of the model. The specific behavior is implemented in children if needed (See Translator.py) - _load_tokens and _save_tokens now use _model_path. server.py: - Updated handle_request to receive the models path in p_data and to use it in the model constructor. translator.py: - Added _load_model method (extracted from model.py. Old name "_load_translator")
- Fixed an error in create_offload_folder: - It was the value of config.OFFLOAD_FOLDER to the temporary directory object while it should be a string. So, when creating a new AI (e.g loading another generator) it was leading to a type error. Moreover, Consistency of this 'temp' folder isn't needed (at least for now).
- Deleted model_type (useless, thanks to polymorphism) model.py: - The class now uses polymorphism. - Removed model_type and all the check related to the type of object. - Extracted all the code related to the Generator to the Generator class. - Extracted all the code related to the Translator to the Translator class. - _set_model_parameters renamed _set_parameters. - Updated __init__ - Removed the p_model_type parameter - p_model_path is now the second parameter of __init__. p_parameters the third. - Added a log message to display the model's name and its path. - Added a log message to display if cuda is supported. generator.py: - The class now uses polymorphism. - Added torch_dtype and more transformers imports. - Added _set_parameters and _download_model methods inherited from Model. translator.py: - Added Logger and transformers imports. - Now uses polymorphism - Added _set_parameters and _download_model methods inherited from Model. server.py: - Removed model_type import and all its use in the code. - Simplified handle_request by removing code repetition and extracting part of code to specific functions. - Added load_translator and load_generator functions. request.py: - LOAD_MODEL becomes LOAD_GENERATOR. - Added LOAD_TRANSLATOR (value 8)
…meters - generate_text now longer receives memory and context as parameters. They are embedded in the prompt parameter by the client.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.