You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
I have a use case where I pre-download the models to be used with the transformers backend and drop them into a my models directory. My server does not have access to the internet. When trying to specify the model for transformers to use it tries to download the model rather than finding the model that already exists in the file system.
Describe the solution you'd like
I'd like to detect if the model has already been downloaded and tell transformers to use that path.
Describe alternatives you've considered
I considered passing the full path as the model name but this fails validation, both in the model yaml config and when trying to pass it in a request to the completions endpoint.
Additional context
I'd like this to work with the transformers backend.
This was a work around I tried, however if fails on some validation of the model name in the API. This would have worked had the model name made it's way all the way to the transformers backend. Seeing as there's already a ModelFile provided as part of the request object it seems like we should check that first rather than changing the model name validation.
Is your feature request related to a problem? Please describe.
I have a use case where I pre-download the models to be used with the transformers backend and drop them into a my models directory. My server does not have access to the internet. When trying to specify the model for transformers to use it tries to download the model rather than finding the model that already exists in the file system.
Describe the solution you'd like
I'd like to detect if the model has already been downloaded and tell transformers to use that path.
Describe alternatives you've considered
I considered passing the full path as the model name but this fails validation, both in the model yaml config and when trying to pass it in a request to the completions endpoint.
Additional context
I'd like this to work with the transformers backend.
This was a work around I tried, however if fails on some validation of the model name in the API. This would have worked had the model name made it's way all the way to the transformers backend. Seeing as there's already a ModelFile provided as part of the request object it seems like we should check that first rather than changing the model name validation.
The text was updated successfully, but these errors were encountered: