load model from API #4301
load model from API #4301
Conversation
|
Isn't |
|
I see the get for
|
|
Also any chance you'd be able to add the variation seed/strength to the img2img post request? |
| @@ -57,6 +57,7 @@ def __init__(self, app: FastAPI, queue_lock: Lock): | |||
| self.app.add_api_route("/sdapi/v1/samplers", self.get_samplers, methods=["GET"], response_model=List[SamplerItem]) | |||
| self.app.add_api_route("/sdapi/v1/upscalers", self.get_upscalers, methods=["GET"], response_model=List[UpscalerItem]) | |||
| self.app.add_api_route("/sdapi/v1/sd-models", self.get_sd_models, methods=["GET"], response_model=List[SDModelItem]) | |||
| self.app.add_api_route("/sdapi/v1/sd-models", self.get_sd_models, methods=["POST"]) | |||
There was a problem hiding this comment.
Shouldn't this point to self.set_sd_models ?
There was a problem hiding this comment.
Yeah looks like that's the issue. Is there anyway to make it where you can specify the model to use in the txt2img and img2img request?
There was a problem hiding this comment.
You can't, but you can set it in the config, but that is broken for now
There was a problem hiding this comment.
im going to be remaking this PR to fix the config instead - this should also allow for model reloading
There was a problem hiding this comment.
There is a override settings field in img2txt and img2img request where you can use specific settings (including using a particular hypernetwork and model) for that inference, then restoring to previous state. Unfortunately, due to long loading times for the model, it doesnt work for model switching, but it is trivial enough to make it load on change. I already have an open PR to enable the hypernetwork loading - though it has not been merged yet. #4120
You can see this PR to see how to add support for reloading model.
There was a problem hiding this comment.
There is a override settings field in img2txt and img2img request where you can use specific settings (including using a particular hypernetwork and model) for that inference, then restoring to previous state. Unfortunately, due to long loading times for the model, it doesnt work for model switching, but it is trivial enough to make it load on change. I already have an open PR to enable the hypernetwork loading - though it has not been merged yet. #4120 You can see this PR to see how to add support for reloading model.
I tried overriding sd_model_checkpoint with the model title, model_name and hash but none seemed to work as the hash for each image came back with the default model hash.
There was a problem hiding this comment.
Yes as I said, this does not work now, but you can quickly patch to make it work, refer to my PR, just call the reload method for model
There was a problem hiding this comment.
Hmm, I've been thinking about this and I agree that it's probably best done as part of the generation request itself. Otherwise, a user that tries to generate with another model may end up switching the model for pending generations, unless the client has its own internal queuing mechanism (which other clients won't be able to share.)
Has anyone made a PR like #4120 for switching the model within the request itself?
There was a problem hiding this comment.
Has anyone made a PR like #4120 for switching the model within the request itself?
I can work on it in early Dec.
|
with the latest commit it should be possible to change model at runtime by just setting the setting via API like you normally do in UI so I'm closing this |
|
It's wired that I can't see the things like "/sdapi/v1/txt2img", or "/sdapi/v1/img2img"... I'm a windows usus so what I did is to modify the 'webui-user.bat' file, add a line like "set COMMANDLINE_ARGS=--xformers --autolaunch --listen --api", which allows the "launch.py" to launch the webui with these parameters. |
I've been playing around with this - POSTing to |
|
I mean, right now, I have to do POST /options and then POST /txt2img - either way, I have to load a model if the user's requesting a model switch. The two-request solution is prone to race conditions and other queuing shenanigans, unlike being able to specify it in the generation request (which is internally queued). The other issue is that I have multiple clients, not talking to each other, using the same API: they have no way of getting on the same page about the model to use, so I have to issue the options-request every time. If load times are an issue, consider adding a setting to disable the ability to dynamically load from a request? but imo, being able to switch from a generation request is a sensible default. (Thanks for the link to the PR - I can do that if all else fails, but I'd like to avoid keeping my own patches, especially as I'm a library author) |
|
Actually load time wont be an issue if you cache model in RAM. I will see if I can make a PR on this soon, but its all up to AUTO whether it gets merged (due to the concerns) |
This PR allows for the API to load models from checkpoints - it also fixes a typo