Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support v-prediction checkpoint models when set in the model manager #7504

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

dunkeroni
Copy link
Contributor

Summary

Capture prediction type from model settings and pass it to the scheduler to enable v_prediction checkpoints to work again.

Related Issues / Discussions

I'm certain there is a more correct way to handle this in the model manager that would also allow automatic detection from checkpoint files, but this at least gets vpred models working again before we go make sweeping changes to the model probe code.

Since our conversion code does not check the settings on the checkpoint, attempting to convert a v_prediction model to Diffusers will still default over to epsilon prediction and it will forever fail afterwards. It is not currently possible for users to change a diffusers model to v_prediction in Invoke without editing the local json files. User Mewt on discord has found that huggingface repos with correct scheduler_config.json are able to work without issues, but I have found others online that still specify epsilon.

Closes #7495

QA Instructions

Before:
image

After:
image

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

@github-actions github-actions bot added python PRs that change python files invocations PRs that change invocations labels Dec 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
invocations PRs that change invocations python PRs that change python files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[bug]: SDXL based V-pred models are treated as epsilon prediction - result noisy images
1 participant