Implement DPM-Solver++ scheduler #59
Conversation
Restricted to order 2, no dynamic thresholding.
|
Thank you for your interest in contributing to Core ML Stable Diffusion! Please review CONTRIBUTING.md. If you would like to proceed with this pull request, please indicate your agreement to the terms outlined in CONTRIBUTING.md by checking the box below. We appreciate your interest in the project!
|
|
| } | ||
|
|
||
| self.alphas = betas.map({ 1.0 - $0 }) | ||
| var alphasCumProd = self.alphas |
There was a problem hiding this comment.
This will potentially create a copy. Should use the property directly instead.
| var alphasCumProd = self.alphas | |
| self.alphasCumProd = self.alphas |
There was a problem hiding this comment.
But alphasCumProd is a let var. This structure was taken from the original PNDM implementation: https://github.com/apple/ml-stable-diffusion/blob/main/swift/StableDiffusion/pipeline/Scheduler.swift#L68
| self.alphasCumProd = alphasCumProd | ||
|
|
||
| // Currently we only support VP-type noise shedule | ||
| self.alpha_t = self.alphasCumProd.map { sqrt($0) } |
There was a problem hiding this comment.
Use Accelerate to compute this, see sqrt
swift/StableDiffusion/pipeline/DPMSolverMultistepScheduler.swift
Outdated
Show resolved
Hide resolved
Co-authored-by: Alejandro Isaza <167236+alejandro-isaza@users.noreply.github.com>
… into dpmpp-solver
| self.alpha_t = vForce.sqrt(self.alphasCumProd) | ||
| self.sigma_t = vForce.sqrt(vDSP.subtract([Float](repeating: 1, count: self.alphasCumProd.count), self.alphasCumProd)) |
There was a problem hiding this comment.
I added Accelerate, as suggested.
Personally, I believe the loss in legibility is not worth the potential performance gains in this case, especially in the line that computes sigma_t. This is just the initializer for the scheduler and not part of the denoising loop.
If there's a way to make it clearer I'd be happy to apply it.
There was a problem hiding this comment.
I agree. But I think it's good either way.
|
Thanks for adding this scheduler! It would be nice to perhaps add a |
|
@msiracusa great idea, just added that option :) |
I just tested at it works great. Thank you! |
* commit 'e07c4d00c387840f70fa3701fb3a51c2a32f37b8': Move guidanceScale as generation parameter (apple#46) Add brief instructions to download weights from the Hub (apple#10) Adds Negative Prompts (apple#61) Changed seed type into UInt32 (apple#47) fixes apple#77 Update README.md (apple#66) Add Filename Character Limit (apple#19) Implement DPM-Solver++ scheduler (apple#59) Fix typos: Successfully facilitate getting pipeline overridden (apple#30) Undefined name: from typing import List (apple#31) Add Availability Annotations (apple#18) README improvements and reduceMemory option in Swift
Restricted to order 2, no dynamic thresholding.
It achieves similar quality to PNDM in 25 steps (~10s in my Mac).