-
Notifications
You must be signed in to change notification settings - Fork 338
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
skip functional tests for models that are not downloaded. #48
Conversation
…_whisper-tiny and openai_whisper-large-v3 are downloaded.
Moved the typo fix to its own branch (for a future PR). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a nice improvement, thank you!
I'd support fixing the typo in this PR, I just re-ran your commit and it passed all the tests, it may have just failed due to a failed api request. If you want to bring it back it lmk, otherwise I can merge this as is.
It failed when building for VisionOS, which I don't have as a platform locally. |
@metropol whatever is more convenient for you, probably just reverting the commit that removes it would be simplest. |
This reverts commit 34e26bc.
@ZachNagengast I've reverted the commit the removes the typo. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good to merge 🚀
hi all, it seems that functional tests require two models to be downloaded: openai_whisper-tiny (because of
testRealTimeFactorTiny()
) and openai_whisper-large-v3 (because oftestInitLarge()
andtestRealTimeFactorLarge()
). This should probably be added to the CONTRIBUTING.md, however this PR does not make this change.Another functional test,
testOutputAll()
- actuallyallModelPaths()
, assumes that folders in./Models/whisperkit-coreml
, which are created from themake setup-model-repo
, contain actual models, while in fact only models explicitly downloaded withmake download-model MODEL=x
ormake download-models
are present.testOutputAll()
fails if not all models have been downloaded, which is not ideal. Running all tests for all models (takes a long time and) makes sense in some scenarios, but probably not to test some code changes.I've implemented some changes so that
allModelPaths()
only returns folders that contain downloaded models. This is done by checking if a proxy file (MelSpectrogram.mlmodelc/coremldata.bin
) is a git lfs pointer file (starting withversion https://…
). If it is, then the folder is not included.