Transcribe audio files using the OpenAI Whisper (speech-to-text) API #451
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…he result. Fire this when a new attachment is added
…rate transcriptions
…ctions class to better support multiple providers, following what was done in #437
…er has access. Use this method anywhere we output our functionality. This fixes a bug our e2e tests found; thanks tests :)
iamdharmesh
previously approved these changes
May 17, 2023
There was a problem hiding this comment.
@dkotter Thanks for the amazing work and very detailed information in the PR description. PR looks good to me and it's working fine.
Just added 2 minor notes to check and we are ready to merge this.
Thanks.
| return new WP_Error( 'not_enabled', esc_html__( 'Transcripts are not enabled.', 'classifai' ) ); | ||
| } | ||
|
|
||
| return true; |
There was a problem hiding this comment.
I think we should also add a check for authenticated. without this when we don't have the API key saved and enable checkbox is checked it shows the "Transcribe audio" option in bulk actions.
// Check if valid authentication is in place.
if ( empty( $settings ) || ( isset( $settings['authenticated'] ) && false === $settings['authenticated'] ) ) {
return new WP_Error( 'auth', esc_html__( 'Please set up valid authentication with OpenAI.', 'classifai' ) );
}| textField.value = resp; | ||
| } | ||
| } | ||
| }, |
There was a problem hiding this comment.
Suggested change
| }, | |
| }, | |
| buttonText: __('Re-transcribe', 'classifai'), |
To keep the button text "Re-transcribe" after the process is complete. Currently, it becomes "Rescan".
…is enabled. Ensure button text stays correct
1 task
4 tasks
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of the Change
This PR introduces a new integration with the OpenAI Whisper (speech-to-text) API. This integration will automatically create a transcript for any supported audio files and store that transcript as the
post_contentfor the item (shows in theDescriptionfield).Workflow
A new settings section is added under
Tools > ClassifAI > Language Processing > OpenAI Whisper. Here there are three options to choose from:Once configured, whenever a valid audio file is uploaded (must be under 25 MB and the file type has to be one of: mp3, mp4, mpeg, mpga, m4a, wav, or webm) we send that file to the Whisper API and if we get a successful response back, we'll store the transcript as the
post_contentof the attachment item.For existing audio items, there's a few options to generate transcripts. You can go to the Media Library grid view and click on an audio file. Within the modal that pops up, there will be an option there to (Re-) Transcribe:
You can also go to the single media view and there will be a custom metabox with an option to (Re-) Transcribe the item. Check the box and then save the item:
If you prefer using the Media Model list view, there's a
Transcribe audiobulk edit option as well as an inlineTranscribeoption:How to test the Change
Tools > ClassifAI > Language Processing > OpenAI Whisperand configure the featureDescriptionfield has contentChangelog Entry
Credits
Props @dkotter
Checklist: