Support audio uploads on iOS?

[drscotthawley]

• [ x] I have searched to see if a similar issue already exists.

Following up on this Discussion and request to add Issue by @abidlabs :

https://github.com/gradio-app/gradio/discussions/3998

iOS is a little funny and while the way Gradio configures Audio input elements should be ok, somehow it's not.

Pasting from the above-mentioned Discussion:

Hi, I have a new audio-in, audio-out Gradio demo I'm working on (sorry, can't share any details yet).

But I notice that if I'm on my iOS devices, when I press the audio upload box ("Drop Audio here -or- click to Upload") and pick "Choose File", and go through the list of files to upload into the audio box, all the (.wav, .mp3, .aiff) audio files are "greyed out", i.e. I'm not allowed to select any to upload. Doesn't matter if I'm selecting files in iCloud Drive, or on my Device, or in Dropbox: All audio files are non-selectable.

I'm not sure if this is an iOS limitation or a Gradio issue....? Can anyone help?

This happens when using Safari or Chrome on my iPhone or iPad, pretty recent iOS versions. Brave browser on latest iPad version behaves similarly.

(No problems running from desktop!)

---

Through experimentation*, I find that this seems to be a bug(/feature) in iOS whereby HTML input field with accept="audio/*" does not actually allow audio to be selected on iOS.

If Gradio wanted support audio uploads on iOS, one workaround on your end end could be to either change the accept string generated for the input field , either to accept="/" or to list each allowed file extension e.g accept="audio/mp3, audio/wav,..." etc. @abidlabs @yvrjsharma

(You shouldn't need to do this but iOS is a part of the market, and this change would still be compatible with other devices.)

(*Source. Trying it manually, on this page: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/file )

[abidlabs]

Thanks for creating the issue @drscotthawley will look into it!

[abidlabs]

Btw would you like to open a PR since it looks like you’ve already started with the solution? Otherwise we can do it on our end

[drscotthawley]

@abidlabs Re. the PR, I'd be happy to try but I'm not sure where in your code you're supplying that (MIME?) type information.

I took a look at the Audio element in components.py but didn't find any place to implement the change, e.g. I couldn't find "accept=" anywhere.

If one of your engineers could suggest a place to look, I could try again.

[abidlabs]

Thanks @drscotthawley, what a coincidence! @aliabid94 on the Gradio team just opened a PR for this if you'd like to take a look: #4071

[drscotthawley]

@abidlabs Oh yes!!! Thanks @aliabid94!
I went to the PR and tried the "all demos" example on my iPhone and it worked fine! :-) We could close this Issue with a "fixed in PR #4071" ...?
maybe I'll wait for the merge and test it in my app first. ;-)

[drscotthawley]

Oh darn, I pulled the PR and put it in my app but got this message:

ValueError: Did you install Gradio from source files? Share mode only works when Gradio is installed through the pip package.

@abidlabs Do let me know when the pip package gets updated! :-)

[drscotthawley]

Related, FYI @abidlabs @aliabid94 : This Issue was about Uploading audio on iOS. What about Downloading audio on iOS?....

Answer: Turns out there appears to be no way to get the HTML5 Audio element controls on iOS to support a Download option. Period.

Users who want to Download audio results will either have to stick to desktop browsers, or developers of Gradio demos who want to support audio downloads on iOS can either supply their own separate "Download Audio" button, or just post an HREF-style link for the output audio file (presumably by using the elem_id of a gr.Markdown() element.)

Just registering that information here in case anyone else asks.

[Alanp777]

@drscotthawley do you know if we will be able to playback audio in ios? my users have been complaining about that, would like to see if I can contribute or be any help on that, and yes, for some reason gradio on mozilla firefox looks so nice including multiple functionalities like downloading the audio, in chrome is similar but you download the audio in a weird format that you can only reproduce using VLC

[Alanp777]

@abidlabs also I dont mean to piggyback this but is there any way to speed up the audio? Ive been trying to do some code and research all options but for some reason gradio doesnt interpret all of the 6 strategies i used, I will keep looking around, and actually I provided the other day the documentation about being able to download audio from browsers, so im not only trying to extract value, will try to do my best to add and contribute more to the community, THANKS so much!

[drscotthawley]

@Alanp777 Not sure what you mean: I have no problem playing back in iOS.
Either using numpy arrays or filepath, works fine. (I recommend filepath as it seems to be more versatile, e.g. works with gr.Examples.)

[Alanp777]

@Alanp777 no it doesnt work, try it yourself https://huggingface.co/spaces/WXM2000/audio_to_text_Chinese

or any other space, try to playback the audio on Safari browser, also my users complain about the same

[drscotthawley]

@Alanp777 All my apps work fine. been doing it for weeks. on iOS. And I just did tried link you sent and it worked fine: recorded myself played it back. On my iPhone. Safari. 🤷 On May 7, 2023, at 4:24 PM, Alanp777 ***@***.***> wrote:  @Alanp777<https://github.com/Alanp777> no it doesnt work, try it yourself https://huggingface.co/spaces/WXM2000/audio_to_text_Chinese or any other space, try to playback the audio on Safari browser, also my users complain about the same — Reply to this email directly, view it on GitHub<#4021 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADKH2NOTALOUOXXNDXJLDZTXFAHHXANCNFSM6AAAAAAXRNJ6Q4>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[Alanp777]

@drscotthawley my apologies, I should have specified, Safari Browser on DESKTOP, not iOS, my mistake.

[drscotthawley]

@Alanp777 No, that works fine for me too. Just tried it. On my Mac. In Safari. ✅

Is perhaps your default audio device assigned incorrectly in System Preferences?

[drscotthawley]

Confirming that iOS audio uploads now work in PyPI gradio==3.29.0. Thanks! :-)

[abidlabs]

Great, thanks for confirming @drscotthawley!