Is there a way to get Audio from the camera streams?

[krasatos]

Hey there, and thank you for this great integration.
Mainly a question, secondly, if possible, a feature request.

I was wondering what's going on with sound. I have seen several posts in the community saying that several onvif streams have audio and others are muted by code.

What can i do about it?
Is there some way to enable sound?
Is it impossible?

Thank you

[JurajNyiri]

RTSP stream does support sound via VLC, I will look into this.

[JurajNyiri]

Audio from my other cameras work via Onvif integration (aac codec)
Audio from Tapo via Onvif integration (and this one) doesn't work.

So this is the codec information of Tapo camera:

This is the PR which added audio capability to Home Assistant.

Quote:
This PR muxes the audio stream into the output HLS stream if the codec is supported. If not, no audio stream is sent along with the video, saving bandwidth. Note that PCM codecs such as G.711 are not supported as they are not supported in mp4 containers.

Supported audio codecs are "aac", "ac3" and "mp3".

Unfortunately not possible. Support will have to come from Home Assistant side.

[mrwsl]

Since the PR got merged into dev: can we expect audio to be working in a future home assistant release? Never mind, can't read..

[krasatos]

@JurajNyiri sorry for bringing this up again, could you instruct me on what further steps i could take to make this happen?
If i understand correctly, we need the PCM ALAW codec to be muxed into the stream.
Right now only "aac", "ac3" and "mp3" codecs are supported.
So this could be solved by including PCM codecs in the stream, from the HA side, or Tapo firmware to use different codec right?

How could one go and request this? In the community forum of HA or somewhere in github, could you please point me to that direction?
I am guessing that changing the tapo firmware would be out of the question.

[JurajNyiri]

Yes you are correct, change would have to happen either from Home Assistant or Tapo firmware.

You can request a new feature here: https://community.home-assistant.io/c/feature-requests/13 .

Browsers seem to support PCM ALAW in webrtc https://developer.mozilla.org/en-US/docs/Web/Media/Formats/WebRTC_codecs but not in audio/video tags afaik.

[krasatos]

Thank you, i will go start a feature request on the HA-side.

[RestOp]

But is it possible just to extract this audio stream and record it into file?
I need only audio recordings not real time listening.

Or maybe you know any other way to record audio from Tapo?

Thanks a lot!

[JurajNyiri]

Yes but out of scope of this integration. You can use ffmpeg directly to output audio to file.

[RestOp]

Yes but out of scope of this integration. You can use ffmpeg directly to output audio to file.

Thanks a lot!
But can i use your itegration (i like it very much!) together with ffmpeg? Will it make any interference\problems?

Thanks a lot!

[VasekPurchart]

Hi, I don't have a camera or HA yet - doing preliminary research. From what I was reading about showing streams from cameras using RTSP and showing the stream in browser, there is currently no direct ( and native+universal) solution (https://flashphoner.com/browser-based-webrtc-stream-from-rtsp-ip-camera-with-low-latency/).

So I gather that HA is already doing some transcoding (probably using ffmpeg since it is based on the stream integration). And from what I understand it is quite simple for ffmpeg to convert the PCM audio to something usable (AAC)?

I tried this today and had a proof of concept with working sound (but only with PCM ulaw stream, could not find a alaw stream), basically using https://video.stackexchange.com/questions/29887/ffmpeg-convert-g-711-audio-to-aac-audio .

And since my result was just an browser-consumable stream, which is processed by ffmpeg in the backgound, isn't it basically the same HA does? And if so, could I just point HA to this new stream and use it directly (ideally without another ffmpeg processing)? Or is there something I am missing?

[h0jeZvgoxFepBQ2C]

@krasatos did you open a feature issue on Home Assistant? Would be nice to know the status

[eerison]

Was it solved?

Audio just works when I run the command link to save the video. But it doesn't work on browser , just the video is reproduced 😢

[ccdunder]

Awesome project! I'd love to start using it with C320WSs. I noticed this issue and want to share what I've learned over the years regarding PCM audio:

Upstream HomeAssistant cannot "fix" or change what audio codecs MP4 allows. It's defined by ISO 14496-14.

So it looks like there's at least two ways forward:

1. Transcode the audio using gstreamer, which is available in hass. I've done this for my C320WS. Happy to share the code if it would help.
2. Use a different container or stream protocol that supports PCM audio e.g. RTSP or WebRTC.

Which approach sounds best to you? (no pun intended)

[markrutten]

Awesome project! I'd love to start using it with C320WSs. I noticed this issue and want to share what I've learned over the years regarding PCM audio:

Upstream HomeAssistant cannot "fix" or change what audio codecs MP4 allows. It's defined by ISO 14496-14.

So it looks like there's at least two ways forward:

1. Transcode the audio using gstreamer, which is available in hass. I've done this for my C320WS. Happy to share the code if it would help.
2. Use a different container or stream protocol that supports PCM audio e.g. RTSP or WebRTC.

Which approach sounds best to you? (no pun intended)

Hi, I am interested in how you solved this, could you please share your solution with some explanation?
Does this solution work with the TAPA add-on from github? I have two C225 camera's.

Thanks in advance, kind regards Mark