Slow/delay surface switching

[HugoGresse]

Since the beginning, I've noticed slow display of video on the surface when switching surface from one activty to another or when resoring it and so on.

You can see the result here : https://www.dropbox.com/s/a8p3zp4kvdwn2jf/device-2015-02-24-104234-reduced.mp4?dl=0

When chaning surface, I use NativeVideoPlayer#attach(context, frameLayout) to pass the new viewGroup where VideoSurfaceView is.

See player code https://gist.github.com/HugoGresse/2bc417a4096eb5f27879#file-nativevideoplayer

[ojw28]

It's a limiting property of the Android platform that switching surface requires the decoder to be recreated. Having recreated the decoder, it's not possible to output any video frames until the following keyframe. The delay you see is simply the renderer waiting for the next keyframe to arrive.

We could have the new decoder start decoding from the previous keyframe, discarding decoded frames up to the first frame to render, but at high resolution that's not a good solution, because the decoder may not be able to decoder at a speed much faster than real-time. We'd also need to keep video that we've already output in the buffer for some people of time, which reduces the amount of buffer we're able to keep in the forward direction, which doesn't seem like a good trade-off given surface switching isn't something that's used extensively.

So yeah, I'm not sure this is something we can really improve unless the limiting property of the Android platform is removed. You may be able to use the information about to make something that works better for your particular use case.

[HugoGresse]

Thank you for your complet answer. So apart from creating my own decoder, will keeping the same surface accross different activity will improve speed ?

Also, can you explain "because the decoder may not be able to decoder at a speed much faster than real-time" ?

[ojw28]

Re the first point: Keeping the same surface would completely eliminate the delay. But it's non-trivial to pass surfaces around. That's something you'd need to experiment with, I guess.

Re the second point: Having the decoder quickly decode from the previous keyframe up to the one you want to start rendering from on the new surface would only work without delay if the decoder were able to do that very quickly. If you're at high resolution on a low end device, the decoder may not be fast enough to do that. So you'd still get a delay anyway.

[HugoGresse]

Keeping the same surface among activities is quiet hard indeed.
The surfaceView will be destroyed anyway and cause decoder to be recreated. Is there a special flag to set to keep/lock decoder ? I've try locking canvas before android destroy the surface but it throw IllegalArgumentException (mSurfaceView.getHolder().lockCanvas();).

[ojw28]

I've never had to do this, so I'm not sure. The specific question about passing a Surface from one activity to another is more of an Android view/graphics question than something that's ExoPlayer specific, so I suspect you'll have more luck asking on StackOverflow.

[HugoGresse]

Thank you for your answer, I will keep trying to remove the delay in background, and currently switching to MediaPlayer. It will be great that this question stay open.

[HugoGresse]

Considering to write my own decoder, should I rewrite MediaCoverVideoTrackRenderer ? what will be the exacte task to do ? Exoplayer code is not well documented so it's a bit hard to undestand the purpose of all methods and members.

[ojw28]

Yes, you would need to implement your own TrackRenderer that hooks into whatever decoder you choose to use. The TrackRenderer interface that you'd need to implement is pretty well documented. As for actually implementing one, it's pretty inherently complicated. Although you can see a good example here, which is an extension TrackRenderer that hooks into a software VP9 decoder.

[HugoGresse]

I've achieving the fast transition using a TextureView which allow me to prevent Android from destroying the Surface. It still use the default renderer.
Here is the working code : https://gist.github.com/HugoGresse/669284a16ea25124228b

SO question : http://stackoverflow.com/questions/28727413/move-surfaceview-across-activities

[qqli007]

mark it,I would test it later,@qqli007

[JonWatson]

@ojw28 The need for switching surfaces is extremely common. Without configChanges 'orientation', seemless playback through rotation requires a new view/surface. configChanges 'orientation' is a poor solution for any semi-complicated design. Really surprised at this limitation of ExoPlayer.

[ojw28]

@JonWatson It's less a limitation of ExoPlayer and more a limitation of the underlying platform, which ties a decoder to a Surface and recreates the Surface on an orientation change. This means we need to recreate the decoder on an orientation switch, which in turn means we need to go back and restart decoding from the previous keyframe (or from the next one). My understanding is it would require quite large architectural changes in the platform to resolve.

[HugoGresse]

It's already discussed, if possible using a SurfaceTexture and doing this:

    @Override
    public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
        mLastTextureDestroyed = true;

        if(mSavedSurfaceTexture != null && mRequestNewAttach){
            mTextureView.setSurfaceTexture(mSavedSurfaceTexture);
            mRequestNewAttach = false;
            if(mAutoPlay){
                start();
            }
        }

        return (mSavedSurfaceTexture == null);
}

See https://developer.android.com/reference/android/view/TextureView.SurfaceTextureListener.html#onSurfaceTextureDestroyed(android.graphics.SurfaceTexture)

Additionally, you can check out my Gist here.

[ojw28]

Yes. Note however that such an approach uses significantly more battery, and isn't compatible with the secure video path (for applications that use Widevine L1 DRM). Still, it's the best approach for now, and is fine for short-running videos where battery consumption is not quite so critical.