RTMP Playback Delayed On Tracks Without Audio

[tylerjroach]

Issue description

If an rtmp stream does not have an audio track, the video will take x amount of time to display a video frame, where x is the amount of time the stream has been running.
Ex. if the rtmp stream was started 100 seconds ago, it will take Exoplayer 100 seconds to show a video frame.

The reason this occurs is because MediaCodecVideoRenderer does not extend MediaClock. With the lack of a media clock for the stream, the time source used will be from standaloneMediaClock in ExoPlayerImplInternal. standaloneMediaClock starts at a 0, so the incoming frames will be considered too early and not render until standaloneMediaClock hits the time of the frame.

if (rendererMediaClockSource != null && !rendererMediaClockSource.isEnded()) {
    rendererPositionUs = rendererMediaClock.getPositionUs();
    standaloneMediaClock.setPositionUs(rendererPositionUs);
} else {
    rendererPositionUs = standaloneMediaClock.getPositionUs();
}

Potential Solution

A quick fix is to set a start offset in FLVExtractor so that our frames start coming in at 0. I realized this is probably a hacky method and you lose the ability to know how long the stream has been running. This may not be a big deal as RTMP streams are already not seekable and do not show a duration. I'd be happy to submit a pull request if this is an acceptable solution.

Note: I'm storing the previous timestamp to re-use in the event that I get timestamps of 0 midstream. I've noticed that I have a tagTimestampUs of 0 if the rtmp server sends a header of type small. Not sure if this is an issue/bug. I didn't notice any playback issues, but it didn't feel right to pass on 0 timestamps to the readers.

/**
 * Reads the body of a tag from the provided {@link ExtractorInput}.
 *
 * @param input The {@link ExtractorInput} from which to read.
 * @return True if the data was consumed by a reader. False if it was skipped.
 * @throws IOException If an error occurred reading or parsing data from the source.
 * @throws InterruptedException If the thread was interrupted.
 */
private boolean readTagData(ExtractorInput input) throws IOException, InterruptedException {
  boolean wasConsumed = true;
  long adjustedTimestamp;
  if (tagTimestampUs > 0) {
    if (tagTimestampOffsetUs == 0 || tagTimestampUs < tagTimestampOffsetUs) {
      tagTimestampOffsetUs = tagTimestampUs;
    }
    adjustedTimestamp = Math.max(0, tagTimestampUs - tagTimestampOffsetUs);
  } else {
    adjustedTimestamp = previousAdjustedTimestamp;
  }
  previousAdjustedTimestamp = adjustedTimestamp;
  if (tagType == TAG_TYPE_AUDIO && audioReader != null) {
    audioReader.consume(prepareTagData(input), adjustedTimestamp);
  } else if (tagType == TAG_TYPE_VIDEO && videoReader != null) {
    videoReader.consume(prepareTagData(input), adjustedTimestamp);
  } else if (tagType == TAG_TYPE_SCRIPT_DATA && metadataReader != null) {
    metadataReader.consume(prepareTagData(input), adjustedTimestamp);
  } else {
    input.skipFully(tagDataSize);
    wasConsumed = false;
  }
  bytesToNextTagHeader = 4; // There's a 4 byte previous tag size before the next header.
  parserState = STATE_SKIPPING_TO_TAG_HEADER;
  return wasConsumed;
}

It looks like the better solution would be to use MediaPeriod similar to how HLS does.
I noticed that HLS sets the initial time at this line:
resetRendererPosition(readingPeriodHolder.info.startPositionUs);
I don't have a solution yet for this method.

Reproduction steps

Play any rtmp video without an audio track.

Link to test content

I was manually starting rtmp videos without audio and I can't leave it up 24x7. I can coordinate a time if anyone would like to test.

Version of ExoPlayer being used

2.5.1

Device(s) and version(s) of Android being used

All

[PanCrucian]

@tylerjroach can u please present a full code of com.google.android.exoplayer2.extractor.flv.FlvExtractor.class ?