Changeset 232014 in webkit

Timestamp:

May 21, 2018 4:52:33 AM (6 years ago)

Author:

aboya@igalia.com

Message:

[MSE][GStreamer] Force segment.start = 0 after matroskademux
​https://bugs.webkit.org/show_bug.cgi?id=185740

Reviewed by Xabier Rodriguez-Calvar.

This patch ensures that when WebM MSE media segments are appended in
an out of order fashion their frames are not discarded by opusparse or
any other potential elements downstream in the AppendPipeline that
perform segment clipping.

This patch fixes the following YTTV 2018 tests:

38. OpusAudioWithOverlap
39. OpusAudioWithSmallGap
40. OpusAudioWithLargeGap
41. VP9VideoWithOverlap
42. VP9VideoWithSmallGap

This patch is necessary, but not sufficient for fixing the following
YTTV 2018 tests:

36. AppendOpusAudioOutOfOrder
37. AppendVP9VideoOutOfOrder

• platform/graphics/gstreamer/mse/AppendPipeline.cpp:

(WebCore::AppendPipeline::connectDemuxerSrcPadToAppsink):
(WebCore::matroskademuxForceSegmentStartToEqualZero):

Location:

trunk/Source/WebCore

Files:

• ChangeLog (modified) (1 diff)
• platform/graphics/gstreamer/mse/AppendPipeline.cpp (modified) (3 diffs)

trunk/Source/WebCore/ChangeLog

@@ +1 @@
+2018-05-21  Alicia Boya García  <aboya@igalia.com>
+
+        [MSE][GStreamer] Force segment.start = 0 after matroskademux
+        https://bugs.webkit.org/show_bug.cgi?id=185740
+
+        Reviewed by Xabier Rodriguez-Calvar.
+
+        This patch ensures that when WebM MSE media segments are appended in
+        an out of order fashion their frames are not discarded by opusparse or
+        any other potential elements downstream in the AppendPipeline that
+        perform segment clipping.
+
+        This patch fixes the following YTTV 2018 tests:
+
+        38. OpusAudioWithOverlap
+        39. OpusAudioWithSmallGap
+        40. OpusAudioWithLargeGap
+        70. VP9VideoWithOverlap
+        71. VP9VideoWithSmallGap
+
+        This patch is necessary, but not sufficient for fixing the following
+        YTTV 2018 tests:
+
+        36. AppendOpusAudioOutOfOrder
+        67. AppendVP9VideoOutOfOrder
+
+        * platform/graphics/gstreamer/mse/AppendPipeline.cpp:
+        (WebCore::AppendPipeline::connectDemuxerSrcPadToAppsink):
+        (WebCore::matroskademuxForceSegmentStartToEqualZero):
+
 2018-05-19  Eric Carlson  <eric.carlson@apple.com>
 

trunk/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp

@@ -86 +86 @@
 static void appendPipelineAppsinkEOS(GstElement*, AppendPipeline*);
 
+static GstPadProbeReturn matroskademuxForceSegmentStartToEqualZero(GstPad*, GstPadProbeInfo*, void*);
+
 static void appendPipelineNeedContextMessageCallback(GstBus*, GstMessage* message, AppendPipeline* appendPipeline)
 {
@@ -979 +981 @@
     GST_DEBUG("Connecting to appsink");
 
+    const String& type = m_sourceBufferPrivate->type().containerType();
+    if (type.endsWith("webm"))
+        gst_pad_add_probe(demuxerSrcPad, GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM, matroskademuxForceSegmentStartToEqualZero, nullptr, nullptr);
+
     LockHolder locker(m_padAddRemoveLock);
     GRefPtr<GstPad> sinkSinkPad = adoptGRef(gst_element_get_static_pad(m_appsink.get(), "sink"));
@@ -1149 +1155 @@
 }
 
-
+static GstPadProbeReturn matroskademuxForceSegmentStartToEqualZero(GstPad*, GstPadProbeInfo* info, void*)
+{
+    // matroskademux sets GstSegment.start to the PTS of the first frame.
+    //
+    // This way in the unlikely case a user made a .mkv or .webm file where a certain portion of the movie is skipped
+    // (e.g. by concatenating a MSE initialization segment with any MSE media segment other than the first) and opened
+    // it with a regular player, playback would start immediately. GstSegment.duration is not modified in any case.
+    //
+    // Leaving the usefulness of that feature aside, the fact that it uses GstSegment.start is problematic for MSE.
+    // In MSE is not unusual to process unordered MSE media segments. In this case, a frame may have
+    // PTS <<< GstSegment.start and be discarded by downstream. This happens for instance in elements derived from
+    // audiobasefilter, such as opusparse.
+    //
+    // This probe remedies the problem by setting GstSegment.start to 0 in all cases, not only when the PTS of the first
+    // frame is zero.
+
+    ASSERT(info->type & GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM);
+    GstEvent* event = static_cast<GstEvent*>(info->data);
+    if (event->type == GST_EVENT_SEGMENT) {
+        GstSegment segment;
+        gst_event_copy_segment(event, &segment);
+
+        segment.start = 0;
+
+        GRefPtr<GstEvent> newEvent = adoptGRef(gst_event_new_segment(&segment));
+        gst_event_replace(reinterpret_cast<GstEvent**>(&info->data), newEvent.get());
+    }
+    return GST_PAD_PROBE_OK;
+}
 
 } // namespace WebCore.