Changeset 159335 in webkit


Ignore:
Timestamp:
Nov 15, 2013 7:05:48 AM (10 years ago)
Author:
commit-queue@webkit.org
Message:

[GStreamer] Add support for Media Source API
​https://bugs.webkit.org/show_bug.cgi?id=99065

Source/WebCore:

The patch integrate a Media Source player for the GStreamer backend. The implementented architecture is:

  • MediaPlayerPrivateGStreamer engine is modified to support Media Source URIs (change blob:// to mediasourceblob://), in addition to the existing support for web (http/https/blob) URIs
  • Similar to the existing WebKitWebSrc gstreamer element that handles web URIs, a new gstreamer element named WebKitMediaSrc is implemented to handle Media Source URIs
  • WebKitMediaSrc registers its URI protocol handler, allowing uridecodebin to dynamically create the appropriate source element.
  • The WebKitMediaSrc element creates a bin with 2 appsrc: One for Audio and One for Video. Pads are dynamically linked at the reception of first video and audio buffers.

Patch by Stephane Jadaud <​sjadaud@sii.fr> on 2013-11-15
Reviewed by Philippe Normand.

Tests: Activate http/tests/media/media-source and media/media-source tests

  • GNUmakefile.am:
  • GNUmakefile.list.am:
  • platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:

(WebCore::initializeGStreamerAndRegisterWebKitElements):
(WebCore::MediaPlayerPrivateGStreamer::load):

  • platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
  • platform/graphics/gstreamer/MediaSourceGStreamer.cpp: Added.

(WebCore::MediaSourceGStreamer::open):
(WebCore::MediaSourceGStreamer::MediaSourceGStreamer):
(WebCore::MediaSourceGStreamer::~MediaSourceGStreamer):
(WebCore::MediaSourceGStreamer::addSourceBuffer):
(WebCore::MediaSourceGStreamer::setDuration):
(WebCore::MediaSourceGStreamer::markEndOfStream):
(WebCore::MediaSourceGStreamer::unmarkEndOfStream):

  • platform/graphics/gstreamer/MediaSourceGStreamer.h: Added.
  • platform/graphics/gstreamer/SourceBufferPrivateGStreamer.cpp: Added.

(WebCore::SourceBufferPrivateGStreamer::SourceBufferPrivateGStreamer):
(WebCore::SourceBufferPrivateGStreamer::append):
(WebCore::SourceBufferPrivateGStreamer::abort):
(WebCore::SourceBufferPrivateGStreamer::removedFromMediaSource):

  • platform/graphics/gstreamer/SourceBufferPrivateGStreamer.h: Added.
  • platform/graphics/gstreamer/WebKitMediaSourceGStreamer.cpp: Added.

(webKitMediaSrcAddSrc):
(webkit_media_src_init):
(webKitMediaSrcFinalize):
(webKitMediaSrcSetProperty):
(webKitMediaSrcGetProperty):
(webKitMediaVideoSrcStop):
(webKitMediaAudioSrcStop):
(webKitMediaVideoSrcStart):
(webKitMediaAudioSrcStart):
(webKitMediaSrcChangeState):
(webKitMediaSrcQueryWithParent):
(webKitMediaSrcUriGetType):
(webKitMediaSrcGetProtocols):
(webKitMediaSrcGetUri):
(webKitMediaSrcSetUri):
(webKitMediaSrcUriHandlerInit):
(webKitMediaVideoSrcNeedDataMainCb):
(webKitMediaAudioSrcNeedDataMainCb):
(webKitMediaVideoSrcNeedDataCb):
(webKitMediaAudioSrcNeedDataCb):
(webKitMediaVideoSrcEnoughDataMainCb):
(webKitMediaAudioSrcEnoughDataMainCb):
(webKitMediaVideoSrcEnoughDataCb):
(webKitMediaAudioSrcEnoughDataCb):
(webKitMediaVideoSrcSeekMainCb):
(webKitMediaAudioSrcSeekMainCb):
(webKitMediaVideoSrcSeekDataCb):
(webKitMediaAudioSrcSeekDataCb):
(webKitMediaSrcSetMediaPlayer):
(webKitMediaSrcSetPlayBin):
(MediaSourceClientGstreamer::MediaSourceClientGstreamer):
(MediaSourceClientGstreamer::~MediaSourceClientGstreamer):
(MediaSourceClientGstreamer::didReceiveDuration):
(MediaSourceClientGstreamer::didReceiveData):
(MediaSourceClientGstreamer::didFinishLoading):
(MediaSourceClientGstreamer::didFail):

  • platform/graphics/gstreamer/WebKitMediaSourceGStreamer.h: Added.

Tools:

Patch by Stephane Jadaud <​sjadaud@sii.fr> on 2013-11-15
Reviewed by Philippe Normand.

  • Scripts/webkitperl/FeatureList.pm:

LayoutTests:

Patch by Stephane Jadaud <​sjadaud@sii.fr> on 2013-11-15
Reviewed by Philippe Normand.

  • platform/gtk/TestExpectations:
Location:
trunk
Files:
6 added
9 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r159331 r159335  
     12013-11-15  Stephane Jadaud  <sjadaud@sii.fr>
     2
     3        [GStreamer] Add support for Media Source API
     4        https://bugs.webkit.org/show_bug.cgi?id=99065
     5
     6        Reviewed by Philippe Normand.
     7
     8        * platform/gtk/TestExpectations:
     9
    1102013-11-15  Michał Pakuła vel Rutka  <m.pakula@samsung.com>
    211

  • trunk/LayoutTests/platform/gtk/TestExpectations

    r159330 r159335  
    250250webkit.org/b/117039 media/track/video-track.html [ Skip ]
    251251
    252 # Tests for MediaSource API. Feature is not yet functional.
    253 webkit.org/b/99065 http/tests/media/media-source [ Skip ]
     252# Tests for MediaSource API. Feature is not totally functional.
     253webkit.org/b/99065 http/tests/media/media-source/mediasource-addsourcebuffer.html [ Failure Timeout ]
     254webkit.org/b/99065 http/tests/media/media-source/mediasource-append-buffer.html [ Crash ]
     255webkit.org/b/99065 http/tests/media/media-source/mediasource-buffered.html [ Failure ]
     256webkit.org/b/99065 http/tests/media/media-source/mediasource-closed.html [ Failure ]
     257webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-mp4-a-bitrate.html [ Failure ]
     258webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-mp4-av-audio-bitrate.html [ Failure ]
     259webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-mp4-av-framesize.html [ Failure ]
     260webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-mp4-av-video-bitrate.html [ Failure ]
     261webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-mp4-v-bitrate.html [ Failure ]
     262webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-mp4-v-framerate.html [ Failure ]
     263webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-mp4-v-framesize.html [ Failure ]
     264webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-webm-a-bitrate.html [ Failure ]
     265webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-webm-av-audio-bitrate.html [ Failure ]
     266webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-webm-av-framesize.html [ Failure ]
     267webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-webm-av-video-bitrate.html [ Failure ]
     268webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-webm-v-bitrate.html [ Failure ]
     269webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-webm-v-framerate.html [ Failure ]
     270webkit.org/b/99065 http/tests/media/media-source/mediasource-config-change-webm-v-framesize.html [ Failure ]
     271webkit.org/b/99065 http/tests/media/media-source/mediasource-is-type-supported.html [ Failure ]
     272webkit.org/b/99065 http/tests/media/media-source/mediasource-play.html [ Failure ]
     273webkit.org/b/99065 http/tests/media/media-source/mediasource-sourcebufferlist.html [ Crash ]
     274webkit.org/b/99065 http/tests/media/media-source/seek-to-end-after-duration-change.html [ Timeout ]
     275webkit.org/b/99065 http/tests/media/media-source/video-media-source-abort.html [ Timeout ]
     276webkit.org/b/99065 http/tests/media/media-source/video-media-source-add-and-remove-buffers.html [ Timeout ]
     277webkit.org/b/99065 http/tests/media/media-source/video-media-source-append-in-ended-state.html [ Timeout ]
     278webkit.org/b/99065 http/tests/media/media-source/video-media-source-append-with-offset-in-ended-state.html [ Timeout ]
     279webkit.org/b/99065 http/tests/media/media-source/video-media-source-async-events.html [ Timeout ]
     280webkit.org/b/99065 http/tests/media/media-source/video-media-source-closed-on-htmlmediaelement-destruction.html [ Timeout ]
     281webkit.org/b/99065 http/tests/media/media-source/video-media-source-duration-boundaryconditions.html [ Timeout ]
     282webkit.org/b/99065 http/tests/media/media-source/video-media-source-duration-changed.html [ Timeout ]
     283webkit.org/b/99065 http/tests/media/media-source/video-media-source-duration.html [ Timeout ]
     284webkit.org/b/99065 http/tests/media/media-source/video-media-source-errors.html [ Timeout ]
     285webkit.org/b/99065 http/tests/media/media-source/video-media-source-event-attributes.html [ Timeout ]
     286webkit.org/b/99065 http/tests/media/media-source/video-media-source-garbage-collection-before-sourceopen.html [ Timeout ]
     287webkit.org/b/99065 http/tests/media/media-source/video-media-source-is-type-supported.html [ Timeout ]
     288webkit.org/b/99065 http/tests/media/media-source/video-media-source-objects.html [ Timeout ]
     289webkit.org/b/99065 http/tests/media/media-source/video-media-source-play.html [ Timeout ]
     290webkit.org/b/99065 http/tests/media/media-source/video-media-source-reject-append-after-reopening.html [ Timeout ]
     291webkit.org/b/99065 http/tests/media/media-source/video-media-source-seek.html [ Timeout ]
     292webkit.org/b/99065 http/tests/media/media-source/video-media-source-sourcebufferlist-crash.html [ Timeout ]
     293webkit.org/b/99065 http/tests/media/media-source/video-media-source-state-changes.html [ Timeout ]
     294webkit.org/b/99065 http/tests/media/media-source/video-media-source-zero-byte-append-in-ended-state.html [ Timeout ]
     295
     296webkit.org/b/99065  media/media-source/media-source-track-enabled.html [ Failure ]
     297webkit.org/b/99065  media/media-source/media-source-tracks.html [ Failure ]
     298webkit.org/b/99065  media/media-source/media-source-append-failed.html [ Timeout ]
     299webkit.org/b/99065  media/media-source/media-source-canplaythrough.html [ Timeout ]
     300webkit.org/b/99065  media/media-source/media-source-play.html [ Timeout ]
    254301
    255302# Encrypted Media Extensions are not enabled.

  • trunk/Source/WebCore/ChangeLog

    r159326 r159335  
     12013-11-15  Stephane Jadaud  <sjadaud@sii.fr>
     2
     3        [GStreamer] Add support for Media Source API
     4        https://bugs.webkit.org/show_bug.cgi?id=99065
     5
     6        The patch integrate a Media Source player for the GStreamer backend. The implementented architecture is:
     7        - MediaPlayerPrivateGStreamer engine is modified to support Media Source URIs (change blob:// to mediasourceblob://), in addition to the existing support for web (http/https/blob) URIs
     8        - Similar to the existing WebKitWebSrc gstreamer element that handles web URIs, a new gstreamer element named WebKitMediaSrc is implemented to handle Media Source URIs
     9        - WebKitMediaSrc registers its URI protocol handler, allowing uridecodebin to dynamically create the appropriate source element.
     10        - The WebKitMediaSrc element creates a bin with 2 appsrc: One for Audio and One for Video. Pads are dynamically linked at the reception of first video and audio buffers.
     11
     12        Reviewed by Philippe Normand.
     13
     14        Tests: Activate http/tests/media/media-source and media/media-source tests
     15
     16        * GNUmakefile.am:
     17        * GNUmakefile.list.am:
     18        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
     19        (WebCore::initializeGStreamerAndRegisterWebKitElements):
     20        (WebCore::MediaPlayerPrivateGStreamer::load):
     21        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
     22        * platform/graphics/gstreamer/MediaSourceGStreamer.cpp: Added.
     23        (WebCore::MediaSourceGStreamer::open):
     24        (WebCore::MediaSourceGStreamer::MediaSourceGStreamer):
     25        (WebCore::MediaSourceGStreamer::~MediaSourceGStreamer):
     26        (WebCore::MediaSourceGStreamer::addSourceBuffer):
     27        (WebCore::MediaSourceGStreamer::setDuration):
     28        (WebCore::MediaSourceGStreamer::markEndOfStream):
     29        (WebCore::MediaSourceGStreamer::unmarkEndOfStream):
     30        * platform/graphics/gstreamer/MediaSourceGStreamer.h: Added.
     31        * platform/graphics/gstreamer/SourceBufferPrivateGStreamer.cpp: Added.
     32        (WebCore::SourceBufferPrivateGStreamer::SourceBufferPrivateGStreamer):
     33        (WebCore::SourceBufferPrivateGStreamer::append):
     34        (WebCore::SourceBufferPrivateGStreamer::abort):
     35        (WebCore::SourceBufferPrivateGStreamer::removedFromMediaSource):
     36        * platform/graphics/gstreamer/SourceBufferPrivateGStreamer.h: Added.
     37        * platform/graphics/gstreamer/WebKitMediaSourceGStreamer.cpp: Added.
     38        (webKitMediaSrcAddSrc):
     39        (webkit_media_src_init):
     40        (webKitMediaSrcFinalize):
     41        (webKitMediaSrcSetProperty):
     42        (webKitMediaSrcGetProperty):
     43        (webKitMediaVideoSrcStop):
     44        (webKitMediaAudioSrcStop):
     45        (webKitMediaVideoSrcStart):
     46        (webKitMediaAudioSrcStart):
     47        (webKitMediaSrcChangeState):
     48        (webKitMediaSrcQueryWithParent):
     49        (webKitMediaSrcUriGetType):
     50        (webKitMediaSrcGetProtocols):
     51        (webKitMediaSrcGetUri):
     52        (webKitMediaSrcSetUri):
     53        (webKitMediaSrcUriHandlerInit):
     54        (webKitMediaVideoSrcNeedDataMainCb):
     55        (webKitMediaAudioSrcNeedDataMainCb):
     56        (webKitMediaVideoSrcNeedDataCb):
     57        (webKitMediaAudioSrcNeedDataCb):
     58        (webKitMediaVideoSrcEnoughDataMainCb):
     59        (webKitMediaAudioSrcEnoughDataMainCb):
     60        (webKitMediaVideoSrcEnoughDataCb):
     61        (webKitMediaAudioSrcEnoughDataCb):
     62        (webKitMediaVideoSrcSeekMainCb):
     63        (webKitMediaAudioSrcSeekMainCb):
     64        (webKitMediaVideoSrcSeekDataCb):
     65        (webKitMediaAudioSrcSeekDataCb):
     66        (webKitMediaSrcSetMediaPlayer):
     67        (webKitMediaSrcSetPlayBin):
     68        (MediaSourceClientGstreamer::MediaSourceClientGstreamer):
     69        (MediaSourceClientGstreamer::~MediaSourceClientGstreamer):
     70        (MediaSourceClientGstreamer::didReceiveDuration):
     71        (MediaSourceClientGstreamer::didReceiveData):
     72        (MediaSourceClientGstreamer::didFinishLoading):
     73        (MediaSourceClientGstreamer::didFail):
     74        * platform/graphics/gstreamer/WebKitMediaSourceGStreamer.h: Added.
     75
    1762013-11-14  Victor Costan  <costan@gmail.com>
    277

  • trunk/Source/WebCore/GNUmakefile.am

    r159264 r159335  
    8383        -I$(srcdir)/Source/WebCore/platform/mediastream/gstreamer \
    8484        -I$(srcdir)/Source/WebCore/platform/mock \
     85        -I$(srcdir)/Source/WebCore/platform/mock/mediasource \
    8586        -I$(srcdir)/Source/WebCore/platform/network \
    8687        -I$(srcdir)/Source/WebCore/platform/network/gtk \

  • trunk/Source/WebCore/GNUmakefile.list.am

    r159268 r159335  
    34213421        Source/WebCore/html/HTMLMarqueeElement.cpp \
    34223422        Source/WebCore/html/HTMLMarqueeElement.h \
     3423        Source/WebCore/html/HTMLMediaSource.cpp \
     3424        Source/WebCore/html/HTMLMediaSource.h \
    34233425        Source/WebCore/html/HTMLMediaElement.cpp \
    34243426        Source/WebCore/html/HTMLMediaElement.h \
    … …  
    54405442        Source/WebCore/platform/mock/ScrollbarThemeMock.cpp \
    54415443        Source/WebCore/platform/mock/ScrollbarThemeMock.h \
     5444        Source/WebCore/platform/mock/mediasource/MockBox.cpp \
     5445        Source/WebCore/platform/mock/mediasource/MockBox.h \
     5446        Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp \
     5447        Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.h \
     5448        Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.cpp \
     5449        Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.h \
     5450        Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp \
     5451        Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.h \
     5452        Source/WebCore/platform/mock/mediasource/MockTracks.cpp \
     5453        Source/WebCore/platform/mock/mediasource/MockTracks.h \
    54425454        Source/WebCore/platform/network/BlobData.cpp \
    54435455        Source/WebCore/platform/network/BlobData.h \
    … …  
    63096321        Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp \
    63106322        Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.h
     6323webcore_platform_sources += \
     6324        Source/WebCore/platform/graphics/gstreamer/SourceBufferPrivateGStreamer.cpp \
     6325        Source/WebCore/platform/graphics/gstreamer/SourceBufferPrivateGStreamer.h \
     6326        Source/WebCore/platform/graphics/gstreamer/MediaSourceGStreamer.cpp \
     6327        Source/WebCore/platform/graphics/gstreamer/MediaSourceGStreamer.h \
     6328        Source/WebCore/platform/graphics/gstreamer/WebKitMediaSourceGStreamer.cpp \
     6329        Source/WebCore/platform/graphics/gstreamer/WebKitMediaSourceGStreamer.h
    63116330endif
    63126331

  • trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp

    r158734 r159335  
    5757#endif
    5858
     59#if ENABLE(MEDIA_SOURCE)
     60#include "MediaSource.h"
     61#include "WebKitMediaSourceGStreamer.h"
     62#endif
     63
    5964// GstPlayFlags flags from playbin2. It is the policy of GStreamer to
    6065// not publicly expose element-specific enums. That's why this
    … …  
    230235    if (!srcFactory) {
    231236        GST_DEBUG_CATEGORY_INIT(webkit_media_player_debug, "webkitmediaplayer", 0, "WebKit media player");
    232         return gst_element_register(0, "webkitwebsrc", GST_RANK_PRIMARY + 100, WEBKIT_TYPE_WEB_SRC);
    233     }
    234 
     237        gst_element_register(0, "webkitwebsrc", GST_RANK_PRIMARY + 100, WEBKIT_TYPE_WEB_SRC);
     238    }
     239
     240#if ENABLE(MEDIA_SOURCE)
     241    GRefPtr<GstElementFactory> WebKitMediaSrcFactory = gst_element_factory_find("webkitmediasrc");
     242    if (!WebKitMediaSrcFactory)
     243        gst_element_register(0, "webkitmediasrc", GST_RANK_PRIMARY + 100, WEBKIT_TYPE_MEDIA_SRC);
     244#endif
    235245    return true;
    236246}
    … …  
    389399
    390400#if ENABLE(MEDIA_SOURCE)
    391 void MediaPlayerPrivateGStreamer::load(const String& url, PassRefPtr<MediaSource>)
    392 {
    393     notImplemented();
     401void MediaPlayerPrivateGStreamer::load(const String& url, PassRefPtr<HTMLMediaSource> mediaSource)
     402{
     403    String mediasourceUri = String::format("mediasource%s", url.utf8().data());
     404    m_mediaSource = mediaSource;
     405    load(mediasourceUri);
    394406}
    395407#endif
    … …  
    13101322    if (WEBKIT_IS_WEB_SRC(m_source.get()))
    13111323        webKitWebSrcSetMediaPlayer(WEBKIT_WEB_SRC(m_source.get()), m_player);
     1324#if ENABLE(MEDIA_SOURCE)
     1325    if (m_mediaSource && WEBKIT_IS_MEDIA_SRC(m_source.get())) {
     1326        MediaSourceGStreamer::open(m_mediaSource.get(), WEBKIT_MEDIA_SRC(m_source.get()));
     1327        webKitMediaSrcSetPlayBin(WEBKIT_MEDIA_SRC(m_source.get()), m_playBin.get());
     1328    }
     1329#endif
    13121330}
    13131331

  • trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h

    r158436 r159335  
    3434#include <wtf/Forward.h>
    3535
     36#if ENABLE(MEDIA_SOURCE)
     37#include "MediaSourceGStreamer.h"
     38#endif
     39
    3640typedef struct _GstBuffer GstBuffer;
    3741typedef struct _GstMessage GstMessage;
    … …  
    5660    void load(const String &url);
    5761#if ENABLE(MEDIA_SOURCE)
    58     void load(const String& url, PassRefPtr<MediaSource>);
     62    void load(const String& url, PassRefPtr<HTMLMediaSource>);
    5963#endif
    6064    void commitLoad();
    … …  
    201205    RefPtr<InbandTextTrackPrivate> m_chaptersTrack;
    202206#endif
     207#if ENABLE(MEDIA_SOURCE)
     208    RefPtr<HTMLMediaSource> m_mediaSource;
     209#endif
    203210};
    204211}

  • trunk/Tools/ChangeLog

    r159330 r159335  
     12013-11-15  Stephane Jadaud  <sjadaud@sii.fr>
     2
     3        [GStreamer] Add support for Media Source API
     4        https://bugs.webkit.org/show_bug.cgi?id=99065
     5
     6        Reviewed by Philippe Normand.
     7
     8        * Scripts/webkitperl/FeatureList.pm:
     9
    1102013-11-15  Xabier Rodriguez Calvar  <calvaris@igalia.com>
    211

  • trunk/Tools/Scripts/webkitperl/FeatureList.pm

    r159321 r159335  
    342342
    343343    { option => "media-source", desc => "Toggle Media Source support",
    344       define => "ENABLE_MEDIA_SOURCE", default => 0, value => \$mediaSourceSupport },
     344      define => "ENABLE_MEDIA_SOURCE", default => isGtk(), value => \$mediaSourceSupport },
    345345
    346346    { option => "media-statistics", desc => "Toggle Media Statistics support",
Note: See TracChangeset for help on using the changeset viewer.