Changeset 228617 in webkit


Ignore:
Timestamp:
Feb 19, 2018 12:58:14 AM (6 years ago)
Author:
Philippe Normand
Message:

[GStreamer] Playbin3 support
https://bugs.webkit.org/show_bug.cgi?id=182530

Reviewed by Xabier Rodriguez-Calvar.

.:

  • Source/cmake/GStreamerDefinitions.cmake: New

USE(GSTREAMER_PLAYBIN3) feature. This should be enabled only for
very recent versions of GStreamer (1.14 at least) and is optional, for now.

Source/WebCore:

This patch introduces opt-in support for the playbin3 GStreamer
element. This new playback element is still considered
experimental but it can still be very useful for media assets
containing multiple tracks of the same type. In such scenario
audio/video decoders would be created only for the selected tracks
whereas playbin2 creates decoder for each track.

  • platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp: Take

a weak pointer of the media player instead of playbin and call the
player when enabling tracks. Also use the GstStream API instead of
inspecting the pad when playbin3 is used.
(WebCore::AudioTrackPrivateGStreamer::AudioTrackPrivateGStreamer):
(WebCore::AudioTrackPrivateGStreamer::disconnect):
(WebCore::AudioTrackPrivateGStreamer::markAsActive):
(WebCore::AudioTrackPrivateGStreamer::setEnabled):

  • platform/graphics/gstreamer/AudioTrackPrivateGStreamer.h:
  • platform/graphics/gstreamer/GRefPtrGStreamer.cpp: GstStream and

GstStreamCollection smart pointer support.
(WTF::adoptGRef):
(WTF::refGPtr<GstStream>):
(WTF::derefGPtr<GstStream>):
(WTF::refGPtr<GstStreamCollection>):
(WTF::derefGPtr<GstStreamCollection>):

  • platform/graphics/gstreamer/GRefPtrGStreamer.h:
  • platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp:

Use the GstStream API instead of inspecting the pad when playbin3
is used.
(WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer):
(WebCore::InbandTextTrackPrivateGStreamer::disconnect):

  • platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h:

(WebCore::InbandTextTrackPrivateGStreamer::create):

  • platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:

(WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer):
m_{audio,video,text}Tracks are now hashmaps, which is a bit more
convenient to use than Vectors.
(WebCore::MediaPlayerPrivateGStreamer::updateTracks): Update our
tracks implementations depending on the streams stored in the
collection.
(WebCore::MediaPlayerPrivateGStreamer::enableTrack): Activate the
given track. This method is called by the TrackPrivate
implementations.
(WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfVideo):
Refactored legacy (playbin2) support for track hashmap storage.
(WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfAudio): Ditto.
(WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Ditto.
(WebCore::MediaPlayerPrivateGStreamer::newTextSample): Use track hashmap storage.
(WebCore::MediaPlayerPrivateGStreamer::handleMessage): React on
playbin3 GstStream messages, store the collection provided and
activate the given selected streams accordingly.
(WebCore::MediaPlayerPrivateGStreamer::purgeInvalidAudioTracks): Invalidate tracks no longer needed.
(WebCore::MediaPlayerPrivateGStreamer::purgeInvalidVideoTracks): Ditto.
(WebCore::MediaPlayerPrivateGStreamer::purgeInvalidTextTracks): Ditto.
(WebCore::MediaPlayerPrivateGStreamer::sourceSetupCallback):
Refactoring, use source-setup signal instead of the source
property which is not supported in playbin3.
(WebCore::MediaPlayerPrivateGStreamer::uriDecodeBinElementAddedCallback):
Promoted logging messages, GST_TRACE should be used for very
frequent and less interesting messages.
(WebCore::MediaPlayerPrivateGStreamer::downloadBufferFileCreatedCallback): Ditto.
(WebCore::MediaPlayerPrivateGStreamer::sourceSetup): Called by the source-setup signal handler.
(WebCore::MediaPlayerPrivateGStreamer::setDownloadBuffering): Debugging message added.
(WebCore::MediaPlayerPrivateGStreamer::setPreload): Ditto.
(WebCore::MediaPlayerPrivateGStreamer::createGSTPlayBin): Create a
playbin3 element if USE(GSTREAMER_PLAYBIN3) was enabled and
connect to playbin2 signals otherwise.

  • platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:

(WebCore::MediaPlayerPrivateGStreamer::createWeakPtr): Promoted to public.

  • platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h:

New StreamCollectionChanged notification type, used the sub-class.

  • platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp:

(WebCore::TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer): Support for GstStream API.
(WebCore::TrackPrivateBaseGStreamer::disconnect): Clear GstStream too.
(WebCore::TrackPrivateBaseGStreamer::tagsChanged): Get tags from GstStream.
(WebCore::TrackPrivateBaseGStreamer::notifyTrackOfActiveChanged): No need to check m_pad twice.
(WebCore::TrackPrivateBaseGStreamer::notifyTrackOfTagsChanged): Ditto.

  • platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h:
  • platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp: Take

a weak pointer of the media player instead of playbin and call the
player when enabling tracks. Also use the GstStream API instead of
inspecting the pad when playbin3 is used.
(WebCore::VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer):
(WebCore::VideoTrackPrivateGStreamer::disconnect):
(WebCore::VideoTrackPrivateGStreamer::markAsActive):
(WebCore::VideoTrackPrivateGStreamer::setSelected):

  • platform/graphics/gstreamer/VideoTrackPrivateGStreamer.h:
  • platform/graphics/gstreamer/mse/AppendPipeline.cpp: Pass player pointer to track constructor.

(WebCore::AppendPipeline::connectDemuxerSrcPadToAppsink):

  • platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp: Source-setup signal handler.

(WebCore::MediaPlayerPrivateGStreamerMSE::sourceSetup):

  • platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h:
Location:
trunk
Files:
19 edited

Legend:

Unmodified
Added
Removed
  • trunk/ChangeLog

    r228608 r228617  
     12018-02-19  Philippe Normand  <pnormand@igalia.com>
     2
     3        [GStreamer] Playbin3 support
     4        https://bugs.webkit.org/show_bug.cgi?id=182530
     5
     6        Reviewed by Xabier Rodriguez-Calvar.
     7
     8        * Source/cmake/GStreamerDefinitions.cmake: New
     9        USE(GSTREAMER_PLAYBIN3) feature. This should be enabled only for
     10        very recent versions of GStreamer (1.14 at least) and is optional, for now.
     11
    1122018-02-18  Philippe Normand  <pnormand@igalia.com>
    213
  • trunk/Source/WebCore/ChangeLog

    r228609 r228617  
     12018-02-19  Philippe Normand  <pnormand@igalia.com>
     2
     3        [GStreamer] Playbin3 support
     4        https://bugs.webkit.org/show_bug.cgi?id=182530
     5
     6        Reviewed by Xabier Rodriguez-Calvar.
     7
     8        This patch introduces opt-in support for the playbin3 GStreamer
     9        element. This new playback element is still considered
     10        experimental but it can still be very useful for media assets
     11        containing multiple tracks of the same type. In such scenario
     12        audio/video decoders would be created only for the selected tracks
     13        whereas playbin2 creates decoder for each track.
     14
     15        * platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp: Take
     16        a weak pointer of the media player instead of playbin and call the
     17        player when enabling tracks. Also use the GstStream API instead of
     18        inspecting the pad when playbin3 is used.
     19        (WebCore::AudioTrackPrivateGStreamer::AudioTrackPrivateGStreamer):
     20        (WebCore::AudioTrackPrivateGStreamer::disconnect):
     21        (WebCore::AudioTrackPrivateGStreamer::markAsActive):
     22        (WebCore::AudioTrackPrivateGStreamer::setEnabled):
     23        * platform/graphics/gstreamer/AudioTrackPrivateGStreamer.h:
     24        * platform/graphics/gstreamer/GRefPtrGStreamer.cpp: GstStream and
     25        GstStreamCollection smart pointer support.
     26        (WTF::adoptGRef):
     27        (WTF::refGPtr<GstStream>):
     28        (WTF::derefGPtr<GstStream>):
     29        (WTF::refGPtr<GstStreamCollection>):
     30        (WTF::derefGPtr<GstStreamCollection>):
     31        * platform/graphics/gstreamer/GRefPtrGStreamer.h:
     32        * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp:
     33        Use the GstStream API instead of inspecting the pad when playbin3
     34        is used.
     35        (WebCore::InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer):
     36        (WebCore::InbandTextTrackPrivateGStreamer::disconnect):
     37        * platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h:
     38        (WebCore::InbandTextTrackPrivateGStreamer::create):
     39        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
     40        (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer):
     41        m_{audio,video,text}Tracks are now hashmaps, which is a bit more
     42        convenient to use than Vectors.
     43        (WebCore::MediaPlayerPrivateGStreamer::updateTracks): Update our
     44        tracks implementations depending on the streams stored in the
     45        collection.
     46        (WebCore::MediaPlayerPrivateGStreamer::enableTrack): Activate the
     47        given track. This method is called by the TrackPrivate
     48        implementations.
     49        (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfVideo):
     50        Refactored legacy (playbin2) support for track hashmap storage.
     51        (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfAudio): Ditto.
     52        (WebCore::MediaPlayerPrivateGStreamer::notifyPlayerOfText): Ditto.
     53        (WebCore::MediaPlayerPrivateGStreamer::newTextSample): Use track hashmap storage.
     54        (WebCore::MediaPlayerPrivateGStreamer::handleMessage): React on
     55        playbin3 GstStream messages, store the collection provided and
     56        activate the given selected streams accordingly.
     57        (WebCore::MediaPlayerPrivateGStreamer::purgeInvalidAudioTracks): Invalidate tracks no longer needed.
     58        (WebCore::MediaPlayerPrivateGStreamer::purgeInvalidVideoTracks): Ditto.
     59        (WebCore::MediaPlayerPrivateGStreamer::purgeInvalidTextTracks): Ditto.
     60        (WebCore::MediaPlayerPrivateGStreamer::sourceSetupCallback):
     61        Refactoring, use source-setup signal instead of the source
     62        property which is not supported in playbin3.
     63        (WebCore::MediaPlayerPrivateGStreamer::uriDecodeBinElementAddedCallback):
     64        Promoted logging messages, GST_TRACE should be used for very
     65        frequent and less interesting messages.
     66        (WebCore::MediaPlayerPrivateGStreamer::downloadBufferFileCreatedCallback): Ditto.
     67        (WebCore::MediaPlayerPrivateGStreamer::sourceSetup): Called by the source-setup signal handler.
     68        (WebCore::MediaPlayerPrivateGStreamer::setDownloadBuffering): Debugging message added.
     69        (WebCore::MediaPlayerPrivateGStreamer::setPreload): Ditto.
     70        (WebCore::MediaPlayerPrivateGStreamer::createGSTPlayBin): Create a
     71        playbin3 element if USE(GSTREAMER_PLAYBIN3) was enabled and
     72        connect to playbin2 signals otherwise.
     73        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
     74        (WebCore::MediaPlayerPrivateGStreamer::createWeakPtr): Promoted to public.
     75        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h:
     76        New StreamCollectionChanged notification type, used the sub-class.
     77        * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp:
     78        (WebCore::TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer): Support for GstStream API.
     79        (WebCore::TrackPrivateBaseGStreamer::disconnect): Clear GstStream too.
     80        (WebCore::TrackPrivateBaseGStreamer::tagsChanged): Get tags from GstStream.
     81        (WebCore::TrackPrivateBaseGStreamer::notifyTrackOfActiveChanged): No need to check m_pad twice.
     82        (WebCore::TrackPrivateBaseGStreamer::notifyTrackOfTagsChanged): Ditto.
     83        * platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h:
     84        * platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp: Take
     85        a weak pointer of the media player instead of playbin and call the
     86        player when enabling tracks. Also use the GstStream API instead of
     87        inspecting the pad when playbin3 is used.
     88        (WebCore::VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer):
     89        (WebCore::VideoTrackPrivateGStreamer::disconnect):
     90        (WebCore::VideoTrackPrivateGStreamer::markAsActive):
     91        (WebCore::VideoTrackPrivateGStreamer::setSelected):
     92        * platform/graphics/gstreamer/VideoTrackPrivateGStreamer.h:
     93        * platform/graphics/gstreamer/mse/AppendPipeline.cpp: Pass player pointer to track constructor.
     94        (WebCore::AppendPipeline::connectDemuxerSrcPadToAppsink):
     95        * platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp: Source-setup signal handler.
     96        (WebCore::MediaPlayerPrivateGStreamerMSE::sourceSetup):
     97        * platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h:
     98
    1992018-02-19  Fujii Hironori  <Hironori.Fujii@sony.com>
    2100
  • trunk/Source/WebCore/platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp

    r213445 r228617  
    3030#include "AudioTrackPrivateGStreamer.h"
    3131
     32#include "MediaPlayerPrivateGStreamer.h"
    3233#include <glib-object.h>
    3334
    3435namespace WebCore {
    3536
    36 AudioTrackPrivateGStreamer::AudioTrackPrivateGStreamer(GRefPtr<GstElement> playbin, gint index, GRefPtr<GstPad> pad)
     37AudioTrackPrivateGStreamer::AudioTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstPad> pad)
    3738    : TrackPrivateBaseGStreamer(this, index, pad)
    38     , m_playbin(playbin)
     39    , m_player(player)
    3940{
    4041    // FIXME: Get a real ID from the tkhd atom.
     
    4344}
    4445
     46#if USE(GSTREAMER_PLAYBIN3)
     47AudioTrackPrivateGStreamer::AudioTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstStream> stream)
     48    : TrackPrivateBaseGStreamer(this, index, stream)
     49    , m_player(player)
     50{
     51    m_id = gst_stream_get_stream_id(stream.get());
     52    notifyTrackOfActiveChanged();
     53}
     54#endif
     55
    4556void AudioTrackPrivateGStreamer::disconnect()
    4657{
    47     m_playbin.clear();
     58    m_player = nullptr;
    4859    TrackPrivateBaseGStreamer::disconnect();
     60}
     61
     62void AudioTrackPrivateGStreamer::markAsActive()
     63{
     64    AudioTrackPrivate::setEnabled(true);
    4965}
    5066
     
    5571    AudioTrackPrivate::setEnabled(enabled);
    5672
    57     if (enabled && m_playbin)
    58         g_object_set(m_playbin.get(), "current-audio", m_index, nullptr);
     73    if (enabled && m_player)
     74        m_player->enableTrack(TrackPrivateBaseGStreamer::TrackType::Audio, m_index);
    5975}
    6076
  • trunk/Source/WebCore/platform/graphics/gstreamer/AudioTrackPrivateGStreamer.h

    r216702 r228617  
    2424 */
    2525
    26 #ifndef AudioTrackPrivateGStreamer_h
    27 #define AudioTrackPrivateGStreamer_h
     26#pragma once
    2827
    2928#if ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
     
    3231#include "GRefPtrGStreamer.h"
    3332#include "TrackPrivateBaseGStreamer.h"
     33#include <gst/gst.h>
     34#include <wtf/WeakPtr.h>
    3435
    3536namespace WebCore {
     37class MediaPlayerPrivateGStreamer;
    3638
    3739class AudioTrackPrivateGStreamer final : public AudioTrackPrivate, public TrackPrivateBaseGStreamer {
    3840public:
    39     static RefPtr<AudioTrackPrivateGStreamer> create(GRefPtr<GstElement> playbin, gint index, GRefPtr<GstPad> pad)
     41    static RefPtr<AudioTrackPrivateGStreamer> create(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstPad> pad)
    4042    {
    41         return adoptRef(*new AudioTrackPrivateGStreamer(playbin, index, pad));
     43        return adoptRef(*new AudioTrackPrivateGStreamer(player, index, pad));
    4244    }
     45
     46#if USE(GSTREAMER_PLAYBIN3)
     47    static RefPtr<AudioTrackPrivateGStreamer> create(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstStream> stream)
     48    {
     49        return adoptRef(*new AudioTrackPrivateGStreamer(player, index, stream));
     50    }
     51#endif
    4352
    4453    void disconnect() override;
    4554
    4655    void setEnabled(bool) override;
     56    void markAsActive();
    4757    void setActive(bool enabled) override { setEnabled(enabled); }
    4858
     
    5464
    5565private:
    56     AudioTrackPrivateGStreamer(GRefPtr<GstElement> playbin, gint index, GRefPtr<GstPad>);
     66    AudioTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer>, gint index, GRefPtr<GstPad>);
     67#if USE(GSTREAMER_PLAYBIN3)
     68    AudioTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer>, gint index, GRefPtr<GstStream>);
     69#endif
    5770
    5871    AtomicString m_id;
    59     GRefPtr<GstElement> m_playbin;
     72    WeakPtr<MediaPlayerPrivateGStreamer> m_player;
    6073};
    6174
     
    6376
    6477#endif // ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
    65 
    66 #endif // AudioTrackPrivateGStreamer_h
  • trunk/Source/WebCore/platform/graphics/gstreamer/GRefPtrGStreamer.cpp

    r221781 r228617  
    354354}
    355355
     356#if USE(GSTREAMER_PLAYBIN3)
     357template <> GRefPtr<GstStream> adoptGRef(GstStream* ptr)
     358{
     359    ASSERT(!ptr || !g_object_is_floating(ptr));
     360    return GRefPtr<GstStream>(ptr, GRefPtrAdopt);
     361}
     362
     363template <> GstStream* refGPtr<GstStream>(GstStream* ptr)
     364{
     365    if (ptr)
     366        gst_object_ref_sink(GST_OBJECT_CAST(ptr));
     367
     368    return ptr;
     369}
     370
     371template <> void derefGPtr<GstStream>(GstStream* ptr)
     372{
     373    if (ptr)
     374        gst_object_unref(ptr);
     375}
     376
     377template <> GRefPtr<GstStreamCollection> adoptGRef(GstStreamCollection* ptr)
     378{
     379    ASSERT(!ptr || !g_object_is_floating(ptr));
     380    return GRefPtr<GstStreamCollection>(ptr, GRefPtrAdopt);
     381}
     382
     383template <> GstStreamCollection* refGPtr<GstStreamCollection>(GstStreamCollection* ptr)
     384{
     385    if (ptr)
     386        gst_object_ref_sink(GST_OBJECT_CAST(ptr));
     387
     388    return ptr;
     389}
     390
     391template <> void derefGPtr<GstStreamCollection>(GstStreamCollection* ptr)
     392{
     393    if (ptr)
     394        gst_object_unref(ptr);
     395}
     396#endif
     397
    356398template <> GRefPtr<WebKitVideoSink> adoptGRef(WebKitVideoSink* ptr)
    357399{
  • trunk/Source/WebCore/platform/graphics/gstreamer/GRefPtrGStreamer.h

    r221781 r228617  
    1818 */
    1919
    20 #ifndef GRefPtrGStreamer_h
    21 #define GRefPtrGStreamer_h
     20#pragma once
     21
    2222#if USE(GSTREAMER)
    2323
     
    4343typedef struct _WebKitVideoSink WebKitVideoSink;
    4444typedef struct _WebKitWebSrc WebKitWebSrc;
     45
     46#if USE(GSTREAMER_PLAYBIN3)
     47typedef struct _GstStream GstStream;
     48typedef struct _GstStreamCollection GstStreamCollection;
     49#endif
    4550
    4651#if USE(GSTREAMER_GL)
     
    128133template<> void derefGPtr<WebKitWebSrc>(WebKitWebSrc* ptr);
    129134
     135#if USE(GSTREAMER_PLAYBIN3)
     136template<> GRefPtr<GstStream> adoptGRef(GstStream*);
     137template<> GstStream* refGPtr<GstStream>(GstStream*);
     138template<> void derefGPtr<GstStream>(GstStream*);
     139
     140template<> GRefPtr<GstStreamCollection> adoptGRef(GstStreamCollection*);
     141template<> GstStreamCollection* refGPtr<GstStreamCollection>(GstStreamCollection*);
     142template<> void derefGPtr<GstStreamCollection>(GstStreamCollection*);
     143#endif
     144
    130145#if USE(GSTREAMER_GL)
    131146template<> GRefPtr<GstGLDisplay> adoptGRef(GstGLDisplay* ptr);
     
    141156
    142157#endif // USE(GSTREAMER)
    143 
    144 #endif // GRefPtrGStreamer_h
  • trunk/Source/WebCore/platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp

    r218471 r228617  
    4141
    4242InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer(gint index, GRefPtr<GstPad> pad)
    43     : InbandTextTrackPrivate(WebVTT), TrackPrivateBaseGStreamer(this, index, pad)
     43    : InbandTextTrackPrivate(WebVTT)
     44    , TrackPrivateBaseGStreamer(this, index, pad)
    4445{
    4546    m_eventProbe = gst_pad_add_probe(m_pad.get(), GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM, [] (GstPad*, GstPadProbeInfo* info, gpointer userData) -> GstPadProbeReturn {
     
    5859}
    5960
     61#if USE(GSTREAMER_PLAYBIN3)
     62InbandTextTrackPrivateGStreamer::InbandTextTrackPrivateGStreamer(gint index, GRefPtr<GstStream> stream)
     63    : InbandTextTrackPrivate(WebVTT)
     64    , TrackPrivateBaseGStreamer(this, index, stream)
     65{
     66    m_streamId = gst_stream_get_stream_id(stream.get());
     67    GST_INFO("Track %d got stream start for stream %s.", m_index, m_streamId.utf8().data());
     68}
     69#endif
     70
    6071void InbandTextTrackPrivateGStreamer::disconnect()
    6172{
    62     if (!m_pad)
    63         return;
    64 
    65     gst_pad_remove_probe(m_pad.get(), m_eventProbe);
     73    if (m_pad)
     74        gst_pad_remove_probe(m_pad.get(), m_eventProbe);
    6675
    6776    TrackPrivateBaseGStreamer::disconnect();
  • trunk/Source/WebCore/platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.h

    r216702 r228617  
    2424 */
    2525
    26 #ifndef InbandTextTrackPrivateGStreamer_h
    27 #define InbandTextTrackPrivateGStreamer_h
     26#pragma once
    2827
    2928#if ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
     
    3231#include "InbandTextTrackPrivate.h"
    3332#include "TrackPrivateBaseGStreamer.h"
     33#include <gst/gst.h>
    3434#include <wtf/Lock.h>
    3535
     
    3737
    3838class MediaPlayerPrivateGStreamer;
    39 typedef struct _GstSample GstSample;
    4039
    4140class InbandTextTrackPrivateGStreamer : public InbandTextTrackPrivate, public TrackPrivateBaseGStreamer {
     
    4544        return adoptRef(*new InbandTextTrackPrivateGStreamer(index, pad));
    4645    }
     46
     47#if USE(GSTREAMER_PLAYBIN3)
     48    static Ref<InbandTextTrackPrivateGStreamer> create(gint index, GRefPtr<GstStream> stream)
     49    {
     50        return adoptRef(*new InbandTextTrackPrivateGStreamer(index, stream));
     51    }
     52#endif
    4753
    4854    void disconnect() override;
     
    5864private:
    5965    InbandTextTrackPrivateGStreamer(gint index, GRefPtr<GstPad>);
     66#if USE(GSTREAMER_PLAYBIN3)
     67    InbandTextTrackPrivateGStreamer(gint index, GRefPtr<GstStream>);
     68#endif
    6069
    6170    void streamChanged();
     
    6574
    6675    gulong m_eventProbe;
    67     Vector<GRefPtr<GstSample> > m_pendingSamples;
     76    Vector<GRefPtr<GstSample>> m_pendingSamples;
    6877    String m_streamId;
    6978    Lock m_sampleMutex;
     
    7382
    7483#endif // ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
    75 
    76 #endif // InbandTextTrackPrivateGStreamer_h
  • trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp

    r228508 r228617  
    167167{
    168168#if ENABLE(VIDEO_TRACK)
    169     for (size_t i = 0; i < m_audioTracks.size(); ++i)
    170         m_audioTracks[i]->disconnect();
    171 
    172     for (size_t i = 0; i < m_textTracks.size(); ++i)
    173         m_textTracks[i]->disconnect();
    174 
    175     for (size_t i = 0; i < m_videoTracks.size(); ++i)
    176         m_videoTracks[i]->disconnect();
     169    for (auto& track : m_audioTracks.values())
     170        track->disconnect();
     171
     172    for (auto& track : m_textTracks.values())
     173        track->disconnect();
     174
     175    for (auto& track : m_videoTracks.values())
     176        track->disconnect();
    177177#endif
    178178    if (m_fillTimer.isActive())
     
    613613}
    614614
     615#if USE(GSTREAMER_PLAYBIN3)
     616void MediaPlayerPrivateGStreamer::updateTracks()
     617{
     618    ASSERT(!m_isLegacyPlaybin);
     619
     620    bool useMediaSource = isMediaSource();
     621    unsigned length = gst_stream_collection_get_size(m_streamCollection.get());
     622    Vector<String> validAudioStreams;
     623    Vector<String> validVideoStreams;
     624    Vector<String> validTextStreams;
     625    for (unsigned i = 0; i < length; i++) {
     626        GRefPtr<GstStream> stream = gst_stream_collection_get_stream(m_streamCollection.get(), i);
     627        String streamId(gst_stream_get_stream_id(stream.get()));
     628        GstStreamType type = gst_stream_get_stream_type(stream.get());
     629        GST_DEBUG("Inspecting %s track with ID %s", gst_stream_type_get_name(type), streamId.utf8().data());
     630        if (type & GST_STREAM_TYPE_AUDIO) {
     631            validAudioStreams.append(streamId);
     632#if ENABLE(VIDEO_TRACK)
     633            if (!useMediaSource) {
     634                unsigned localIndex = i - validVideoStreams.size() - validTextStreams.size();
     635                if (localIndex < m_audioTracks.size()) {
     636                    if (m_audioTracks.contains(streamId))
     637                        continue;
     638                }
     639
     640                RefPtr<AudioTrackPrivateGStreamer> track = AudioTrackPrivateGStreamer::create(createWeakPtr(), i, stream);
     641                m_audioTracks.add(track->id(), track);
     642                m_player->addAudioTrack(*track);
     643            }
     644#endif
     645        } else if (type & GST_STREAM_TYPE_VIDEO) {
     646            validVideoStreams.append(streamId);
     647#if ENABLE(VIDEO_TRACK)
     648            if (!useMediaSource) {
     649                unsigned localIndex = i - validAudioStreams.size() - validTextStreams.size();
     650                if (localIndex < m_videoTracks.size()) {
     651                    if (m_videoTracks.contains(streamId))
     652                        continue;
     653                }
     654
     655                RefPtr<VideoTrackPrivateGStreamer> track = VideoTrackPrivateGStreamer::create(createWeakPtr(), i, stream);
     656                m_videoTracks.add(track->id(), track);
     657                m_player->addVideoTrack(*track);
     658            }
     659#endif
     660        } else if (type & GST_STREAM_TYPE_TEXT) {
     661            validTextStreams.append(streamId);
     662#if ENABLE(VIDEO_TRACK)
     663            if (!useMediaSource) {
     664                unsigned localIndex = i - validVideoStreams.size() - validAudioStreams.size();
     665                if (localIndex < m_textTracks.size()) {
     666                    if (m_textTracks.contains(streamId))
     667                        continue;
     668                }
     669
     670                RefPtr<InbandTextTrackPrivateGStreamer> track = InbandTextTrackPrivateGStreamer::create(localIndex, stream);
     671                m_textTracks.add(streamId, track);
     672                m_player->addTextTrack(*track);
     673            }
     674#endif
     675        } else
     676            GST_WARNING("Unknown track type found for stream %s", streamId.utf8().data());
     677    }
     678
     679    GST_INFO("Media has %u video tracks, %u audio tracks and %u text tracks", validVideoStreams.size(), validAudioStreams.size(), validTextStreams.size());
     680
     681    bool oldHasAudio = m_hasAudio;
     682    bool oldHasVideo = m_hasVideo;
     683    m_hasAudio = !validAudioStreams.isEmpty();
     684    m_hasVideo = !validVideoStreams.isEmpty();
     685    if ((oldHasVideo != m_hasVideo) || (oldHasAudio != m_hasAudio))
     686        m_player->characteristicChanged();
     687
     688    if (m_hasVideo)
     689        m_player->sizeChanged();
     690
     691    if (useMediaSource) {
     692        GST_DEBUG("Tracks managed by source element. Bailing out now.");
     693        m_player->client().mediaPlayerEngineUpdated(m_player);
     694        return;
     695    }
     696
     697#if ENABLE(VIDEO_TRACK)
     698    purgeInvalidAudioTracks(validAudioStreams);
     699    purgeInvalidVideoTracks(validVideoStreams);
     700    purgeInvalidTextTracks(validTextStreams);
     701#endif
     702
     703    m_player->client().mediaPlayerEngineUpdated(m_player);
     704}
     705#endif
     706
     707void MediaPlayerPrivateGStreamer::enableTrack(TrackPrivateBaseGStreamer::TrackType trackType, unsigned index)
     708{
     709    const char* propertyName;
     710    const char* trackTypeAsString;
     711    GList* selectedStreams = nullptr;
     712
     713    switch (trackType) {
     714    case TrackPrivateBaseGStreamer::TrackType::Audio:
     715        propertyName = "current-audio";
     716        trackTypeAsString = "audio";
     717        if (!m_currentTextStreamId.isEmpty())
     718            selectedStreams = g_list_append(selectedStreams, g_strdup(m_currentTextStreamId.utf8().data()));
     719        if (!m_currentVideoStreamId.isEmpty())
     720            selectedStreams = g_list_append(selectedStreams, g_strdup(m_currentVideoStreamId.utf8().data()));
     721        break;
     722    case TrackPrivateBaseGStreamer::TrackType::Video:
     723        propertyName = "current-video";
     724        trackTypeAsString = "video";
     725        if (!m_currentAudioStreamId.isEmpty())
     726            selectedStreams = g_list_append(selectedStreams, g_strdup(m_currentAudioStreamId.utf8().data()));
     727        if (!m_currentTextStreamId.isEmpty())
     728            selectedStreams = g_list_append(selectedStreams, g_strdup(m_currentTextStreamId.utf8().data()));
     729        break;
     730    case TrackPrivateBaseGStreamer::TrackType::Text:
     731        propertyName = "current-text";
     732        trackTypeAsString = "text";
     733        if (!m_currentAudioStreamId.isEmpty())
     734            selectedStreams = g_list_append(selectedStreams, g_strdup(m_currentAudioStreamId.utf8().data()));
     735        if (!m_currentVideoStreamId.isEmpty())
     736            selectedStreams = g_list_append(selectedStreams, g_strdup(m_currentVideoStreamId.utf8().data()));
     737        break;
     738    case TrackPrivateBaseGStreamer::TrackType::Unknown:
     739    default:
     740        ASSERT_NOT_REACHED();
     741    }
     742
     743    GST_INFO("Enabling %s track with index: %lu", trackTypeAsString, index);
     744    // FIXME: Remove isMediaSource() test below when fixing https://bugs.webkit.org/show_bug.cgi?id=182531
     745    if (m_isLegacyPlaybin || isMediaSource()) {
     746        GstElement* element = isMediaSource() ? m_source.get() : m_pipeline.get();
     747        g_object_set(element, propertyName, index, nullptr);
     748    }
     749#if USE(GSTREAMER_PLAYBIN3)
     750    else {
     751        GstStream* stream = gst_stream_collection_get_stream(m_streamCollection.get(), index);
     752        if (stream) {
     753            String streamId = gst_stream_get_stream_id(stream);
     754            selectedStreams = g_list_append(selectedStreams, g_strdup(streamId.utf8().data()));
     755        } else
     756            GST_WARNING("%s stream %lu not found", trackTypeAsString, index);
     757
     758        // TODO: MSE GstStream API support: https://bugs.webkit.org/show_bug.cgi?id=182531
     759        gst_element_send_event(m_pipeline.get(), gst_event_new_select_streams(selectedStreams));
     760    }
     761#endif
     762
     763    if (selectedStreams)
     764        g_list_free_full(selectedStreams, reinterpret_cast<GDestroyNotify>(g_free));
     765}
     766
    615767void MediaPlayerPrivateGStreamer::videoChangedCallback(MediaPlayerPrivateGStreamer* player)
    616768{
     
    622774    if (UNLIKELY(!m_pipeline || !m_source))
    623775        return;
     776
     777    ASSERT(m_isLegacyPlaybin || isMediaSource());
    624778
    625779    gint numTracks = 0;
     
    645799
    646800#if ENABLE(VIDEO_TRACK)
     801    Vector<String> validVideoStreams;
    647802    for (gint i = 0; i < numTracks; ++i) {
    648803        GRefPtr<GstPad> pad;
     
    650805        ASSERT(pad);
    651806
     807        String streamId = "V" + String::number(i);
     808        validVideoStreams.append(streamId);
    652809        if (i < static_cast<gint>(m_videoTracks.size())) {
    653             RefPtr<VideoTrackPrivateGStreamer> existingTrack = m_videoTracks[i];
    654             existingTrack->setIndex(i);
    655             if (existingTrack->pad() == pad)
    656                 continue;
    657         }
    658 
    659         RefPtr<VideoTrackPrivateGStreamer> track = VideoTrackPrivateGStreamer::create(m_pipeline, i, pad);
    660         m_videoTracks.append(track);
     810            RefPtr<VideoTrackPrivateGStreamer> existingTrack = m_videoTracks.get(streamId);
     811            if (existingTrack) {
     812                existingTrack->setIndex(i);
     813                if (existingTrack->pad() == pad)
     814                    continue;
     815            }
     816        }
     817
     818        RefPtr<VideoTrackPrivateGStreamer> track = VideoTrackPrivateGStreamer::create(createWeakPtr(), i, pad);
     819        ASSERT(streamId == track->id());
     820        m_videoTracks.add(streamId, track);
    661821        m_player->addVideoTrack(*track);
    662822    }
    663823
    664     while (static_cast<gint>(m_videoTracks.size()) > numTracks) {
    665         RefPtr<VideoTrackPrivateGStreamer> track = m_videoTracks.last();
    666         track->disconnect();
    667         m_videoTracks.removeLast();
    668         m_player->removeVideoTrack(*track);
    669     }
     824    purgeInvalidVideoTracks(validVideoStreams);
    670825#endif
    671826
     
    693848    if (UNLIKELY(!m_pipeline || !m_source))
    694849        return;
     850
     851    ASSERT(m_isLegacyPlaybin || isMediaSource());
    695852
    696853    gint numTracks = 0;
     
    712869
    713870#if ENABLE(VIDEO_TRACK)
     871    Vector<String> validAudioStreams;
    714872    for (gint i = 0; i < numTracks; ++i) {
    715873        GRefPtr<GstPad> pad;
     
    717875        ASSERT(pad);
    718876
     877        String streamId = "A" + String::number(i);
     878        validAudioStreams.append(streamId);
    719879        if (i < static_cast<gint>(m_audioTracks.size())) {
    720             RefPtr<AudioTrackPrivateGStreamer> existingTrack = m_audioTracks[i];
    721             existingTrack->setIndex(i);
    722             if (existingTrack->pad() == pad)
    723                 continue;
    724         }
    725 
    726         RefPtr<AudioTrackPrivateGStreamer> track = AudioTrackPrivateGStreamer::create(m_pipeline, i, pad);
    727         m_audioTracks.insert(i, track);
     880            RefPtr<AudioTrackPrivateGStreamer> existingTrack = m_audioTracks.get(streamId);
     881            if (existingTrack) {
     882                existingTrack->setIndex(i);
     883                if (existingTrack->pad() == pad)
     884                    continue;
     885            }
     886        }
     887
     888        RefPtr<AudioTrackPrivateGStreamer> track = AudioTrackPrivateGStreamer::create(createWeakPtr(), i, pad);
     889        ASSERT(streamId == track->id());
     890        m_audioTracks.add(streamId, track);
    728891        m_player->addAudioTrack(*track);
    729892    }
    730893
    731     while (static_cast<gint>(m_audioTracks.size()) > numTracks) {
    732         RefPtr<AudioTrackPrivateGStreamer> track = m_audioTracks.last();
    733         track->disconnect();
    734         m_audioTracks.removeLast();
    735         m_player->removeAudioTrack(*track);
    736     }
     894    purgeInvalidAudioTracks(validAudioStreams);
    737895#endif
    738896
     
    750908    if (UNLIKELY(!m_pipeline || !m_source))
    751909        return;
     910
     911    ASSERT(m_isLegacyPlaybin || isMediaSource());
    752912
    753913    gint numTracks = 0;
     
    761921    }
    762922
     923    Vector<String> validTextStreams;
    763924    for (gint i = 0; i < numTracks; ++i) {
    764925        GRefPtr<GstPad> pad;
     
    766927        ASSERT(pad);
    767928
     929        GRefPtr<GstEvent> event = adoptGRef(gst_pad_get_sticky_event(pad.get(), GST_EVENT_STREAM_START, 0));
     930        if (!event)
     931            continue;
     932
     933        const char* streamId;
     934        gst_event_parse_stream_start(event.get(), &streamId);
     935
     936        validTextStreams.append(streamId);
    768937        if (i < static_cast<gint>(m_textTracks.size())) {
    769             RefPtr<InbandTextTrackPrivateGStreamer> existingTrack = m_textTracks[i];
    770             existingTrack->setIndex(i);
    771             if (existingTrack->pad() == pad)
    772                 continue;
     938            RefPtr<InbandTextTrackPrivateGStreamer> existingTrack = m_textTracks.get(streamId);
     939            if (existingTrack) {
     940                existingTrack->setIndex(i);
     941                if (existingTrack->pad() == pad)
     942                    continue;
     943            }
    773944        }
    774945
    775946        RefPtr<InbandTextTrackPrivateGStreamer> track = InbandTextTrackPrivateGStreamer::create(i, pad);
    776         m_textTracks.insert(i, track);
     947        m_textTracks.add(streamId, track);
    777948        m_player->addTextTrack(*track);
    778949    }
    779950
    780     while (static_cast<gint>(m_textTracks.size()) > numTracks) {
    781         RefPtr<InbandTextTrackPrivateGStreamer> track = m_textTracks.last();
    782         track->disconnect();
    783         m_textTracks.removeLast();
    784         m_player->removeTextTrack(*track);
    785     }
     951    purgeInvalidTextTracks(validTextStreams);
    786952}
    787953
     
    809975        gst_event_parse_stream_start(streamStartEvent.get(), &id);
    810976        for (size_t i = 0; i < m_textTracks.size(); ++i) {
    811             RefPtr<InbandTextTrackPrivateGStreamer> track = m_textTracks[i];
    812             if (track->streamId() == id) {
     977            RefPtr<InbandTextTrackPrivateGStreamer> track = m_textTracks.get(id);
     978            if (track) {
    813979                track->handleSample(sample);
    814980                found = true;
     
    10981264        break;
    10991265    }
     1266#if USE(GSTREAMER_PLAYBIN3)
     1267    case GST_MESSAGE_STREAM_COLLECTION: {
     1268        GRefPtr<GstStreamCollection> collection;
     1269        gst_message_parse_stream_collection(message, &collection.outPtr());
     1270
     1271        if (collection) {
     1272            m_streamCollection.swap(collection);
     1273            m_notifier->notify(MainThreadNotification::StreamCollectionChanged, [this] {
     1274                this->updateTracks();
     1275            });
     1276        }
     1277        break;
     1278    }
     1279    case GST_MESSAGE_STREAMS_SELECTED: {
     1280        GRefPtr<GstStreamCollection> collection;
     1281        gst_message_parse_streams_selected(message, &collection.outPtr());
     1282
     1283        if (!collection)
     1284            break;
     1285
     1286        m_streamCollection.swap(collection);
     1287        m_currentAudioStreamId = "";
     1288        m_currentVideoStreamId = "";
     1289        m_currentTextStreamId = "";
     1290
     1291        unsigned length = gst_message_streams_selected_get_size(message);
     1292        for (unsigned i = 0; i < length; i++) {
     1293            GRefPtr<GstStream> stream = adoptGRef(gst_message_streams_selected_get_stream(message, i));
     1294            if (!stream)
     1295                continue;
     1296            GstStreamType type = gst_stream_get_stream_type(stream.get());
     1297            String streamId(gst_stream_get_stream_id(stream.get()));
     1298
     1299            GST_DEBUG("Selecting %s track with ID: %s", gst_stream_type_get_name(type), streamId.utf8().data());
     1300            // Playbin3 can send more than one selected stream of the same type
     1301            // but there's no priority or ordering system in place, so we assume
     1302            // the selected stream is the last one as reported by playbin3.
     1303            if (type & GST_STREAM_TYPE_AUDIO) {
     1304                m_currentAudioStreamId = streamId;
     1305                auto track = m_audioTracks.get(m_currentAudioStreamId);
     1306                ASSERT(track);
     1307                track->markAsActive();
     1308            } else if (type & GST_STREAM_TYPE_VIDEO) {
     1309                m_currentVideoStreamId = streamId;
     1310                auto track = m_videoTracks.get(m_currentVideoStreamId);
     1311                ASSERT(track);
     1312                track->markAsActive();
     1313            } else if (type & GST_STREAM_TYPE_TEXT)
     1314                m_currentTextStreamId = streamId;
     1315            else
     1316                GST_WARNING("Unknown stream type with stream-id %s", streamId);
     1317        }
     1318        break;
     1319    }
     1320#endif
    11001321    default:
    11011322        GST_DEBUG("Unhandled GStreamer message type: %s", GST_MESSAGE_TYPE_NAME(message));
     
    12141435        processTableOfContentsEntry(static_cast<GstTocEntry*>(i->data));
    12151436}
     1437
     1438void MediaPlayerPrivateGStreamer::purgeInvalidAudioTracks(Vector<String> validTrackIds)
     1439{
     1440    for (auto audioTrackId : m_audioTracks.keys()) {
     1441        if (validTrackIds.contains(audioTrackId))
     1442            continue;
     1443        auto track = m_audioTracks.get(audioTrackId);
     1444        track->disconnect();
     1445        m_player->removeAudioTrack(*track);
     1446        m_audioTracks.remove(audioTrackId);
     1447    }
     1448}
     1449
     1450void MediaPlayerPrivateGStreamer::purgeInvalidVideoTracks(Vector<String> validTrackIds)
     1451{
     1452    for (auto videoTrackId : m_videoTracks.keys()) {
     1453        if (validTrackIds.contains(videoTrackId))
     1454            continue;
     1455        auto track = m_videoTracks.get(videoTrackId);
     1456        track->disconnect();
     1457        m_player->removeVideoTrack(*track);
     1458        m_videoTracks.remove(videoTrackId);
     1459    }
     1460}
     1461
     1462void MediaPlayerPrivateGStreamer::purgeInvalidTextTracks(Vector<String> validTrackIds)
     1463{
     1464    for (auto textTrackId : m_textTracks.keys()) {
     1465        if (validTrackIds.contains(textTrackId))
     1466            continue;
     1467        auto track = m_textTracks.get(textTrackId);
     1468        track->disconnect();
     1469        m_player->removeTextTrack(*track);
     1470        m_textTracks.remove(textTrackId);
     1471    }
     1472}
    12161473#endif
    12171474
     
    14011658}
    14021659
    1403 void MediaPlayerPrivateGStreamer::sourceChangedCallback(MediaPlayerPrivateGStreamer* player)
    1404 {
    1405     player->sourceChanged();
     1660void MediaPlayerPrivateGStreamer::sourceSetupCallback(MediaPlayerPrivateGStreamer* player, GstElement* sourceElement)
     1661{
     1662    player->sourceSetup(sourceElement);
    14061663}
    14071664
     
    14201677    GUniquePtr<char> newDownloadTemplate(g_build_filename(G_DIR_SEPARATOR_S, "var", "tmp", "WebKit-Media-XXXXXX", nullptr));
    14211678    g_object_set(element, "temp-template", newDownloadTemplate.get(), nullptr);
    1422     GST_TRACE("Reconfigured file download template from '%s' to '%s'", oldDownloadTemplate.get(), newDownloadTemplate.get());
     1679    GST_DEBUG("Reconfigured file download template from '%s' to '%s'", oldDownloadTemplate.get(), newDownloadTemplate.get());
    14231680
    14241681    player->purgeOldDownloadFiles(oldDownloadTemplate.get());
     
    14401697    }
    14411698
    1442     GST_TRACE("Unlinked media temporary file %s after creation", downloadFile.get());
     1699    GST_DEBUG("Unlinked media temporary file %s after creation", downloadFile.get());
    14431700}
    14441701
     
    14621719}
    14631720
    1464 void MediaPlayerPrivateGStreamer::sourceChanged()
    1465 {
     1721void MediaPlayerPrivateGStreamer::sourceSetup(GstElement* sourceElement)
     1722{
     1723    GST_DEBUG("Source element set-up for %s", GST_ELEMENT_NAME(sourceElement));
     1724
    14661725    if (WEBKIT_IS_WEB_SRC(m_source.get()) && GST_OBJECT_PARENT(m_source.get()))
    14671726        g_signal_handlers_disconnect_by_func(GST_ELEMENT_PARENT(m_source.get()), reinterpret_cast<gpointer>(uriDecodeBinElementAddedCallback), this);
    14681727
    1469     m_source.clear();
    1470     g_object_get(m_pipeline.get(), "source", &m_source.outPtr(), nullptr);
     1728    m_source = sourceElement;
    14711729
    14721730    if (WEBKIT_IS_WEB_SRC(m_source.get())) {
     
    20202278
    20212279    // We don't want to stop downloading if we already started it.
    2022     if (flags & flagDownload && m_readyState > MediaPlayer::HaveNothing && !m_resetPipeline)
    2023         return;
     2280    if (flags & flagDownload && m_readyState > MediaPlayer::HaveNothing && !m_resetPipeline) {
     2281        GST_DEBUG("Download already started, not starting again");
     2282        return;
     2283    }
    20242284
    20252285    bool shouldDownload = !isLiveStream() && m_preload == MediaPlayer::Auto;
     
    20372297void MediaPlayerPrivateGStreamer::setPreload(MediaPlayer::Preload preload)
    20382298{
     2299    GST_DEBUG("Setting preload to %s", convertEnumerationToString(preload).utf8().data());
    20392300    if (preload == MediaPlayer::Auto && isLiveStream())
    20402301        return;
     
    21412402    // gst_element_factory_make() returns a floating reference so
    21422403    // we should not adopt.
     2404#if USE(GSTREAMER_PLAYBIN3)
     2405    m_isLegacyPlaybin = false;
     2406    setPipeline(gst_element_factory_make("playbin3", "play"));
     2407#else
     2408    m_isLegacyPlaybin = true;
    21432409    setPipeline(gst_element_factory_make("playbin", "play"));
     2410#endif
    21442411    setStreamVolumeElement(GST_STREAM_VOLUME(m_pipeline.get()));
     2412
     2413    GST_INFO("Using legacy playbin element: %s", boolForPrinting(m_isLegacyPlaybin));
    21452414
    21462415    // Let also other listeners subscribe to (application) messages in this bus.
     
    21512420    g_object_set(m_pipeline.get(), "mute", m_player->muted(), nullptr);
    21522421
    2153     g_signal_connect_swapped(m_pipeline.get(), "notify::source", G_CALLBACK(sourceChangedCallback), this);
    2154     g_signal_connect_swapped(m_pipeline.get(), "video-changed", G_CALLBACK(videoChangedCallback), this);
    2155     g_signal_connect_swapped(m_pipeline.get(), "audio-changed", G_CALLBACK(audioChangedCallback), this);
     2422    g_signal_connect_swapped(m_pipeline.get(), "source-setup", G_CALLBACK(sourceSetupCallback), this);
     2423    if (m_isLegacyPlaybin) {
     2424        g_signal_connect_swapped(m_pipeline.get(), "video-changed", G_CALLBACK(videoChangedCallback), this);
     2425        g_signal_connect_swapped(m_pipeline.get(), "audio-changed", G_CALLBACK(audioChangedCallback), this);
     2426    }
     2427
    21562428#if ENABLE(VIDEO_TRACK)
    2157     g_signal_connect_swapped(m_pipeline.get(), "text-changed", G_CALLBACK(textChangedCallback), this);
     2429    if (m_isLegacyPlaybin)
     2430        g_signal_connect_swapped(m_pipeline.get(), "text-changed", G_CALLBACK(textChangedCallback), this);
    21582431
    21592432    GstElement* textCombiner = webkitTextCombinerNew();
  • trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h

    r227061 r228617  
    2323 */
    2424
    25 #ifndef MediaPlayerPrivateGStreamer_h
    26 #define MediaPlayerPrivateGStreamer_h
     25#pragma once
     26
    2727#if ENABLE(VIDEO) && USE(GSTREAMER)
    2828
     
    3737#include <wtf/WeakPtr.h>
    3838
    39 #if ENABLE(VIDEO_TRACK) && USE(GSTREAMER_MPEGTS)
     39#if ENABLE(VIDEO_TRACK)
     40#include "TrackPrivateBaseGStreamer.h"
    4041#include <wtf/text/AtomicStringHash.h>
    4142#endif
    4243
    43 typedef struct _GstBuffer GstBuffer;
    44 typedef struct _GstMessage GstMessage;
    45 typedef struct _GstElement GstElement;
    4644typedef struct _GstMpegtsSection GstMpegtsSection;
    4745
     
    6866    virtual ~MediaPlayerPrivateGStreamer();
    6967
     68    WeakPtr<MediaPlayerPrivateGStreamer> createWeakPtr() { return m_weakPtrFactory.createWeakPtr(*this); }
     69
    7070    static void registerMediaEngine(MediaEngineRegistrar);
    7171    void handleMessage(GstMessage*);
     
    117117    void loadingFailed(MediaPlayer::NetworkState);
    118118
    119     virtual void sourceChanged();
     119    virtual void sourceSetup(GstElement*);
    120120
    121121    GstElement* audioSink() const override;
     
    131131
    132132    bool isLiveStream() const override { return m_isStreaming; }
     133
     134    void enableTrack(TrackPrivateBaseGStreamer::TrackType, unsigned index);
    133135
    134136private:
     
    138140    static bool isAvailable();
    139141
    140     WeakPtr<MediaPlayerPrivateGStreamer> createWeakPtr() { return m_weakPtrFactory.createWeakPtr(*this); }
    141 
    142142    GstElement* createAudioSink() override;
    143143
     
    154154    virtual void setDownloadBuffering();
    155155    void processBufferingStats(GstMessage*);
    156 #if ENABLE(VIDEO_TRACK) && USE(GSTREAMER_MPEGTS)
     156#if ENABLE(VIDEO_TRACK)
     157#if USE(GSTREAMER_MPEGTS)
    157158    void processMpegTsSection(GstMpegtsSection*);
    158159#endif
    159 #if ENABLE(VIDEO_TRACK)
     160
    160161    void processTableOfContents(GstMessage*);
    161162    void processTableOfContentsEntry(GstTocEntry*);
     163
     164    void purgeInvalidAudioTracks(Vector<String> validTrackIds);
     165    void purgeInvalidVideoTracks(Vector<String> validTrackIds);
     166    void purgeInvalidTextTracks(Vector<String> validTrackIds);
    162167#endif
    163168    virtual bool doSeek(const MediaTime& position, float rate, GstSeekFlags seekType);
     
    173178
    174179    void setPlaybinURL(const URL& urlString);
     180
     181#if USE(GSTREAMER_PLAYBIN3)
     182    void updateTracks();
     183#endif
    175184
    176185protected:
     
    215224    static void setAudioStreamPropertiesCallback(MediaPlayerPrivateGStreamer*, GObject*);
    216225
    217     static void sourceChangedCallback(MediaPlayerPrivateGStreamer*);
     226    static void sourceSetupCallback(MediaPlayerPrivateGStreamer*, GstElement*);
    218227    static void videoChangedCallback(MediaPlayerPrivateGStreamer*);
    219228    static void videoSinkCapsChangedCallback(MediaPlayerPrivateGStreamer*);
     
    248257    URL m_url;
    249258    bool m_preservesPitch;
     259    bool m_isLegacyPlaybin;
     260#if USE(GSTREAMER_PLAYBIN3)
     261    GRefPtr<GstStreamCollection> m_streamCollection;
     262#endif
     263    String m_currentAudioStreamId;
     264    String m_currentVideoStreamId;
     265    String m_currentTextStreamId;
    250266#if ENABLE(WEB_AUDIO)
    251267    std::unique_ptr<AudioSourceProviderGStreamer> m_audioSourceProvider;
     
    255271    RefPtr<MediaPlayerRequestInstallMissingPluginsCallback> m_missingPluginsCallback;
    256272#if ENABLE(VIDEO_TRACK)
    257     Vector<RefPtr<AudioTrackPrivateGStreamer>> m_audioTracks;
    258     Vector<RefPtr<InbandTextTrackPrivateGStreamer>> m_textTracks;
    259     Vector<RefPtr<VideoTrackPrivateGStreamer>> m_videoTracks;
     273    HashMap<AtomicString, RefPtr<AudioTrackPrivateGStreamer>> m_audioTracks;
     274    HashMap<AtomicString, RefPtr<InbandTextTrackPrivateGStreamer>> m_textTracks;
     275    HashMap<AtomicString, RefPtr<VideoTrackPrivateGStreamer>> m_videoTracks;
    260276    RefPtr<InbandMetadataTextTrackPrivateGStreamer> m_chaptersTrack;
    261 #endif
    262 #if ENABLE(VIDEO_TRACK) && USE(GSTREAMER_MPEGTS)
     277#if USE(GSTREAMER_MPEGTS)
    263278    HashMap<AtomicString, RefPtr<InbandMetadataTextTrackPrivateGStreamer>> m_metadataTracks;
     279#endif
    264280#endif
    265281    virtual bool isMediaSource() const { return false; }
     
    268284
    269285#endif // USE(GSTREAMER)
    270 #endif
  • trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h

    r226947 r228617  
    210210        TextChanged = 1 << 5,
    211211#endif
    212         SizeChanged = 1 << 6
     212        SizeChanged = 1 << 6,
     213#if ENABLE(GSTREAMER_PLAYBIN3)
     214        StreamCollectionChanged = 1 << 7
     215#endif
    213216    };
    214217
  • trunk/Source/WebCore/platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp

    r218471 r228617  
    5555    g_signal_connect_swapped(m_pad.get(), "notify::tags", G_CALLBACK(tagsChangedCallback), this);
    5656
    57     // We can't call notifyTrackOfTagsChanged() directly, because we need tagsChanged()
    58     // to setup m_tags.
     57    // We can't call notifyTrackOfTagsChanged() directly, because we need tagsChanged() to setup m_tags.
    5958    tagsChanged();
    6059}
     60
     61#if USE(GSTREAMER_PLAYBIN3)
     62TrackPrivateBaseGStreamer::TrackPrivateBaseGStreamer(TrackPrivateBase* owner, gint index, GRefPtr<GstStream> stream)
     63    : m_notifier(MainThreadNotifier<MainThreadNotification>::create())
     64    , m_index(index)
     65    , m_stream(stream)
     66    , m_owner(owner)
     67{
     68    ASSERT(m_stream);
     69
     70    // We can't call notifyTrackOfTagsChanged() directly, because we need tagsChanged() to setup m_tags.
     71    tagsChanged();
     72}
     73#endif
    6174
    6275TrackPrivateBaseGStreamer::~TrackPrivateBaseGStreamer()
     
    6881void TrackPrivateBaseGStreamer::disconnect()
    6982{
     83    m_tags.clear();
     84
     85#if USE(GSTREAMER_PLAYBIN3)
     86    if (m_stream)
     87        m_stream.clear();
     88#endif
     89
     90    m_notifier->cancelPendingNotifications();
     91
    7092    if (!m_pad)
    7193        return;
    7294
    73     m_notifier->cancelPendingNotifications();
    7495    g_signal_handlers_disconnect_matched(m_pad.get(), G_SIGNAL_MATCH_DATA, 0, 0, nullptr, nullptr, this);
    75 
    7696    m_pad.clear();
    77     m_tags.clear();
    7897}
    7998
     
    91110{
    92111    GRefPtr<GstTagList> tags;
    93     if (g_object_class_find_property(G_OBJECT_GET_CLASS(m_pad.get()), "tags"))
    94         g_object_get(m_pad.get(), "tags", &tags.outPtr(), nullptr);
     112    if (m_pad) {
     113        if (g_object_class_find_property(G_OBJECT_GET_CLASS(m_pad.get()), "tags"))
     114            g_object_get(m_pad.get(), "tags", &tags.outPtr(), nullptr);
     115        else
     116            tags = adoptGRef(gst_tag_list_new_empty());
     117    }
     118#if USE(GSTREAMER_PLAYBIN3)
     119    else if (m_stream)
     120        tags = adoptGRef(gst_stream_get_tags(m_stream.get()));
     121#endif
    95122    else
    96123        tags = adoptGRef(gst_tag_list_new_empty());
    97124
     125    GST_DEBUG("Inspecting track at index %d with tags: %" GST_PTR_FORMAT, m_index, tags.get());
    98126    {
    99127        LockHolder lock(m_tagMutex);
     
    110138
    111139    gboolean active = false;
    112     if (m_pad && g_object_class_find_property(G_OBJECT_GET_CLASS(m_pad.get()), "active"))
     140    if (g_object_class_find_property(G_OBJECT_GET_CLASS(m_pad.get()), "active"))
    113141        g_object_get(m_pad.get(), "active", &active, nullptr);
    114142
     
    144172void TrackPrivateBaseGStreamer::notifyTrackOfTagsChanged()
    145173{
    146     if (!m_pad)
    147         return;
    148 
    149174    TrackPrivateBaseClient* client = m_owner->client();
    150175    if (!client)
  • trunk/Source/WebCore/platform/graphics/gstreamer/TrackPrivateBaseGStreamer.h

    r218799 r228617  
    2424 */
    2525
    26 #ifndef TrackPrivateBaseGStreamer_h
    27 #define TrackPrivateBaseGStreamer_h
     26#pragma once
    2827
    2928#if ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
     
    3130#include "GRefPtrGStreamer.h"
    3231#include "MainThreadNotifier.h"
     32#include <gst/gst.h>
    3333#include <wtf/Lock.h>
    3434#include <wtf/text/WTFString.h>
     
    4242    virtual ~TrackPrivateBaseGStreamer();
    4343
     44    enum TrackType {
     45        Audio,
     46        Video,
     47        Text,
     48        Unknown
     49    };
     50
    4451    GstPad* pad() const { return m_pad.get(); }
    4552
     
    5259protected:
    5360    TrackPrivateBaseGStreamer(TrackPrivateBase* owner, gint index, GRefPtr<GstPad>);
    54 
     61#if USE(GSTREAMER_PLAYBIN3)
     62    TrackPrivateBaseGStreamer(TrackPrivateBase* owner, gint index, GRefPtr<GstStream>);
     63#endif
    5564    void notifyTrackOfActiveChanged();
    5665    void notifyTrackOfTagsChanged();
     
    6877    AtomicString m_language;
    6978    GRefPtr<GstPad> m_pad;
     79#if USE(GSTREAMER_PLAYBIN3)
     80    GRefPtr<GstStream> m_stream;
     81#endif
    7082
    7183private:
     
    88100
    89101#endif // ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
    90 
    91 #endif // TrackPrivateBaseGStreamer_h
  • trunk/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp

    r213445 r228617  
    3030#include "VideoTrackPrivateGStreamer.h"
    3131
     32#include "MediaPlayerPrivateGStreamer.h"
    3233#include <glib-object.h>
    3334
    3435namespace WebCore {
    3536
    36 VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer(GRefPtr<GstElement> playbin, gint index, GRefPtr<GstPad> pad)
     37VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstPad> pad)
    3738    : TrackPrivateBaseGStreamer(this, index, pad)
    38     , m_playbin(playbin)
     39    , m_player(player)
    3940{
    4041    // FIXME: Get a real ID from the tkhd atom.
     
    4344}
    4445
     46#if USE(GSTREAMER_PLAYBIN3)
     47VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstStream> stream)
     48    : TrackPrivateBaseGStreamer(this, index, stream)
     49    , m_player(player)
     50{
     51    m_id = gst_stream_get_stream_id(stream.get());
     52    notifyTrackOfActiveChanged();
     53}
     54#endif
     55
    4556void VideoTrackPrivateGStreamer::disconnect()
    4657{
    47     m_playbin.clear();
     58    m_player = nullptr;
    4859    TrackPrivateBaseGStreamer::disconnect();
     60}
     61
     62void VideoTrackPrivateGStreamer::markAsActive()
     63{
     64    VideoTrackPrivate::setSelected(true);
    4965}
    5066
     
    5571    VideoTrackPrivate::setSelected(selected);
    5672
    57     if (selected && m_playbin)
    58         g_object_set(m_playbin.get(), "current-video", m_index, nullptr);
     73    if (selected && m_player)
     74        m_player->enableTrack(TrackPrivateBaseGStreamer::TrackType::Video, m_index);
    5975}
    6076
  • trunk/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.h

    r216702 r228617  
    2424 */
    2525
    26 #ifndef VideoTrackPrivateGStreamer_h
    27 #define VideoTrackPrivateGStreamer_h
     26#pragma once
    2827
    2928#if ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
     
    3332#include "VideoTrackPrivate.h"
    3433
     34#include <gst/gst.h>
     35#include <wtf/WeakPtr.h>
     36
    3537namespace WebCore {
     38class MediaPlayerPrivateGStreamer;
    3639
    3740class VideoTrackPrivateGStreamer final : public VideoTrackPrivate, public TrackPrivateBaseGStreamer {
    3841public:
    39     static Ref<VideoTrackPrivateGStreamer> create(GRefPtr<GstElement> playbin, gint index, GRefPtr<GstPad> pad)
     42    static Ref<VideoTrackPrivateGStreamer> create(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstPad> pad)
    4043    {
    41         return adoptRef(*new VideoTrackPrivateGStreamer(playbin, index, pad));
     44        return adoptRef(*new VideoTrackPrivateGStreamer(player, index, pad));
    4245    }
     46#if USE(GSTREAMER_PLAYBIN3)
     47    static Ref<VideoTrackPrivateGStreamer> create(WeakPtr<MediaPlayerPrivateGStreamer> player, gint index, GRefPtr<GstStream> stream)
     48    {
     49        return adoptRef(*new VideoTrackPrivateGStreamer(player, index, stream));
     50    }
     51#endif
    4352
    4453    void disconnect() override;
    4554
     55    void markAsActive();
    4656    void setSelected(bool) override;
    4757    void setActive(bool enabled) override { setSelected(enabled); }
     
    5464
    5565private:
    56     VideoTrackPrivateGStreamer(GRefPtr<GstElement> playbin, gint index, GRefPtr<GstPad>);
    57 
     66    VideoTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer>, gint index, GRefPtr<GstPad>);
     67#if USE(GSTREAMER_PLAYBIN3)
     68    VideoTrackPrivateGStreamer(WeakPtr<MediaPlayerPrivateGStreamer>, gint index, GRefPtr<GstStream>);
     69#endif
    5870    AtomicString m_id;
    59     GRefPtr<GstElement> m_playbin;
     71    WeakPtr<MediaPlayerPrivateGStreamer> m_player;
    6072};
    6173
     
    6375
    6476#endif // ENABLE(VIDEO) && USE(GSTREAMER) && ENABLE(VIDEO_TRACK)
    65 
    66 #endif // VideoTrackPrivateGStreamer_h
  • trunk/Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp

    r228316 r228617  
    10741074    case WebCore::MediaSourceStreamTypeGStreamer::Audio:
    10751075        if (m_playerPrivate)
    1076             m_track = WebCore::AudioTrackPrivateGStreamer::create(m_playerPrivate->pipeline(), id(), sinkSinkPad.get());
     1076            m_track = WebCore::AudioTrackPrivateGStreamer::create(m_playerPrivate->createWeakPtr(), id(), sinkSinkPad.get());
    10771077        break;
    10781078    case WebCore::MediaSourceStreamTypeGStreamer::Video:
    10791079        if (m_playerPrivate)
    1080             m_track = WebCore::VideoTrackPrivateGStreamer::create(m_playerPrivate->pipeline(), id(), sinkSinkPad.get());
     1080            m_track = WebCore::VideoTrackPrivateGStreamer::create(m_playerPrivate->createWeakPtr(), id(), sinkSinkPad.get());
    10811081        break;
    10821082    case WebCore::MediaSourceStreamTypeGStreamer::Text:
  • trunk/Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp

    r228316 r228617  
    487487}
    488488
    489 void MediaPlayerPrivateGStreamerMSE::sourceChanged()
    490 {
    491     m_source = nullptr;
    492     g_object_get(m_pipeline.get(), "source", &m_source.outPtr(), nullptr);
     489void MediaPlayerPrivateGStreamerMSE::sourceSetup(GstElement* sourceElement)
     490{
     491    m_source = sourceElement;
    493492
    494493    ASSERT(WEBKIT_IS_MEDIA_SRC(m_source.get()));
  • trunk/Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h

    r225594 r228617  
    7272    MediaTime maxMediaTimeSeekable() const override;
    7373
    74     void sourceChanged() override;
     74    void sourceSetup(GstElement*) override;
    7575
    7676    void setReadyState(MediaPlayer::ReadyState);
  • trunk/Source/cmake/GStreamerDefinitions.cmake

    r225400 r228617  
    66WEBKIT_OPTION_DEFINE(USE_GSTREAMER_GL "Whether to enable support for GStreamer GL" PRIVATE ON)
    77WEBKIT_OPTION_DEFINE(USE_GSTREAMER_MPEGTS "Whether to enable support for MPEG-TS" PRIVATE OFF)
     8WEBKIT_OPTION_DEFINE(USE_GSTREAMER_PLAYBIN3 "Whether to enable support for GStreamer's playbin3 element" PRIVATE OFF)
Note: See TracChangeset for help on using the changeset viewer.