Changeset 145811 in webkit


Ignore:
Timestamp:
Mar 14, 2013 7:39:17 AM (11 years ago)
Author:
commit-queue@webkit.org
Message:

[GStreamer] Stopping playback of html5 media when receiving a higher priority audio event needs implementation
​https://bugs.webkit.org/show_bug.cgi?id=91611

Source/WebCore:

React to REQUEST_STATE GStreamer message to stop the pipeline when
a higher priority stream is played. When this happens, states are
updated accordingly.

A method was added in the MediaPlayer class and internals to allow
the the test runner to simulate an audio interruption.

Patch by Xabier Rodriguez Calvar <​calvaris@igalia.com> on 2013-03-14
Reviewed by Philippe Normand.

Test: media/media-higher-prio-audio-stream.html

  • platform/graphics/MediaPlayer.h:
  • platform/graphics/MediaPlayer.cpp:

(WebCore):
(MediaPlayer):
(WebCore::MediaPlayer::simulateAudioInterruption): New method
delegating an audio interruption to the private backend to
simulate the use-case where an external application needs
exclusive access to the audio device.

  • platform/graphics/MediaPlayerPrivate.h:

(MediaPlayerPrivateInterface):
(WebCore::MediaPlayerPrivateInterface::simulateAudioInterruption):
Added default empty method in the common private header.

  • platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:

(WebCore):
(WebCore::MediaPlayerPrivateGStreamer::createAudioSink):
(WebCore::setAudioStreamPropertiesCallback): Hooked to child-added
signal on the audio sink, delegates on setAudioStreamProperties.
(WebCore::MediaPlayerPrivateGStreamer::setAudioStreamProperties):
Sets the audio stream properties.
(WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer):
Initializes the new attribute.
(WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer):
Disconnects autoaudiosink signal.
(WebCore::MediaPlayerPrivateGStreamer::changePipelineState):
Changed logging.
(WebCore::MediaPlayerPrivateGStreamer::handleMessage): Reacting to
the REQUEST_STATE message.
(WebCore::MediaPlayerPrivateGStreamer::simulateAudioInterruption):
Added. Injects the REQUEST_STATE message to the pipeline.
(WebCore::MediaPlayerPrivateGStreamer::updateStates): Updating the
playback state if REQUEST_STATE.

  • platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:

(MediaPlayerPrivateGStreamer): Added new method and attribute.

  • testing/Internals.h:
  • testing/Internals.idl:
  • testing/Internals.cpp:

(WebCore):
(WebCore::Internals::simulateAudioInterruption): Added to call the
method to stop the element because of a higher prio stream at the
tests.

LayoutTests:

Created test, expected result and updated other ports
expectations.

Patch by Xabier Rodriguez Calvar <​calvaris@igalia.com> on 2013-03-14
Reviewed by Philippe Normand.

  • media/media-higher-prio-audio-stream-expected.txt: Added.
  • media/media-higher-prio-audio-stream.html: Added.
  • platform/chromium/TestExpectations: Skipped the new test.
  • platform/mac/TestExpectations: Skipped the new test.
  • platform/qt/TestExpectations: Skipped the new test for Mac and

Win.

Location:
trunk
Files:
2 added
13 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r145799 r145811  
     12013-03-14  Xabier Rodriguez Calvar  <calvaris@igalia.com>
     2
     3        [GStreamer] Stopping playback of html5 media when receiving a higher priority audio event needs implementation
     4        https://bugs.webkit.org/show_bug.cgi?id=91611
     5
     6        Created test, expected result and updated other ports
     7        expectations.
     8
     9        Reviewed by Philippe Normand.
     10
     11        * media/media-higher-prio-audio-stream-expected.txt: Added.
     12        * media/media-higher-prio-audio-stream.html: Added.
     13        * platform/chromium/TestExpectations: Skipped the new test.
     14        * platform/mac/TestExpectations: Skipped the new test.
     15        * platform/qt/TestExpectations: Skipped the new test for Mac and
     16        Win.
     17
    1182013-03-14  Zoltan Arvai  <zarvai@inf.u-szeged.hu>
    219

  • trunk/LayoutTests/platform/chromium/TestExpectations

    r145794 r145811  
    44544454webkit.org/b/112219 fast/layers/no-clipping-overflow-hidden-hardware-acceleration.html [ ImageOnlyFailure Pass ]
    44554455webkit.org/b/112219 fast/layers/no-clipping-overflow-hidden-added-after-transition.html [ ImageOnlyFailure Pass ]
     4456
     4457# Feature not implemented
     4458webkit.org/b/91611 media/media-higher-prio-audio-stream.html [ Skip ]

  • trunk/LayoutTests/platform/mac/TestExpectations

    r145797 r145811  
    14811481# webkit.org/b/105998 [ Lion Release ] fast/css/sticky/inline-sticky-abspos-child.html [ ImageOnlyFailure ]
    14821482# webkit.org/b/105998 [ MountainLion Debug ] fast/css/sticky/inline-sticky-abspos-child.html [ ImageOnlyFailure Pass ]
     1483
     1484# Feature not implemented
     1485webkit.org/b/91611 media/media-higher-prio-audio-stream.html [ Skip ]

  • trunk/LayoutTests/platform/qt/TestExpectations

    r145798 r145811  
    26932693# [Qt] Two tests hit assertion fail after r140999.
    26942694webkit.org/b/108257 compositing/overflow/composited-scrolling-creates-a-stacking-container.html [ Skip ]
     2695
     2696# Feature not implemented
     2697webkit.org/b/91611 [ Mac Win ] media/media-higher-prio-audio-stream.html [ Skip ]

  • trunk/Source/WebCore/ChangeLog

    r145810 r145811  
     12013-03-14  Xabier Rodriguez Calvar  <calvaris@igalia.com>
     2
     3        [GStreamer] Stopping playback of html5 media when receiving a higher priority audio event needs implementation
     4        https://bugs.webkit.org/show_bug.cgi?id=91611
     5
     6        React to REQUEST_STATE GStreamer message to stop the pipeline when
     7        a higher priority stream is played. When this happens, states are
     8        updated accordingly.
     9
     10        A method was added in the MediaPlayer class and internals to allow
     11        the the test runner to simulate an audio interruption.
     12
     13        Reviewed by Philippe Normand.
     14
     15        Test: media/media-higher-prio-audio-stream.html
     16
     17        * platform/graphics/MediaPlayer.h:
     18        * platform/graphics/MediaPlayer.cpp:
     19        (WebCore):
     20        (MediaPlayer):
     21        (WebCore::MediaPlayer::simulateAudioInterruption): New method
     22        delegating an audio interruption to the private backend to
     23        simulate the use-case where an external application needs
     24        exclusive access to the audio device.
     25        * platform/graphics/MediaPlayerPrivate.h:
     26        (MediaPlayerPrivateInterface):
     27        (WebCore::MediaPlayerPrivateInterface::simulateAudioInterruption):
     28        Added default empty method in the common private header.
     29        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
     30        (WebCore):
     31        (WebCore::MediaPlayerPrivateGStreamer::createAudioSink):
     32        (WebCore::setAudioStreamPropertiesCallback): Hooked to child-added
     33        signal on the audio sink, delegates on setAudioStreamProperties.
     34        (WebCore::MediaPlayerPrivateGStreamer::setAudioStreamProperties):
     35        Sets the audio stream properties.
     36        (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer):
     37        Initializes the new attribute.
     38        (WebCore::MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer):
     39        Disconnects autoaudiosink signal.
     40        (WebCore::MediaPlayerPrivateGStreamer::changePipelineState):
     41        Changed logging.
     42        (WebCore::MediaPlayerPrivateGStreamer::handleMessage): Reacting to
     43        the REQUEST_STATE message.
     44        (WebCore::MediaPlayerPrivateGStreamer::simulateAudioInterruption):
     45        Added. Injects the REQUEST_STATE message to the pipeline.
     46        (WebCore::MediaPlayerPrivateGStreamer::updateStates): Updating the
     47        playback state if REQUEST_STATE.
     48        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
     49        (MediaPlayerPrivateGStreamer): Added new method and attribute.
     50        * testing/Internals.h:
     51        * testing/Internals.idl:
     52        * testing/Internals.cpp:
     53        (WebCore):
     54        (WebCore::Internals::simulateAudioInterruption): Added to call the
     55        method to stop the element because of a higher prio stream at the
     56        tests.
     57
    1582013-03-14  Allan Sandfeld Jensen  <allan.jensen@digia.com>
    259

  • trunk/Source/WebCore/platform/graphics/MediaPlayer.cpp

    r145669 r145811  
    11451145}
    11461146
    1147 }
    1148 
    1149 #endif
     1147#if USE(GSTREAMER)
     1148void MediaPlayer::simulateAudioInterruption()
     1149{
     1150    if (!m_private)
     1151        return;
     1152
     1153    m_private->simulateAudioInterruption();
     1154}
     1155#endif
     1156
     1157}
     1158
     1159#endif

  • trunk/Source/WebCore/platform/graphics/MediaPlayer.h

    r145669 r145811  
    463463#endif
    464464
     465#if USE(GSTREAMER)
     466    virtual void simulateAudioInterruption();
     467#endif
     468
    465469private:
    466470    MediaPlayer(MediaPlayerClient*);

  • trunk/Source/WebCore/platform/graphics/MediaPlayerPrivate.h

    r145669 r145811  
    189189#endif
    190190
     191#if USE(GSTREAMER)
     192    virtual void simulateAudioInterruption() { }
     193#endif
    191194};
    192195

  • trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp

    r142810 r145811  
    114114}
    115115
     116#ifdef GST_API_VERSION_1
     117static void setAudioStreamPropertiesCallback(GstChildProxy*, GObject* object, gchar*,
     118    MediaPlayerPrivateGStreamer* player)
     119#else
     120static void setAudioStreamPropertiesCallback(GstChildProxy*, GObject* object, MediaPlayerPrivateGStreamer* player)
     121#endif
     122{
     123    player->setAudioStreamProperties(object);
     124}
     125
    116126static gboolean mediaPlayerPrivateVideoChangeTimeoutCallback(MediaPlayerPrivateGStreamer* player)
    117127{
    … …  
    119129    player->notifyPlayerOfVideo();
    120130    return FALSE;
     131}
     132
     133void MediaPlayerPrivateGStreamer::setAudioStreamProperties(GObject* object)
     134{
     135    if (g_strcmp0(G_OBJECT_TYPE_NAME(object), "GstPulseSink"))
     136        return;
     137
     138    const char* role = m_player->mediaPlayerClient() && m_player->mediaPlayerClient()->mediaPlayerIsVideo()
     139        ? "video" : "music";
     140    GstStructure* structure = gst_structure_new("stream-properties", "media.role", G_TYPE_STRING, role, NULL);
     141    g_object_set(object, "stream-properties", structure, NULL);
     142    gst_structure_free(structure);
     143    LOG_MEDIA_MESSAGE("Set media.role as %s at %s", role, gst_element_get_name(GST_ELEMENT(object)));
    121144}
    122145
    … …  
    188211    , m_originalPreloadWasAutoAndWasOverridden(false)
    189212    , m_preservesPitch(false)
     213    , m_requestedState(GST_STATE_VOID_PENDING)
    190214{
    191215}
    … …  
    200224        m_mediaLocations = 0;
    201225    }
     226
     227    if (m_autoAudioSink)
     228        g_signal_handlers_disconnect_by_func(G_OBJECT(m_autoAudioSink.get()),
     229            reinterpret_cast<gpointer>(setAudioStreamPropertiesCallback), this);
    202230
    203231    if (m_playBin) {
    … …  
    322350
    323351    gst_element_get_state(m_playBin.get(), &currentState, &pending, 0);
    324     LOG_MEDIA_MESSAGE("Current state: %s, pending: %s", gst_element_state_get_name(currentState), gst_element_state_get_name(pending));
    325     if (currentState == newState || pending == newState)
     352    if (currentState == newState || pending == newState) {
     353        LOG_MEDIA_MESSAGE("Rejected state change to %s from %s with %s pending", gst_element_state_get_name(newState),
     354            gst_element_state_get_name(currentState), gst_element_state_get_name(pending));
    326355        return true;
     356    }
     357
     358    LOG_MEDIA_MESSAGE("Changing state change to %s from %s with %s pending", gst_element_state_get_name(newState),
     359        gst_element_state_get_name(currentState), gst_element_state_get_name(pending));
    327360
    328361    GstStateChangeReturn setStateResult = gst_element_set_state(m_playBin.get(), newState);
    … …  
    628661    bool attemptNextLocation = false;
    629662    const GstStructure* structure = gst_message_get_structure(message);
     663    GstState requestedState, currentState;
    630664
    631665    if (structure) {
    … …  
    693727
    694728            // Construct a filename for the graphviz dot file output.
    695             GstState oldState, newState;
    696             gst_message_parse_state_changed(message, &oldState, &newState, 0);
     729            GstState newState;
     730            gst_message_parse_state_changed(message, &currentState, &newState, 0);
    697731
    698732            CString dotFileName = String::format("webkit-video.%s_%s",
    699                                                  gst_element_state_get_name(oldState),
    700                                                  gst_element_state_get_name(newState)).utf8();
     733                gst_element_state_get_name(currentState),
     734                gst_element_state_get_name(newState)).utf8();
    701735
    702736            GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS(GST_BIN(m_playBin.get()), GST_DEBUG_GRAPH_SHOW_ALL, dotFileName.data());
    … …  
    713747        LOG_MEDIA_MESSAGE("Duration changed");
    714748        durationChanged();
     749        break;
     750    case GST_MESSAGE_REQUEST_STATE:
     751        gst_message_parse_request_state(message, &requestedState);
     752        gst_element_get_state(m_playBin.get(), &currentState, NULL, 250);
     753        if (requestedState < currentState) {
     754            LOG_MEDIA_MESSAGE("Element %s requested state change to %s", gst_element_get_name(GST_MESSAGE_SRC(message)),
     755                gst_element_state_get_name(requestedState));
     756            m_requestedState = requestedState;
     757            changePipelineState(requestedState);
     758        }
    715759        break;
    716760    default:
    … …  
    9631007
    9641008    bool shouldUpdateAfterSeek = false;
     1009    bool shouldUpdatePlaybackState = false;
    9651010    switch (ret) {
    9661011    case GST_STATE_CHANGE_SUCCESS:
    … …  
    10491094        }
    10501095
     1096        if (m_requestedState == GST_STATE_PAUSED && state == GST_STATE_PAUSED) {
     1097            shouldUpdatePlaybackState = true;
     1098            LOG_MEDIA_MESSAGE("Requested state change to %s was completed", gst_element_state_get_name(state));
     1099        }
     1100
    10511101        break;
    10521102    case GST_STATE_CHANGE_ASYNC:
    … …  
    11151165    }
    11161166
     1167    m_requestedState = GST_STATE_VOID_PENDING;
     1168
    11171169    if (seeking())
    11181170        m_readyState = MediaPlayer::HaveNothing;
    … …  
    11201172    if (shouldUpdateAfterSeek)
    11211173        timeChanged();
     1174
     1175    if (shouldUpdatePlaybackState)
     1176        m_player->playbackStateChanged();
    11221177
    11231178    if (m_networkState != oldNetworkState) {
    … …  
    14951550    GstElement* sink = gst_element_factory_make("autoaudiosink", 0);
    14961551
     1552    m_autoAudioSink = sink;
     1553
     1554    g_signal_connect(sink, "child-added", G_CALLBACK(setAudioStreamPropertiesCallback), this);
     1555
    14971556    GstElement* audioSink = gst_bin_new("audio-sink");
    14981557    gst_bin_add_many(GST_BIN(audioSink), scale, convert, resample, sink, NULL);
    … …  
    15401599}
    15411600
     1601void MediaPlayerPrivateGStreamer::simulateAudioInterruption()
     1602{
     1603    GstMessage* message = gst_message_new_request_state(GST_OBJECT(m_playBin.get()), GST_STATE_PAUSED);
     1604    gst_element_post_message(m_playBin.get(), message);
     1605}
     1606
    15421607}
    15431608

  • trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h

    r142845 r145811  
    8989    GstElement* audioSink() const;
    9090
     91    void setAudioStreamProperties(GObject*);
     92
     93    void simulateAudioInterruption();
     94
    9195private:
    9296    MediaPlayerPrivateGStreamer(MediaPlayer*);
    … …  
    154158    bool m_originalPreloadWasAutoAndWasOverridden;
    155159    bool m_preservesPitch;
     160    GstState m_requestedState;
     161    GRefPtr<GstElement> m_autoAudioSink;
    156162};
    157163}

  • trunk/Source/WebCore/testing/Internals.cpp

    r145750 r145811  
    5252#include "HTMLContentElement.h"
    5353#include "HTMLInputElement.h"
     54#include "HTMLMediaElement.h"
    5455#include "HTMLNames.h"
    5556#include "HTMLTextAreaElement.h"
    … …  
    20722073}
    20732074
    2074 }
     2075void Internals::simulateAudioInterruption(Node* node)
     2076{
     2077#if USE(GSTREAMER)
     2078    HTMLMediaElement* element = toMediaElement(node);
     2079    element->player()->simulateAudioInterruption();
     2080#else
     2081    UNUSED_PARAM(node);
     2082#endif
     2083}
     2084
     2085}

  • trunk/Source/WebCore/testing/Internals.h

    r145750 r145811  
    301301    String getImageSourceURL(Element*, ExceptionCode&);
    302302                   
     303    void simulateAudioInterruption(Node*);
     304
    303305private:
    304306    explicit Internals(Document*);

  • trunk/Source/WebCore/testing/Internals.idl

    r145750 r145811  
    260260    void forceReload(in boolean endToEnd);
    261261
     262    void simulateAudioInterruption(in Node node);
     263
    262264    [Conditional=ENCRYPTED_MEDIA_V2] void initializeMockCDM();
    263265
Note: See TracChangeset for help on using the changeset viewer.