Changeset 208606 in webkit


Ignore:
Timestamp:
Nov 11, 2016, 1:45:21 PM (8 years ago)
Author:
eric.carlson@apple.com
Message:

[MediaStream] defer resolution of getUserMedia promise made in a background tab
​https://bugs.webkit.org/show_bug.cgi?id=164643
<rdar://problem/29048317>

Reviewed by Brady Eidson.

Source/WebCore:

Test: fast/mediastream/get-user-media-background-tab.html

Do not start producing data when the document does not allow media
to start playing. Instead, register with the document for a callback
when playback is allowed and start then.

  • Modules/mediastream/MediaStream.cpp:

(WebCore::MediaStream::~MediaStream):
(WebCore::MediaStream::mediaCanStart):
(WebCore::MediaStream::startProducingData):
(WebCore::MediaStream::stopProducingData):
(WebCore::MediaStream::pageMutedStateDidChange):

  • Modules/mediastream/MediaStream.h:
  • Modules/webaudio/AudioContext.cpp:

(WebCore::AudioContext::mediaCanStart): Deal with API change.

  • Modules/webaudio/AudioContext.h:
  • html/HTMLMediaElement.cpp:

(WebCore::HTMLMediaElement::mediaCanStart): Ditto.

  • html/HTMLMediaElement.h:
  • page/MediaCanStartListener.h:
  • page/Page.cpp:

(WebCore::Page::takeAnyMediaCanStartListener): Return the listener and document.
(WebCore::Page::setCanStartMedia): Pass the document to the listener.

  • page/Page.h:

Source/WebKit2:

Do not ask for user for access to capture devices when the document does not
allow media to play, wait until playback is allowed.

  • WebProcess/MediaStream/UserMediaPermissionRequestManager.cpp:

(WebKit::UserMediaPermissionRequestManager::startUserMediaRequest):
(WebKit::UserMediaPermissionRequestManager::sendUserMediaRequest):
(WebKit::UserMediaPermissionRequestManager::mediaCanStart):
(WebKit::UserMediaPermissionRequestManager::grantUserMediaDevicesSandboxExtension):

  • WebProcess/MediaStream/UserMediaPermissionRequestManager.h:
  • WebProcess/Plugins/PluginView.cpp:

(WebKit::PluginView::mediaCanStart):

  • WebProcess/Plugins/PluginView.h:

LayoutTests:

  • fast/mediastream/get-user-media-background-tab-expected.txt: Added.
  • fast/mediastream/get-user-media-background-tab.html: Added.
Location:
trunk
Files:
2 added
16 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r208602 r208606  
     12016-11-11  Eric Carlson  <eric.carlson@apple.com>
     2
     3        [MediaStream] defer resolution of getUserMedia promise made in a background tab
     4        https://bugs.webkit.org/show_bug.cgi?id=164643
     5        <rdar://problem/29048317>
     6
     7        Reviewed by Brady Eidson.
     8
     9        * fast/mediastream/get-user-media-background-tab-expected.txt: Added.
     10        * fast/mediastream/get-user-media-background-tab.html: Added.
     11
    1122016-11-10  Jiewen Tan  <jiewen_tan@apple.com>
    213

  • trunk/Source/WebCore/ChangeLog

    r208605 r208606  
     12016-11-11  Eric Carlson  <eric.carlson@apple.com>
     2
     3        [MediaStream] defer resolution of getUserMedia promise made in a background tab
     4        https://bugs.webkit.org/show_bug.cgi?id=164643
     5        <rdar://problem/29048317>
     6
     7        Reviewed by Brady Eidson.
     8
     9        Test: fast/mediastream/get-user-media-background-tab.html
     10
     11        Do not start producing data when the document does not allow media
     12        to start playing. Instead, register with the document for a callback
     13        when playback is allowed and start then.
     14        * Modules/mediastream/MediaStream.cpp:
     15        (WebCore::MediaStream::~MediaStream):
     16        (WebCore::MediaStream::mediaCanStart):
     17        (WebCore::MediaStream::startProducingData):
     18        (WebCore::MediaStream::stopProducingData):
     19        (WebCore::MediaStream::pageMutedStateDidChange):
     20        * Modules/mediastream/MediaStream.h:
     21
     22        * Modules/webaudio/AudioContext.cpp:
     23        (WebCore::AudioContext::mediaCanStart): Deal with API change.
     24        * Modules/webaudio/AudioContext.h:
     25
     26        * html/HTMLMediaElement.cpp:
     27        (WebCore::HTMLMediaElement::mediaCanStart): Ditto.
     28        * html/HTMLMediaElement.h:
     29        * page/MediaCanStartListener.h:
     30
     31        * page/Page.cpp:
     32        (WebCore::Page::takeAnyMediaCanStartListener): Return the listener and document.
     33        (WebCore::Page::setCanStartMedia): Pass the document to the listener.
     34        * page/Page.h:
     35
    1362016-11-11  Zalan Bujtas  <zalan@apple.com>
    237

  • trunk/Source/WebCore/Modules/mediastream/MediaStream.cpp

    r207839 r208606  
    22 * Copyright (C) 2011 Google Inc. All rights reserved.
    33 * Copyright (C) 2011, 2012, 2015 Ericsson AB. All rights reserved.
    4  * Copyright (C) 2013-2015 Apple Inc. All rights reserved.
     4 * Copyright (C) 2013-2016 Apple Inc. All rights reserved.
    55 * Copyright (C) 2013 Nokia Corporation and/or its subsidiary(-ies).
    66 *
    … …  
    3535#include "EventNames.h"
    3636#include "ExceptionCode.h"
     37#include "Logging.h"
    3738#include "MediaStreamRegistry.h"
    3839#include "MediaStreamTrackEvent.h"
    … …  
    112113    for (auto& track : m_trackSet.values())
    113114        track->removeObserver(this);
    114     if (Document* document = this->document())
     115    if (Document* document = this->document()) {
    115116        document->removeAudioProducer(this);
     117        if (m_isWaitingUntilMediaCanStart)
     118            document->removeMediaCanStartListener(this);
     119    }
    116120}
    117121
    … …  
    248252}
    249253
     254void MediaStream::mediaCanStart(Document& document)
     255{
     256    ASSERT_UNUSED(document, &document == this->document());
     257    ASSERT(m_isWaitingUntilMediaCanStart);
     258    if (m_isWaitingUntilMediaCanStart) {
     259        m_isWaitingUntilMediaCanStart = false;
     260        startProducingData();
     261    }
     262}
     263
     264void MediaStream::startProducingData()
     265{
     266    Document* document = this->document();
     267    if (!document || !document->page())
     268        return;
     269
     270    // If we can't start a load right away, start it later.
     271    if (!document->page()->canStartMedia()) {
     272        LOG(Media, "MediaStream::startProducingData(%p) - not allowed to start in background, waiting", this);
     273        if (m_isWaitingUntilMediaCanStart)
     274            return;
     275
     276        m_isWaitingUntilMediaCanStart = true;
     277        document->addMediaCanStartListener(this);
     278        return;
     279    }
     280
     281    m_private->startProducingData();
     282}
     283
     284void MediaStream::stopProducingData()
     285{
     286    m_private->stopProducingData();
     287}
     288
    250289void MediaStream::pageMutedStateDidChange()
    251290{
    … …  
    263302    m_externallyMuted = pageMuted;
    264303    if (pageMuted)
    265         m_private->stopProducingData();
     304        stopProducingData();
    266305    else
    267         m_private->startProducingData();
     306        startProducingData();
    268307}
    269308

  • trunk/Source/WebCore/Modules/mediastream/MediaStream.h

    r200231 r208606  
    3434#include "EventTarget.h"
    3535#include "ExceptionBase.h"
     36#include "MediaCanStartListener.h"
    3637#include "MediaProducer.h"
    3738#include "MediaStreamPrivate.h"
    … …  
    4849class Document;
    4950
    50 class MediaStream final : public URLRegistrable, public EventTargetWithInlineData, public ContextDestructionObserver, public MediaStreamTrack::Observer, public MediaStreamPrivate::Observer, private MediaProducer, public RefCounted<MediaStream> {
     51class MediaStream final
     52    : public URLRegistrable
     53    , public EventTargetWithInlineData
     54    , public ContextDestructionObserver
     55    , public MediaStreamTrack::Observer
     56    , public MediaStreamPrivate::Observer
     57    , private MediaProducer
     58    , private MediaCanStartListener
     59    , public RefCounted<MediaStream> {
    5160public:
    5261    class Observer {
    … …  
    7887
    7988    MediaStreamPrivate* privateStream() const { return m_private.get(); }
     89
     90    void startProducingData();
     91    void stopProducingData();
    8092
    8193    // EventTarget
    … …  
    119131    MediaProducer::MediaStateFlags mediaState() const final;
    120132
     133    // MediaCanStartListener
     134    void mediaCanStart(Document&) final;
     135
    121136    bool internalAddTrack(Ref<MediaStreamTrack>&&, StreamModifier);
    122137    bool internalRemoveTrack(const String&, StreamModifier);
    … …  
    143158    bool m_isMuted { true };
    144159    bool m_externallyMuted { false };
     160    bool m_isWaitingUntilMediaCanStart { false };
    145161};
    146162

  • trunk/Source/WebCore/Modules/webaudio/AudioContext.cpp

    r208474 r208606  
    958958}
    959959
    960 void AudioContext::mediaCanStart()
    961 {
     960void AudioContext::mediaCanStart(Document& document)
     961{
     962    ASSERT_UNUSED(document, &document == this->document());
    962963    removeBehaviorRestriction(AudioContext::RequirePageConsentForAudioStartRestriction);
    963964    mayResumePlayback(true);

  • trunk/Source/WebCore/Modules/webaudio/AudioContext.h

    r208476 r208606  
    285285    void scheduleNodeDeletion();
    286286
    287     void mediaCanStart() override;
     287    void mediaCanStart(Document&) override;
    288288
    289289    // MediaProducer

  • trunk/Source/WebCore/html/HTMLMediaElement.cpp

    r208603 r208606  
    56655665#endif
    56665666
    5667 void HTMLMediaElement::mediaCanStart()
    5668 {
     5667void HTMLMediaElement::mediaCanStart(Document& document)
     5668{
     5669    ASSERT_UNUSED(document, &document == &this->document());
    56695670    LOG(Media, "HTMLMediaElement::mediaCanStart(%p) - m_isWaitingUntilMediaCanStart = %s, m_pausedInternal = %s",
    56705671        this, boolString(m_isWaitingUntilMediaCanStart), boolString(m_pausedInternal) );

  • trunk/Source/WebCore/html/HTMLMediaElement.h

    r208539 r208606  
    728728    void setPlaybackRateInternal(double);
    729729
    730     void mediaCanStart() override;
     730    void mediaCanStart(Document&) final;
    731731
    732732    void invalidateCachedTime() const;

  • trunk/Source/WebCore/page/MediaCanStartListener.h

    r208179 r208606  
    2828namespace WebCore {
    2929
     30class Document;
     31
    3032class MediaCanStartListener {
    3133public:
    32     virtual void mediaCanStart() = 0;
     34    virtual void mediaCanStart(Document&) = 0;
    3335protected:
    3436    virtual ~MediaCanStartListener() { }

  • trunk/Source/WebCore/page/Page.cpp

    r208479 r208606  
    556556}
    557557
    558 inline MediaCanStartListener* Page::takeAnyMediaCanStartListener()
     558inline Optional<std::pair<MediaCanStartListener&, Document&>> Page::takeAnyMediaCanStartListener()
    559559{
    560560    for (Frame* frame = &mainFrame(); frame; frame = frame->tree().traverseNext()) {
    … …  
    562562            continue;
    563563        if (MediaCanStartListener* listener = frame->document()->takeAnyMediaCanStartListener())
    564             return listener;
    565     }
    566     return 0;
     564            return { { *listener, *frame->document() } };
     565    }
     566    return Nullopt;
    567567}
    568568
    … …  
    575575
    576576    while (m_canStartMedia) {
    577         MediaCanStartListener* listener = takeAnyMediaCanStartListener();
     577        auto listener = takeAnyMediaCanStartListener();
    578578        if (!listener)
    579579            break;
    580         listener->mediaCanStart();
     580        listener->first.mediaCanStart(listener->second);
    581581    }
    582582}

  • trunk/Source/WebCore/page/Page.h

    r208361 r208606  
    550550    unsigned findMatchesForText(const String&, FindOptions, unsigned maxMatchCount, ShouldHighlightMatches, ShouldMarkMatches);
    551551
    552     MediaCanStartListener* takeAnyMediaCanStartListener();
     552    Optional<std::pair<MediaCanStartListener&, Document&>> takeAnyMediaCanStartListener();
    553553
    554554    Vector<Ref<PluginViewBase>> pluginViews();

  • trunk/Source/WebKit2/ChangeLog

    r208603 r208606  
     12016-11-11  Eric Carlson  <eric.carlson@apple.com>
     2
     3        [MediaStream] defer resolution of getUserMedia promise made in a background tab
     4        https://bugs.webkit.org/show_bug.cgi?id=164643
     5        <rdar://problem/29048317>
     6
     7        Reviewed by Brady Eidson.
     8
     9        Do not ask for user for access to capture devices when the document does not
     10        allow media to play, wait until playback is allowed.
     11
     12        * WebProcess/MediaStream/UserMediaPermissionRequestManager.cpp:
     13        (WebKit::UserMediaPermissionRequestManager::startUserMediaRequest):
     14        (WebKit::UserMediaPermissionRequestManager::sendUserMediaRequest):
     15        (WebKit::UserMediaPermissionRequestManager::mediaCanStart):
     16        (WebKit::UserMediaPermissionRequestManager::grantUserMediaDevicesSandboxExtension):
     17        * WebProcess/MediaStream/UserMediaPermissionRequestManager.h:
     18        * WebProcess/Plugins/PluginView.cpp:
     19        (WebKit::PluginView::mediaCanStart):
     20        * WebProcess/Plugins/PluginView.h:
     21
    1222016-11-11  Darin Adler  <darin@apple.com>
    223

  • trunk/Source/WebKit2/WebProcess/MediaStream/UserMediaPermissionRequestManager.cpp

    r208558 r208606  
    6161    Frame* frame = document ? document->frame() : nullptr;
    6262
     63    if (!frame || !document->page()) {
     64        request.deny(UserMediaRequest::OtherFailure, emptyString());
     65        return;
     66    }
     67
     68    if (document->page()->canStartMedia()) {
     69        sendUserMediaRequest(request);
     70        return;
     71    }
     72
     73    auto& pendingRequests = m_blockedRequests.add(document, Vector<RefPtr<UserMediaRequest>>()).iterator->value;
     74    if (pendingRequests.isEmpty())
     75        document->addMediaCanStartListener(this);
     76    pendingRequests.append(&request);
     77}
     78
     79void UserMediaPermissionRequestManager::sendUserMediaRequest(UserMediaRequest& request)
     80{
     81    Document* document = request.document();
     82    Frame* frame = document ? document->frame() : nullptr;
     83
    6384    if (!frame) {
    6485        request.deny(UserMediaRequest::OtherFailure, emptyString());
    … …  
    84105        return;
    85106    m_idToUserMediaRequestMap.remove(requestID);
     107    removeMediaRequestFromMaps(request);
     108}
     109
     110void UserMediaPermissionRequestManager::mediaCanStart(Document& document)
     111{
     112    auto pendingRequests = m_blockedRequests.take(&document);
     113    while (!pendingRequests.isEmpty()) {
     114        if (!document.page()->canStartMedia()) {
     115            m_blockedRequests.add(&document, pendingRequests);
     116            document.addMediaCanStartListener(this);
     117            break;
     118        }
     119
     120        sendUserMediaRequest(*pendingRequests.takeLast());
     121    }
     122}
     123
     124void UserMediaPermissionRequestManager::removeMediaRequestFromMaps(UserMediaRequest& request)
     125{
     126    auto pendingRequests = m_blockedRequests.take(request.document());
     127    for (auto& pendingRequest : pendingRequests) {
     128        if (&request != pendingRequest.get())
     129            continue;
     130
     131        if (pendingRequests.isEmpty())
     132            request.document()->removeMediaCanStartListener(this);
     133        else
     134            m_blockedRequests.add(request.document(), pendingRequests);
     135        break;
     136    }
     137
     138    m_userMediaRequestToIDMap.remove(&request);
    86139}
    87140
    … …  
    91144    if (!request)
    92145        return;
    93     m_userMediaRequestToIDMap.remove(request);
     146    removeMediaRequestFromMaps(*request);
    94147
    95148    request->allow(audioDeviceUID, videoDeviceUID);
    … …  
    101154    if (!request)
    102155        return;
    103     m_userMediaRequestToIDMap.remove(request);
     156    removeMediaRequestFromMaps(*request);
    104157
    105158    request->deny(reason, invalidConstraint);
    … …  
    151204
    152205    for (size_t i = 0; i < sandboxExtensionHandles.size(); i++) {
    153         if (RefPtr<SandboxExtension> extension = SandboxExtension::create(sandboxExtensionHandles[i])) {
     206        if (auto extension = SandboxExtension::create(sandboxExtensionHandles[i])) {
    154207            extension->consume();
    155208            m_userMediaDeviceSandboxExtensions.append(extension.release());

  • trunk/Source/WebKit2/WebProcess/MediaStream/UserMediaPermissionRequestManager.h

    r207704 r208606  
    2424
    2525#include "SandboxExtension.h"
     26#include <WebCore/MediaCanStartListener.h>
    2627#include <WebCore/MediaConstraints.h>
    2728#include <WebCore/MediaDevicesEnumerationRequest.h>
    … …  
    3637class WebPage;
    3738
    38 class UserMediaPermissionRequestManager {
     39class UserMediaPermissionRequestManager
     40    : private WebCore::MediaCanStartListener {
    3941public:
    4042    explicit UserMediaPermissionRequestManager(WebPage&);
    … …  
    5355
    5456private:
     57    void sendUserMediaRequest(WebCore::UserMediaRequest&);
     58
     59    // WebCore::MediaCanStartListener
     60    void mediaCanStart(WebCore::Document&) override;
     61
     62    void removeMediaRequestFromMaps(WebCore::UserMediaRequest&);
     63
    5564    WebPage& m_page;
    5665
    … …  
    6271
    6372    Vector<RefPtr<SandboxExtension>> m_userMediaDeviceSandboxExtensions;
     73
     74    HashMap<RefPtr<WebCore::Document>, Vector<RefPtr<WebCore::UserMediaRequest>>> m_blockedRequests;
    6475};
    6576

  • trunk/Source/WebKit2/WebProcess/Plugins/PluginView.cpp

    r208329 r208606  
    13721372}
    13731373
    1374 void PluginView::mediaCanStart()
     1374void PluginView::mediaCanStart(WebCore::Document&)
    13751375{
    13761376    ASSERT(m_isWaitingUntilMediaCanStart);

  • trunk/Source/WebKit2/WebProcess/Plugins/PluginView.h

    r208329 r208606  
    185185
    186186    // WebCore::MediaCanStartListener
    187     void mediaCanStart() override;
     187    void mediaCanStart(WebCore::Document&) override;
    188188
    189189    // WebCore::MediaProducer
Note: See TracChangeset for help on using the changeset viewer.