Changeset 214721 in webkit


Ignore:
Timestamp:
Apr 1, 2017 5:55:20 PM (7 years ago)
Author:
Chris Dumez
Message:

We should pause silent WebAudio rendering in background tabs
​https://bugs.webkit.org/show_bug.cgi?id=170299
<rdar://problem/31289132>

Reviewed by Eric Carlson.

Source/WebCore:

We should pause silent WebAudio rendering in background tabs since it uses CPU and is
not observable by the user. Such silent WebAudio rendering seems to be used by
doubleclick ads.

Test: webaudio/silent-audio-interrupted-in-background.html

  • Modules/webaudio/AudioContext.cpp:

(WebCore::AudioContext::lazyInitialize):
(WebCore::AudioContext::uninitialize):
Have AudioContext register / unregister itself with the Document to get
visibility change notifications, similarly to what HTMLMediaElement was
already doing.

(WebCore::AudioContext::visibilityStateChanged):
Begin / End session interruption whenever the page visiblity changes.

  • Modules/webaudio/AudioContext.h:
  • WebCore.xcodeproj/project.pbxproj:
  • dom/Document.cpp:

(WebCore::Document::registerForVisibilityStateChangedCallbacks):
(WebCore::Document::unregisterForVisibilityStateChangedCallbacks):
(WebCore::Document::visibilityStateChanged):

  • dom/Document.h:
  • dom/Element.h:
  • dom/VisibilityChangeClient.h: Added.

(WebCore::VisibilityChangeClient::~VisibilityChangeClient):

  • html/HTMLMediaElement.h:

Introduce a new VisibilityChangeClient interface and have both AudioContext
and HTMLMediaElement subclass it. Previously, the visibilityStateChanged()
function was on Element but this prevented AudioContext from registering
itself since AudioContext is not an Element.

LayoutTests:

Add layout test coverage.

  • webaudio/silent-audio-interrupted-in-background-expected.txt: Added.
  • webaudio/silent-audio-interrupted-in-background.html: Added.
Location:
trunk
Files:
3 added
10 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r214713 r214721  
     12017-04-01  Chris Dumez  <cdumez@apple.com>
     2
     3        We should pause silent WebAudio rendering in background tabs
     4        https://bugs.webkit.org/show_bug.cgi?id=170299
     5        <rdar://problem/31289132>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Add layout test coverage.
     10
     11        * webaudio/silent-audio-interrupted-in-background-expected.txt: Added.
     12        * webaudio/silent-audio-interrupted-in-background.html: Added.
     13
    1142017-04-01  Alexey Proskuryakov  <ap@apple.com>
    215

  • trunk/LayoutTests/platform/ios/TestExpectations

    r214561 r214721  
    312312webkit.org/b/165799 svg/animations/animations-paused-in-background-page-iframe.html [ Skip ]
    313313webkit.org/b/165799 svg/animations/animations-paused-in-background-page.html [ Skip ]
     314webkit.org/b/165799 webaudio/silent-audio-interrupted-in-background.html [ Skip ]
    314315
    315316# AutoFill button is not supported

  • trunk/Source/WebCore/ChangeLog

    r214720 r214721  
     12017-04-01  Chris Dumez  <cdumez@apple.com>
     2
     3        We should pause silent WebAudio rendering in background tabs
     4        https://bugs.webkit.org/show_bug.cgi?id=170299
     5        <rdar://problem/31289132>
     6
     7        Reviewed by Eric Carlson.
     8
     9        We should pause silent WebAudio rendering in background tabs since it uses CPU and is
     10        not observable by the user. Such silent WebAudio rendering seems to be used by
     11        doubleclick ads.
     12
     13        Test: webaudio/silent-audio-interrupted-in-background.html
     14
     15        * Modules/webaudio/AudioContext.cpp:
     16        (WebCore::AudioContext::lazyInitialize):
     17        (WebCore::AudioContext::uninitialize):
     18        Have AudioContext register / unregister itself with the Document to get
     19        visibility change notifications, similarly to what HTMLMediaElement was
     20        already doing.
     21
     22        (WebCore::AudioContext::visibilityStateChanged):
     23        Begin / End session interruption whenever the page visiblity changes.
     24
     25        * Modules/webaudio/AudioContext.h:
     26        * WebCore.xcodeproj/project.pbxproj:
     27
     28        * dom/Document.cpp:
     29        (WebCore::Document::registerForVisibilityStateChangedCallbacks):
     30        (WebCore::Document::unregisterForVisibilityStateChangedCallbacks):
     31        (WebCore::Document::visibilityStateChanged):
     32        * dom/Document.h:
     33        * dom/Element.h:
     34        * dom/VisibilityChangeClient.h: Added.
     35        (WebCore::VisibilityChangeClient::~VisibilityChangeClient):
     36        * html/HTMLMediaElement.h:
     37        Introduce a new VisibilityChangeClient interface and have both AudioContext
     38        and HTMLMediaElement subclass it. Previously, the visibilityStateChanged()
     39        function was on Element but this prevented AudioContext from registering
     40        itself since AudioContext is not an Element.
     41
    1422017-04-01  Dan Bernstein  <mitz@apple.com>
    243

  • trunk/Source/WebCore/Modules/webaudio/AudioContext.cpp

    r211964 r214721  
    5555#include "HRTFPanner.h"
    5656#include "JSDOMPromise.h"
     57#include "Logging.h"
    5758#include "NetworkingContext.h"
    5859#include "OfflineAudioCompletionEvent.h"
    … …  
    105106
    106107namespace WebCore {
     108
     109#define RELEASE_LOG_IF_ALLOWED(fmt, ...) RELEASE_LOG_IF(document()->page() && document()->page()->isAlwaysOnLoggingAllowed(), Media, "%p - AudioContext::" fmt, this, ##__VA_ARGS__)
    107110   
    108111bool AudioContext::isSampleRateRangeGood(float sampleRate)
    … …  
    216219        if (!isOfflineContext()) {
    217220            document()->addAudioProducer(this);
     221            document()->registerForVisibilityStateChangedCallbacks(this);
    218222
    219223            // This starts the audio thread. The destination node's provideInput() method will now be called repeatedly to render audio.
    … …  
    260264    if (!isOfflineContext()) {
    261265        document()->removeAudioProducer(this);
     266        document()->unregisterForVisibilityStateChangedCallbacks(this);
    262267
    263268        ASSERT(s_hardwareContextCount);
    … …  
    362367    }
    363368    return emptyString();
     369}
     370
     371void AudioContext::visibilityStateChanged()
     372{
     373    // Do not suspend if audio is audible.
     374    if (mediaState() == MediaProducer::IsPlayingAudio)
     375        return;
     376
     377    if (document()->hidden()) {
     378        if (state() == State::Running) {
     379            RELEASE_LOG_IF_ALLOWED("visibilityStateChanged() Suspending playback after going to the background");
     380            m_mediaSession->beginInterruption(PlatformMediaSession::EnteringBackground);
     381        }
     382    } else {
     383        if (state() == State::Interrupted) {
     384            RELEASE_LOG_IF_ALLOWED("visibilityStateChanged() Resuming playback after entering foreground");
     385            m_mediaSession->endInterruption(PlatformMediaSession::MayResumePlaying);
     386        }
     387    }
    364388}
    365389

  • trunk/Source/WebCore/Modules/webaudio/AudioContext.h

    r211240 r214721  
    3636#include "MediaProducer.h"
    3737#include "PlatformMediaSession.h"
     38#include "VisibilityChangeClient.h"
    3839#include <atomic>
    3940#include <wtf/HashSet.h>
    … …  
    7677// For thread safety between the audio thread and the main thread, it has a rendering graph locking mechanism.
    7778
    78 class AudioContext : public ActiveDOMObject, public ThreadSafeRefCounted<AudioContext>, public EventTargetWithInlineData, public MediaCanStartListener, public MediaProducer, private PlatformMediaSessionClient {
     79class AudioContext : public ActiveDOMObject, public ThreadSafeRefCounted<AudioContext>, public EventTargetWithInlineData, public MediaCanStartListener, public MediaProducer, private PlatformMediaSessionClient, private VisibilityChangeClient {
    7980public:
    8081    // Create an AudioContext for rendering to the audio hardware.
    … …  
    320321    bool canProduceAudio() const final { return true; }
    321322
     323    void visibilityStateChanged() final;
     324
    322325    // EventTarget
    323326    void refEventTarget() override { ref(); }

  • trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj

    r214680 r214721  
    32833283                832B843419D8E55100B26055 /* SVGAnimateElementBase.h in Headers */ = {isa = PBXBuildFile; fileRef = 832B843319D8E55100B26055 /* SVGAnimateElementBase.h */; };
    32843284                832B843619D8E57400B26055 /* SVGAnimateElementBase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 832B843519D8E57400B26055 /* SVGAnimateElementBase.cpp */; };
     3285                83407FC11E8D9C1700E048D3 /* VisibilityChangeClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 83407FC01E8D9C1200E048D3 /* VisibilityChangeClient.h */; settings = {ATTRIBUTES = (Private, ); }; };
    32853286                834476EE1DA5BC57002B6ED2 /* JSScrollToOptions.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 83E9B3001DA5A51E00FFE8F6 /* JSScrollToOptions.cpp */; };
    32863287                834476EF1DA5BC5E002B6ED2 /* JSScrollToOptions.h in Headers */ = {isa = PBXBuildFile; fileRef = 83E9B3011DA5A51E00FFE8F6 /* JSScrollToOptions.h */; };
    … …  
    1126211263                832B843319D8E55100B26055 /* SVGAnimateElementBase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SVGAnimateElementBase.h; sourceTree = "<group>"; };
    1126311264                832B843519D8E57400B26055 /* SVGAnimateElementBase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SVGAnimateElementBase.cpp; sourceTree = "<group>"; };
     11265                83407FC01E8D9C1200E048D3 /* VisibilityChangeClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = VisibilityChangeClient.h; sourceTree = "<group>"; };
    1126411266                8348BFA91B85729500912F36 /* ClassCollection.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ClassCollection.cpp; sourceTree = "<group>"; };
    1126511267                8348BFAA1B85729500912F36 /* ClassCollection.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ClassCollection.h; sourceTree = "<group>"; };
    … …  
    2568825690                                CEF418CC1179678C009D112C /* ViewportArguments.cpp */,
    2568925691                                CEF418CD1179678C009D112C /* ViewportArguments.h */,
     25692                                83407FC01E8D9C1200E048D3 /* VisibilityChangeClient.h */,
    2569025693                                419BC2DC1685329900D64D6D /* VisitedLinkState.cpp */,
    2569125694                                419BC2DD1685329900D64D6D /* VisitedLinkState.h */,
    … …  
    2964229645                                3FFFF9AE159D9B060020BBD5 /* ViewportStyleResolver.h in Headers */,
    2964329646                                93309E20099E64920056E581 /* VisiblePosition.h in Headers */,
     29647                                83407FC11E8D9C1700E048D3 /* VisibilityChangeClient.h in Headers */,
    2964429648                                A883DF280F3D045D00F19BF6 /* VisibleSelection.h in Headers */,
    2964529649                                93309E1E099E64920056E581 /* VisibleUnits.h in Headers */,

  • trunk/Source/WebCore/dom/Document.cpp

    r214659 r214721  
    15311531}
    15321532
    1533 void Document::registerForVisibilityStateChangedCallbacks(Element* element)
    1534 {
    1535     m_visibilityStateCallbackElements.add(element);
    1536 }
    1537 
    1538 void Document::unregisterForVisibilityStateChangedCallbacks(Element* element)
    1539 {
    1540     m_visibilityStateCallbackElements.remove(element);
     1533void Document::registerForVisibilityStateChangedCallbacks(VisibilityChangeClient* client)
     1534{
     1535    m_visibilityStateCallbackClients.add(client);
     1536}
     1537
     1538void Document::unregisterForVisibilityStateChangedCallbacks(VisibilityChangeClient* client)
     1539{
     1540    m_visibilityStateCallbackClients.remove(client);
    15411541}
    15421542
    … …  
    15441544{
    15451545    dispatchEvent(Event::create(eventNames().visibilitychangeEvent, false, false));
    1546     for (auto* element : m_visibilityStateCallbackElements)
    1547         element->visibilityStateChanged();
     1546    for (auto* client : m_visibilityStateCallbackClients)
     1547        client->visibilityStateChanged();
    15481548}
    15491549

  • trunk/Source/WebCore/dom/Document.h

    r214702 r214721  
    164164class TextResourceDecoder;
    165165class TreeWalker;
     166class VisibilityChangeClient;
    166167class VisitedLinkState;
    167168class XPathEvaluator;
    … …  
    10071008#endif
    10081009
    1009     void registerForVisibilityStateChangedCallbacks(Element*);
    1010     void unregisterForVisibilityStateChangedCallbacks(Element*);
     1010    void registerForVisibilityStateChangedCallbacks(VisibilityChangeClient*);
     1011    void unregisterForVisibilityStateChangedCallbacks(VisibilityChangeClient*);
    10111012
    10121013#if ENABLE(VIDEO)
    … …  
    15211522#endif
    15221523
    1523     HashSet<Element*> m_visibilityStateCallbackElements;
     1524    HashSet<VisibilityChangeClient*> m_visibilityStateCallbackClients;
    15241525#if ENABLE(VIDEO)
    15251526    HashSet<HTMLMediaElement*> m_allowsMediaDocumentInlinePlaybackElements;

  • trunk/Source/WebCore/dom/Element.h

    r214510 r214721  
    401401    virtual void willStopBeingFullscreenElement() { }
    402402
    403     // Use Document::registerForVisibilityStateChangedCallbacks() to subscribe to this.
    404     virtual void visibilityStateChanged() { }
    405 
    406403#if ENABLE(VIDEO_TRACK)
    407404    virtual void captionPreferencesChanged() { }

  • trunk/Source/WebCore/html/HTMLMediaElement.h

    r214443 r214721  
    3838#include "MediaProducer.h"
    3939#include "UserInterfaceLayoutDirection.h"
     40#include "VisibilityChangeClient.h"
    4041
    4142#if ENABLE(VIDEO_TRACK)
    … …  
    112113    , private MediaPlayerClient
    113114    , private MediaProducer
     115    , private VisibilityChangeClient
    114116#if ENABLE(VIDEO_TRACK)
    115117    , private AudioTrackClient
Note: See TracChangeset for help on using the changeset viewer.