Changeset 211240 in webkit


Ignore:
Timestamp:
Jan 26, 2017 4:38:22 PM (7 years ago)
Author:
jer.noble@apple.com
Message:

Autoplay muted videos stop playback of any streaming app in the background
​https://bugs.webkit.org/show_bug.cgi?id=163993
<rdar://problem/29020431>

Reviewed by Eric Carlson.

Source/WebCore:

Added test in TestWebKitAPI, WebKit1.AudioSessionCategoryIOS.

Previously, we would set the audio session category to "playback" if there was a media-
element-type media session, and if there was a session capable of playing audio. But because
this was an "or" operation, we would incorrectly set the category to "playback" if there was
a video element incapable of rendering audio (due to being muted, without an audio track,
etc.), and also a session capable of producing audio, such as WebAudio.

With this change, this turns into an "and" operation; there must be a media element capable
of rendering audio in order to switch the audio session category to "playback".

Additionally, we no longer cache the value of "canProduceAudio()"; it's queried directly
whenever updateSessionState() is called.

  • Modules/webaudio/AudioContext.cpp:

(WebCore::AudioContext::constructCommon):

  • Modules/webaudio/AudioContext.h:
  • html/HTMLMediaElement.cpp:

(WebCore::HTMLMediaElement::insertedInto):
(WebCore::HTMLMediaElement::loadResource):
(WebCore::HTMLMediaElement::setMuted):
(WebCore::HTMLMediaElement::mediaPlayerCurrentPlaybackTargetIsWirelessChanged):
(WebCore::HTMLMediaElement::mediaPlayerCharacteristicChanged):
(WebCore::HTMLMediaElement::clearMediaPlayer):
(WebCore::HTMLMediaElement::mediaPlayerCurrentPlaybackTargetIsWirelessChanged):
(WebCore::HTMLMediaElement::presentationType):
(WebCore::HTMLMediaElement::characteristics):
(WebCore::HTMLMediaElement::canProduceAudio):

  • html/HTMLMediaElement.h:
  • platform/audio/PlatformMediaSession.cpp:

(WebCore::PlatformMediaSession::activeAudioSessionRequired):
(WebCore::PlatformMediaSession::canProduceAudio):
(WebCore::PlatformMediaSession::canProduceAudioChanged):
(WebCore::PlatformMediaSession::setCanProduceAudio): Deleted.

  • platform/audio/PlatformMediaSession.h:

(WebCore::PlatformMediaSessionClient::canProduceAudio):
(WebCore::PlatformMediaSession::canProduceAudio): Deleted.

  • platform/audio/cocoa/MediaSessionManagerCocoa.cpp:

(PlatformMediaSessionManager::updateSessionState):

Tools:

  • TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
  • TestWebKitAPI/Tests/WebKit/ios/AudioSessionCategoryIOS.mm:

(TestWebKitAPI::TEST):

  • TestWebKitAPI/Tests/WebKit/ios/video-with-muted-audio-and-webaudio.html: Added.
Location:
trunk
Files:
1 added
11 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r211238 r211240  
     12017-01-26  Jer Noble  <jer.noble@apple.com>
     2
     3        Autoplay muted videos stop playback of any streaming app in the background
     4        https://bugs.webkit.org/show_bug.cgi?id=163993
     5        <rdar://problem/29020431>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Added test in TestWebKitAPI, WebKit1.AudioSessionCategoryIOS.
     10
     11        Previously, we would set the audio session category to "playback" if there was a media-
     12        element-type media session, and if there was a session capable of playing audio. But because
     13        this was an "or" operation, we would incorrectly set the category to "playback" if there was
     14        a video element incapable of rendering audio (due to being muted, without an audio track,
     15        etc.), and also a session capable of producing audio, such as WebAudio.
     16
     17        With this change, this turns into an "and" operation; there must be a media element capable
     18        of rendering audio in order to switch the audio session category to "playback".
     19
     20        Additionally, we no longer cache the value of "canProduceAudio()"; it's queried directly
     21        whenever updateSessionState() is called.
     22
     23        * Modules/webaudio/AudioContext.cpp:
     24        (WebCore::AudioContext::constructCommon):
     25        * Modules/webaudio/AudioContext.h:
     26        * html/HTMLMediaElement.cpp:
     27        (WebCore::HTMLMediaElement::insertedInto):
     28        (WebCore::HTMLMediaElement::loadResource):
     29        (WebCore::HTMLMediaElement::setMuted):
     30        (WebCore::HTMLMediaElement::mediaPlayerCurrentPlaybackTargetIsWirelessChanged):
     31        (WebCore::HTMLMediaElement::mediaPlayerCharacteristicChanged):
     32        (WebCore::HTMLMediaElement::clearMediaPlayer):
     33        (WebCore::HTMLMediaElement::mediaPlayerCurrentPlaybackTargetIsWirelessChanged):
     34        (WebCore::HTMLMediaElement::presentationType):
     35        (WebCore::HTMLMediaElement::characteristics):
     36        (WebCore::HTMLMediaElement::canProduceAudio):
     37        * html/HTMLMediaElement.h:
     38        * platform/audio/PlatformMediaSession.cpp:
     39        (WebCore::PlatformMediaSession::activeAudioSessionRequired):
     40        (WebCore::PlatformMediaSession::canProduceAudio):
     41        (WebCore::PlatformMediaSession::canProduceAudioChanged):
     42        (WebCore::PlatformMediaSession::setCanProduceAudio): Deleted.
     43        * platform/audio/PlatformMediaSession.h:
     44        (WebCore::PlatformMediaSessionClient::canProduceAudio):
     45        (WebCore::PlatformMediaSession::canProduceAudio): Deleted.
     46        * platform/audio/cocoa/MediaSessionManagerCocoa.cpp:
     47        (PlatformMediaSessionManager::updateSessionState):
     48
    1492017-01-26  Filip Pizlo  <fpizlo@apple.com>
    250

  • trunk/Source/WebCore/Modules/webaudio/AudioContext.cpp

    r209390 r211240  
    182182    addBehaviorRestriction(RequirePageConsentForAudioStartRestriction);
    183183#endif
    184 
    185     m_mediaSession->setCanProduceAudio(true);
    186184}
    187185

  • trunk/Source/WebCore/Modules/webaudio/AudioContext.h

    r208606 r211240  
    318318    bool shouldOverrideBackgroundPlaybackRestriction(PlatformMediaSession::InterruptionType) const override { return false; }
    319319    String sourceApplicationIdentifier() const override;
     320    bool canProduceAudio() const final { return true; }
    320321
    321322    // EventTarget

  • trunk/Source/WebCore/html/HTMLMediaElement.cpp

    r211226 r211240  
    833833        m_explicitlyMuted = true;
    834834        m_muted = hasAttributeWithoutSynchronization(mutedAttr);
     835        m_mediaSession->canProduceAudioChanged();
    835836    }
    836837
    … …  
    14861487        m_explicitlyMuted = true;
    14871488        m_muted = hasAttributeWithoutSynchronization(mutedAttr);
     1489        m_mediaSession->canProduceAudioChanged();
    14881490    }
    14891491
    … …  
    33313333        updateMediaState(UpdateState::Asynchronously);
    33323334#endif
     3335        m_mediaSession->canProduceAudioChanged();
    33333336    }
    33343337
    … …  
    46644667        pauseInternal();
    46654668
    4666     m_mediaSession->setCanProduceAudio(m_player && m_readyState >= HAVE_METADATA && hasAudio());
    4667 
    46684669#if ENABLE(MEDIA_SESSION)
    46694670    document().updateIsPlayingMedia(m_elementID);
    … …  
    50835084#endif
    50845085
    5085     m_mediaSession->setCanProduceAudio(false);
    50865086    m_mediaSession->clientCharacteristicsChanged();
     5087    m_mediaSession->canProduceAudioChanged();
    50875088
    50885089    updateSleepDisabling();
    … …  
    52755276    scheduleEvent(eventNames().webkitcurrentplaybacktargetiswirelesschangedEvent);
    52765277    m_mediaSession->isPlayingToWirelessPlaybackTargetChanged(m_isPlayingToWirelessTarget);
    5277     if (m_isPlayingToWirelessTarget)
    5278         m_mediaSession->setCanProduceAudio(true);
     5278    m_mediaSession->canProduceAudioChanged();
    52795279    updateMediaState(UpdateState::Asynchronously);
    52805280}
    … …  
    68206820{
    68216821    if (hasTagName(HTMLNames::videoTag))
    6822         return PlatformMediaSession::VideoAudio;
     6822        return muted() ? PlatformMediaSession::Video : PlatformMediaSession::VideoAudio;
    68236823
    68246824    return PlatformMediaSession::Audio;
    … …  
    68506850
    68516851    return state;
     6852}
     6853
     6854bool HTMLMediaElement::canProduceAudio() const
     6855{
     6856#if ENABLE(WIRELESS_PLAYBACK_TARGET)
     6857    // Because the remote target could unmute playback without notifying us, we must assume
     6858    // that we may be playing audio.
     6859    if (m_isPlayingToWirelessTarget)
     6860        return true;
     6861#endif
     6862
     6863    if (muted())
     6864        return false;
     6865
     6866    return m_player && m_readyState >= HAVE_METADATA && hasAudio();
    68526867}
    68536868

  • trunk/Source/WebCore/html/HTMLMediaElement.h

    r211008 r211240  
    777777    bool shouldOverrideBackgroundPlaybackRestriction(PlatformMediaSession::InterruptionType) const override;
    778778    bool shouldOverrideBackgroundLoadingRestriction() const override;
     779    bool canProduceAudio() const final;
    779780
    780781    void pageMutedStateDidChange() override;

  • trunk/Source/WebCore/platform/audio/PlatformMediaSession.cpp

    r206193 r211240  
    330330    if (state() != PlatformMediaSession::State::Playing)
    331331        return false;
    332     return m_canProduceAudio;
    333 }
    334 
    335 void PlatformMediaSession::setCanProduceAudio(bool canProduceAudio)
    336 {
    337     if (m_canProduceAudio == canProduceAudio)
    338         return;
    339     m_canProduceAudio = canProduceAudio;
    340 
     332    return canProduceAudio();
     333}
     334
     335bool PlatformMediaSession::canProduceAudio() const
     336{
     337    return m_client.canProduceAudio();
     338}
     339
     340void PlatformMediaSession::canProduceAudioChanged()
     341{
    341342    PlatformMediaSessionManager::sharedManager().sessionCanProduceAudioChanged(*this);
    342343}

  • trunk/Source/WebCore/platform/audio/PlatformMediaSession.h

    r208149 r211240  
    163163
    164164    bool activeAudioSessionRequired();
    165     bool canProduceAudio() const { return m_canProduceAudio; }
    166     void setCanProduceAudio(bool);
     165    bool canProduceAudio() const;
     166    void canProduceAudioChanged();
    167167
    168168    void scheduleClientDataBufferingCheck();
    … …  
    187187    bool m_notifyingClient;
    188188    bool m_isPlayingToWirelessPlaybackTarget { false };
    189     bool m_canProduceAudio { false };
    190189
    191190    friend class PlatformMediaSessionManager;
    … …  
    218217    virtual void setShouldBufferData(bool) { }
    219218    virtual bool elementIsHidden() const { return false; }
     219    virtual bool canProduceAudio() const { return false; }
    220220
    221221    virtual bool shouldOverrideBackgroundPlaybackRestriction(PlatformMediaSession::InterruptionType) const = 0;

  • trunk/Source/WebCore/platform/audio/cocoa/MediaSessionManagerCocoa.cpp

    r210100 r211240  
    3232#include "Logging.h"
    3333#include "Settings.h"
     34#include <wtf/Function.h>
    3435
    3536using namespace WebCore;
    … …  
    6162        return;
    6263
    63     if (has(PlatformMediaSession::VideoAudio) || has(PlatformMediaSession::Audio)) {
    64         if (canProduceAudio())
    65             AudioSession::sharedSession().setCategory(AudioSession::MediaPlayback);
    66         else
    67             AudioSession::sharedSession().setCategory(AudioSession::AmbientSound);
    68     } else if (has(PlatformMediaSession::WebAudio))
     64    bool hasAudioMediaType = false;
     65    bool hasAudibleAudioOrVideoMediaType = anyOfSessions([this, hasAudioMediaType] (PlatformMediaSession& session, size_t) mutable {
     66        auto type = session.mediaType();
     67        if (type == PlatformMediaSession::VideoAudio || type == PlatformMediaSession::Audio || type == PlatformMediaSession::WebAudio)
     68            hasAudioMediaType = true;
     69        return (type == PlatformMediaSession::VideoAudio || type == PlatformMediaSession::Audio) && session.canProduceAudio();
     70    });
     71
     72    if (hasAudibleAudioOrVideoMediaType)
     73        AudioSession::sharedSession().setCategory(AudioSession::MediaPlayback);
     74    else if (hasAudioMediaType)
    6975        AudioSession::sharedSession().setCategory(AudioSession::AmbientSound);
     76    else
     77        AudioSession::sharedSession().setCategory(AudioSession::None);
    7078#endif
    7179}

  • trunk/Tools/ChangeLog

    r211232 r211240  
     12017-01-26  Jer Noble  <jer.noble@apple.com>
     2
     3        Autoplay muted videos stop playback of any streaming app in the background
     4        https://bugs.webkit.org/show_bug.cgi?id=163993
     5        <rdar://problem/29020431>
     6
     7        Reviewed by Eric Carlson.
     8
     9        * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
     10        * TestWebKitAPI/Tests/WebKit/ios/AudioSessionCategoryIOS.mm:
     11        (TestWebKitAPI::TEST):
     12        * TestWebKitAPI/Tests/WebKit/ios/video-with-muted-audio-and-webaudio.html: Added.
     13
    1142017-01-26  Ryan Haddad  <ryanhaddad@apple.com>
    215

  • trunk/Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj

    r211226 r211240  
    513513                C99B675D1E39722000FC6C80 /* js-play-with-controls.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = C99B675B1E3971FC00FC6C80 /* js-play-with-controls.html */; };
    514514                C99B675F1E39736F00FC6C80 /* no-autoplay-with-controls.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = C99B675E1E39735C00FC6C80 /* no-autoplay-with-controls.html */; };
     515                CD321B041E3A85FA00EB21C8 /* video-with-muted-audio-and-webaudio.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = CD321B031E3A84B700EB21C8 /* video-with-muted-audio-and-webaudio.html */; };
    515516                CD59F53419E9110D00CF1835 /* file-with-mse.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = CD59F53219E910AA00CF1835 /* file-with-mse.html */; };
    516517                CD59F53519E9110D00CF1835 /* test-mse.mp4 in Copy Resources */ = {isa = PBXBuildFile; fileRef = CD59F53319E910BC00CF1835 /* test-mse.mp4 */; };
    … …  
    614615                                C99B675D1E39722000FC6C80 /* js-play-with-controls.html in Copy Resources */,
    615616                                C99B675C1E39721A00FC6C80 /* autoplay-with-controls.html in Copy Resources */,
     617                                CD321B041E3A85FA00EB21C8 /* video-with-muted-audio-and-webaudio.html in Copy Resources */,
    616618                                C95984F71E36BCEF002C0D45 /* test-without-audio-track.mp4 in Copy Resources */,
    617619                                C95984F41E36BC6B002C0D45 /* autoplay-check.html in Copy Resources */,
    … …  
    12851287                C99B675E1E39735C00FC6C80 /* no-autoplay-with-controls.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "no-autoplay-with-controls.html"; sourceTree = "<group>"; };
    12861288                CD225C071C45A69200140761 /* ParsedContentRange.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ParsedContentRange.cpp; sourceTree = "<group>"; };
     1289                CD321B031E3A84B700EB21C8 /* video-with-muted-audio-and-webaudio.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "video-with-muted-audio-and-webaudio.html"; sourceTree = "<group>"; };
    12871290                CD5393C71757BA9700C07123 /* MD5.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = MD5.cpp; sourceTree = "<group>"; };
    12881291                CD5393C91757BAC400C07123 /* SHA1.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SHA1.cpp; sourceTree = "<group>"; };
    … …  
    22362239                                CDC8E48C1BC5C96200594FEC /* video-without-audio.mp4 */,
    22372240                                CDB411591E09DA8E00EAD352 /* video-with-muted-audio.html */,
     2241                                CD321B031E3A84B700EB21C8 /* video-with-muted-audio-and-webaudio.html */,
    22382242                        );
    22392243                        name = Resources;

  • trunk/Tools/TestWebKitAPI/Tests/WebKit/ios/AudioSessionCategoryIOS.mm

    r210100 r211240  
    9898
    9999    EXPECT_WK_STREQ(getAVAudioSessionCategoryAmbient(), [[getAVAudioSessionClass() sharedInstance] category]);
     100
     101    didBeginPlaying = false;
     102
     103    [uiWebView loadRequest:[NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"video-with-muted-audio-and-webaudio" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]]];
     104
     105    Util::run(&didBeginPlaying);
     106
     107    EXPECT_WK_STREQ(getAVAudioSessionCategoryAmbient(), [[getAVAudioSessionClass() sharedInstance] category]);
    100108}
    101109
Note: See TracChangeset for help on using the changeset viewer.