Changeset 205412 in webkit


Ignore:
Timestamp:
Sep 3, 2016 5:24:14 PM (8 years ago)
Author:
Wenson Hsieh
Message:

Refactor the heuristic for showing media controls to take all media sessions into account
https://bugs.webkit.org/show_bug.cgi?id=161503
<rdar://problem/28033783>

Reviewed by Darin Adler.

Source/WebCore:

Currently, when selecting a media session to show playback controls for, we grab the first media session that
passes our heuristic. Using this method, we are unable to take additional factors into account, such as whether
another media session's element is scrolled in view, or if another media session has been interacted with more
recently. To address this, we make the following changes:

  1. Consider the list of all MediaElementSessions.
  1. Select only the MediaElementSessions capable of showing media controls and sort the list by a special

heuristic that takes visibility and time of last user interaction into account. The first element on
this list is the strongest candidate for main content.

  1. If this strongest candidate is visible in the viewport, or it is playing with audio, we return this

as the chosen candidate. Otherwise, we return this session only if no other non-candidate video could be
confused as the main content (i.e. the non-candidate video is not only visible in the viewport, but also
large enough to be considered main content).

Using this new method of determining the video to show controls for, we retain previous behavior for pages with
a single video. On pages with multiple videos, the above logic ensures that if the current controlled video is
paused, scrolled out of view, and then a new video is scrolled into view, we will either hide media controls to
avoid confusion if that video could be confused for main content (using the mechanism in step 3), or we
hook up the media controls to the new video if it satisfies main content (using the mechanism in step 2).

This patch also adds 6 new TestWebKitAPI unit tests.

  • html/HTMLMediaElement.cpp:

(WebCore::mediaElementSessionInfoForSession):
(WebCore::preferMediaControlsForCandidateSessionOverOtherCandidateSession):
(WebCore::mediaSessionMayBeConfusedWithMainContent):
(WebCore::bestMediaSessionForShowingPlaybackControlsManager):
(WebCore::HTMLMediaElement::didAttachRenderers):
(WebCore::HTMLMediaElement::layoutSizeChanged):
(WebCore::HTMLMediaElement::isVisibleInViewportChanged):
(WebCore::HTMLMediaElement::resetPlaybackSessionState):
(WebCore::HTMLMediaElement::isVisibleInViewport):
(WebCore::HTMLMediaElement::updatePlaybackControlsManager):

  • html/HTMLMediaElement.h:
  • html/MediaElementSession.cpp:

(WebCore::MediaElementSession::removeBehaviorRestriction):
(WebCore::MediaElementSession::canShowControlsManager):
(WebCore::MediaElementSession::isLargeEnoughForMainContent):
(WebCore::MediaElementSession::mostRecentUserInteractionTime):
(WebCore::MediaElementSession::wantsToObserveViewportVisibilityForMediaControls):
(WebCore::MediaElementSession::wantsToObserveViewportVisibilityForAutoplay):
(WebCore::MediaElementSession::resetPlaybackSessionState):
(WebCore::MediaElementSession::canControlControlsManager): Deleted.

  • html/MediaElementSession.h:
  • platform/audio/PlatformMediaSession.h:

(WebCore::PlatformMediaSession::resetPlaybackSessionState):
(WebCore::PlatformMediaSession::canControlControlsManager): Deleted.

  • platform/audio/PlatformMediaSessionManager.cpp:

(WebCore::PlatformMediaSessionManager::currentSessionsMatching):
(WebCore::PlatformMediaSessionManager::currentSessionMatching): Deleted.

  • platform/audio/PlatformMediaSessionManager.h:
  • platform/cocoa/WebPlaybackSessionModelMediaElement.mm:

(WebPlaybackSessionModelMediaElement::setMediaElement):

Source/WebKit2:

Adds an SPI testing hook for sending the element ID of the currently controlled video element from the web
process to the UI process. See VideoControlsManager.mm in Tools/TestWebKitAPI/ for usage.

  • UIProcess/API/Cocoa/WKWebView.mm:

(-[WKWebView _requestControlledElementID]):
(-[WKWebView _handleControlledElementIDResponse:]):
(-[WKWebView _hasActiveVideoForControlsManager]): Deleted.

  • UIProcess/API/Cocoa/WKWebViewPrivate.h:
  • UIProcess/Cocoa/WebPlaybackSessionManagerProxy.h:
  • UIProcess/Cocoa/WebPlaybackSessionManagerProxy.messages.in:
  • UIProcess/Cocoa/WebPlaybackSessionManagerProxy.mm:

(WebKit::WebPlaybackSessionManagerProxy::handleControlledElementIDResponse):
(WebKit::WebPlaybackSessionManagerProxy::requestControlledElementID):

  • UIProcess/PageClient.h:
  • UIProcess/WebPageProxy.cpp:

(WebKit::WebPageProxy::requestControlledElementID):
(WebKit::WebPageProxy::handleControlledElementIDResponse):

  • UIProcess/WebPageProxy.h:
  • UIProcess/mac/PageClientImpl.h:
  • UIProcess/mac/PageClientImpl.mm:

(WebKit::PageClientImpl::handleControlledElementIDResponse):

  • WebProcess/cocoa/WebPlaybackSessionManager.h:
  • WebProcess/cocoa/WebPlaybackSessionManager.messages.in:
  • WebProcess/cocoa/WebPlaybackSessionManager.mm:

(WebKit::WebPlaybackSessionManager::handleControlledElementIDRequest):

Tools:

Adds new unit tests verifying the behavior of media playback controls when scrolling another video into view.
Please see the WebCore ChangeLog for more details about this change. Also refactors existing
VideoControlsManager tests by folding duplicated setup and testing logic into helper methods to make the unit
tests more readable.

  • TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
  • TestWebKitAPI/Tests/WebKit2Cocoa/VideoControlsManager.mm:

(-[MessageHandler initWithMessage:handler:]):
(-[MessageHandler userContentController:didReceiveScriptMessage:]):
(-[VideoControlsManagerTestWebView performAfterLoading:]):
(-[VideoControlsManagerTestWebView loadTestPageNamed:]):
(-[VideoControlsManagerTestWebView loadTestPageNamed:andExpectControlsManager:afterReceivingMessage:]):
(-[VideoControlsManagerTestWebView performAfterReceivingMessage:action:]):
(-[VideoControlsManagerTestWebView controlledElementID]):
(-[VideoControlsManagerTestWebView _handleControlledElementIDResponse:]):
(TestWebKitAPI::setUpWebViewForTestingVideoControlsManager):
(TestWebKitAPI::TEST):
(-[MediaPlaybackMessageHandler initWithWKWebView:finalMessageString:]): Deleted.
(-[MediaPlaybackMessageHandler userContentController:didReceiveScriptMessage:]): Deleted.
(-[OnLoadMessageHandler initWithWKWebView:handler:]): Deleted.
(-[OnLoadMessageHandler userContentController:didReceiveScriptMessage:]): Deleted.
(-[WKWebView performAfterLoading:]): Deleted.

  • TestWebKitAPI/Tests/WebKit2Cocoa/large-video-playing-scroll-away.html: Added.
  • TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-autoplaying-click-to-pause.html: Added.
  • TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-autoplaying-scroll-to-video.html: Added.
  • TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-paused-video-hides-controls.html: Added.
  • TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-playing-muted-video-hides-controls.html: Added.
  • TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-playing-video-keeps-controls.html: Added.
Location:
trunk
Files:
6 added
26 edited

Legend:

Unmodified
Added
Removed
  • trunk/Source/WebCore/ChangeLog

    r205411 r205412  
     12016-09-03  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        Refactor the heuristic for showing media controls to take all media sessions into account
     4        https://bugs.webkit.org/show_bug.cgi?id=161503
     5        <rdar://problem/28033783>
     6
     7        Reviewed by Darin Adler.
     8
     9        Currently, when selecting a media session to show playback controls for, we grab the first media session that
     10        passes our heuristic. Using this method, we are unable to take additional factors into account, such as whether
     11        another media session's element is scrolled in view, or if another media session has been interacted with more
     12        recently. To address this, we make the following changes:
     13
     14            1.  Consider the list of all MediaElementSessions.
     15
     16            2.  Select only the MediaElementSessions capable of showing media controls and sort the list by a special
     17                heuristic that takes visibility and time of last user interaction into account. The first element on
     18                this list is the strongest candidate for main content.
     19
     20            3.  If this strongest candidate is visible in the viewport, or it is playing with audio, we return this
     21                as the chosen candidate. Otherwise, we return this session only if no other non-candidate video could be
     22                confused as the main content (i.e. the non-candidate video is not only visible in the viewport, but also
     23                large enough to be considered main content).
     24
     25        Using this new method of determining the video to show controls for, we retain previous behavior for pages with
     26        a single video. On pages with multiple videos, the above logic ensures that if the current controlled video is
     27        paused, scrolled out of view, and then a new video is scrolled into view, we will either hide media controls to
     28        avoid confusion if that video could be confused for main content (using the mechanism in step 3), or we
     29        hook up the media controls to the new video if it satisfies main content (using the mechanism in step 2).
     30
     31        This patch also adds 6 new TestWebKitAPI unit tests.
     32
     33        * html/HTMLMediaElement.cpp:
     34        (WebCore::mediaElementSessionInfoForSession):
     35        (WebCore::preferMediaControlsForCandidateSessionOverOtherCandidateSession):
     36        (WebCore::mediaSessionMayBeConfusedWithMainContent):
     37        (WebCore::bestMediaSessionForShowingPlaybackControlsManager):
     38        (WebCore::HTMLMediaElement::didAttachRenderers):
     39        (WebCore::HTMLMediaElement::layoutSizeChanged):
     40        (WebCore::HTMLMediaElement::isVisibleInViewportChanged):
     41        (WebCore::HTMLMediaElement::resetPlaybackSessionState):
     42        (WebCore::HTMLMediaElement::isVisibleInViewport):
     43        (WebCore::HTMLMediaElement::updatePlaybackControlsManager):
     44        * html/HTMLMediaElement.h:
     45        * html/MediaElementSession.cpp:
     46        (WebCore::MediaElementSession::removeBehaviorRestriction):
     47        (WebCore::MediaElementSession::canShowControlsManager):
     48        (WebCore::MediaElementSession::isLargeEnoughForMainContent):
     49        (WebCore::MediaElementSession::mostRecentUserInteractionTime):
     50        (WebCore::MediaElementSession::wantsToObserveViewportVisibilityForMediaControls):
     51        (WebCore::MediaElementSession::wantsToObserveViewportVisibilityForAutoplay):
     52        (WebCore::MediaElementSession::resetPlaybackSessionState):
     53        (WebCore::MediaElementSession::canControlControlsManager): Deleted.
     54        * html/MediaElementSession.h:
     55        * platform/audio/PlatformMediaSession.h:
     56        (WebCore::PlatformMediaSession::resetPlaybackSessionState):
     57        (WebCore::PlatformMediaSession::canControlControlsManager): Deleted.
     58        * platform/audio/PlatformMediaSessionManager.cpp:
     59        (WebCore::PlatformMediaSessionManager::currentSessionsMatching):
     60        (WebCore::PlatformMediaSessionManager::currentSessionMatching): Deleted.
     61        * platform/audio/PlatformMediaSessionManager.h:
     62        * platform/cocoa/WebPlaybackSessionModelMediaElement.mm:
     63        (WebPlaybackSessionModelMediaElement::setMediaElement):
     64
    1652016-09-03  Darin Adler  <darin@apple.com>
    266
  • trunk/Source/WebCore/html/HTMLMediaElement.cpp

    r205348 r205412  
    347347#endif
    348348
     349struct MediaElementSessionInfo {
     350    const MediaElementSession* session;
     351
     352    double timeOfLastUserInteraction;
     353    bool canShowControlsManager : 1;
     354    bool isVisibleInViewportOrFullscreen : 1;
     355    bool isLargeEnoughForMainContent : 1;
     356    bool isPlayingAudio : 1;
     357};
     358
     359static MediaElementSessionInfo mediaElementSessionInfoForSession(const MediaElementSession& session)
     360{
     361    const HTMLMediaElement& element = session.element();
     362    return {
     363        &session,
     364        session.mostRecentUserInteractionTime(),
     365        session.canShowControlsManager(),
     366        element.isFullscreen() || element.isVisibleInViewport(),
     367        session.isLargeEnoughForMainContent(),
     368        element.isPlaying() && element.hasAudio() && !element.muted()
     369    };
     370}
     371
     372static bool preferMediaControlsForCandidateSessionOverOtherCandidateSession(const MediaElementSessionInfo& session, const MediaElementSessionInfo& otherSession)
     373{
     374    // Prioritize visible media over offscreen media.
     375    if (session.isVisibleInViewportOrFullscreen != otherSession.isVisibleInViewportOrFullscreen)
     376        return session.isVisibleInViewportOrFullscreen;
     377
     378    // As a tiebreaker, prioritize elements that the user recently interacted with.
     379    return session.timeOfLastUserInteraction > otherSession.timeOfLastUserInteraction;
     380}
     381
     382static bool mediaSessionMayBeConfusedWithMainContent(const MediaElementSessionInfo& session)
     383{
     384    if (!session.isVisibleInViewportOrFullscreen)
     385        return false;
     386
     387    if (!session.isLargeEnoughForMainContent)
     388        return false;
     389
     390    // Even if this video is not a candidate, if it is visible to the user and large enough
     391    // to be main content, it poses a risk for being confused with main content.
     392    return true;
     393}
     394
     395static const MediaElementSession* bestMediaSessionForShowingPlaybackControlsManager()
     396{
     397    auto allSessions = PlatformMediaSessionManager::sharedManager().currentSessionsMatching([] (const PlatformMediaSession& session) {
     398        return is<MediaElementSession>(session);
     399    });
     400
     401    Vector<MediaElementSessionInfo> candidateSessions;
     402    bool atLeastOneNonCandidateMayBeConfusedForMainContent = false;
     403    for (auto& session : allSessions) {
     404        auto mediaElementSessionInfo = mediaElementSessionInfoForSession(downcast<MediaElementSession>(*session));
     405        if (mediaElementSessionInfo.canShowControlsManager)
     406            candidateSessions.append(mediaElementSessionInfo);
     407        else if (mediaSessionMayBeConfusedWithMainContent(mediaElementSessionInfo))
     408            atLeastOneNonCandidateMayBeConfusedForMainContent = true;
     409    }
     410
     411    if (!candidateSessions.size())
     412        return nullptr;
     413
     414    std::sort(candidateSessions.begin(), candidateSessions.end(), preferMediaControlsForCandidateSessionOverOtherCandidateSession);
     415    auto strongestSessionCandidate = candidateSessions.first();
     416    if (!strongestSessionCandidate.isVisibleInViewportOrFullscreen && !strongestSessionCandidate.isPlayingAudio && atLeastOneNonCandidateMayBeConfusedForMainContent)
     417        return nullptr;
     418
     419    return strongestSessionCandidate.session;
     420}
     421
    349422HTMLMediaElement::HTMLMediaElement(const QualifiedName& tagName, Document& document, bool createdByParser)
    350423    : HTMLElement(tagName, document)
     
    838911    if (auto* renderer = this->renderer()) {
    839912        renderer->updateFromElement();
    840         if (m_mediaSession->hasBehaviorRestriction(MediaElementSession::InvisibleAutoplayNotPermitted)
    841             || m_mediaSession->hasBehaviorRestriction(MediaElementSession::OverrideUserGestureRequirementForMainContent))
     913        if (m_mediaSession && m_mediaSession->wantsToObserveViewportVisibilityForAutoplay())
    842914            renderer->registerForVisibleInViewportCallback();
    843915    }
     
    40234095        scheduleUpdatePlaybackControlsManager();
    40244096    }
     4097
     4098    // If the video is a candidate for main content, we should register it for viewport visibility callbacks
     4099    // if it hasn't already been registered.
     4100    if (renderer() && m_mediaSession && !m_mediaSession->wantsToObserveViewportVisibilityForAutoplay() && m_mediaSession->wantsToObserveViewportVisibilityForMediaControls())
     4101        renderer()->registerForVisibleInViewportCallback();
    40254102}
    40264103
     
    71427219{
    71437220    updateShouldAutoplay();
     7221    scheduleUpdatePlaybackControlsManager();
    71447222}
    71457223
     
    71707248}
    71717249
     7250void HTMLMediaElement::resetPlaybackSessionState()
     7251{
     7252    if (m_mediaSession)
     7253        m_mediaSession->resetPlaybackSessionState();
     7254}
     7255
     7256bool HTMLMediaElement::isVisibleInViewport() const
     7257{
     7258    auto renderer = this->renderer();
     7259    return renderer && renderer->visibleInViewportState() == RenderElement::VisibleInViewport;
     7260}
     7261
    71727262void HTMLMediaElement::updatePlaybackControlsManager()
    71737263{
     
    71767266        return;
    71777267
    7178     PlatformMediaSession* session = PlatformMediaSessionManager::sharedManager().currentSessionMatching([] (const PlatformMediaSession& session) {
    7179         return session.canControlControlsManager();
    7180     });
    7181 
    7182     if (!is<MediaElementSession>(session))
     7268    // FIXME: Ensure that the renderer here should be up to date.
     7269    if (auto bestMediaSession = bestMediaSessionForShowingPlaybackControlsManager())
     7270        page->chrome().client().setUpPlaybackControlsManager(bestMediaSession->element());
     7271    else
    71837272        page->chrome().client().clearPlaybackControlsManager();
    7184     else
    7185         page->chrome().client().setUpPlaybackControlsManager(downcast<MediaElementSession>(session)->element());
    71867273}
    71877274
  • trunk/Source/WebCore/html/HTMLMediaElement.h

    r204989 r205412  
    469469    RenderMedia* renderer() const;
    470470
     471    void resetPlaybackSessionState();
     472    bool isVisibleInViewport() const;
     473
    471474protected:
    472475    HTMLMediaElement(const QualifiedName&, Document&, bool createdByParser);
  • trunk/Source/WebCore/html/MediaElementSession.cpp

    r204989 r205412  
    4848#include "ScriptController.h"
    4949#include "SourceBuffer.h"
     50#include <wtf/CurrentTime.h>
    5051
    5152#if PLATFORM(IOS)
     
    138139void MediaElementSession::removeBehaviorRestriction(BehaviorRestrictions restriction)
    139140{
     141    if (restriction & RequireUserGestureToControlControlsManager)
     142        m_mostRecentUserInteractionTime = monotonicallyIncreasingTime();
     143
    140144    LOG(Media, "MediaElementSession::removeBehaviorRestriction - removing %s", restrictionName(restriction).utf8().data());
    141145    m_restrictions &= ~restriction;
     
    213217}
    214218
    215 bool MediaElementSession::canControlControlsManager() const
     219bool MediaElementSession::canShowControlsManager() const
    216220{
    217221    if (m_element.isFullscreen()) {
    218         LOG(Media, "MediaElementSession::canControlControlsManager - returning TRUE: Is fullscreen");
     222        LOG(Media, "MediaElementSession::canShowControlsManager - returning TRUE: Is fullscreen");
    219223        return true;
    220224    }
    221225
    222226    if (!m_element.hasAudio()) {
    223         LOG(Media, "MediaElementSession::canControlControlsManager - returning FALSE: No audio");
     227        LOG(Media, "MediaElementSession::canShowControlsManager - returning FALSE: No audio");
    224228        return false;
    225229    }
    226230
    227231    if (m_element.document().activeDOMObjectsAreSuspended()) {
    228         LOG(Media, "MediaElementSession::canControlControlsManager - returning FALSE: activeDOMObjectsAreSuspended()");
     232        LOG(Media, "MediaElementSession::canShowControlsManager - returning FALSE: activeDOMObjectsAreSuspended()");
    229233        return false;
    230234    }
    231235
    232236    if (!playbackPermitted(m_element)) {
    233         LOG(Media, "MediaElementSession::canControlControlsManager - returning FALSE: Playback not permitted");
     237        LOG(Media, "MediaElementSession::canShowControlsManager - returning FALSE: Playback not permitted");
    234238        return false;
    235239    }
    236240
    237241    if (!hasBehaviorRestriction(RequireUserGestureToControlControlsManager) || ScriptController::processingUserGestureForMedia()) {
    238         LOG(Media, "MediaElementSession::canControlControlsManager - returning TRUE: No user gesture required");
     242        LOG(Media, "MediaElementSession::canShowControlsManager - returning TRUE: No user gesture required");
    239243        return true;
    240244    }
    241245
    242246    if (hasBehaviorRestriction(RequirePlaybackToControlControlsManager) && !m_element.isPlaying()) {
    243         LOG(Media, "MediaElementSession::canControlControlsManager - returning FALSE: Needs to be playing");
     247        LOG(Media, "MediaElementSession::canShowControlsManager - returning FALSE: Needs to be playing");
    244248        return false;
    245249    }
    246250
    247251    if (m_element.muted()) {
    248         LOG(Media, "MediaElementSession::canControlControlsManager - returning FALSE: Muted");
     252        LOG(Media, "MediaElementSession::canShowControlsManager - returning FALSE: Muted");
    249253        return false;
    250254    }
     
    252256    if (m_element.isVideo()) {
    253257        if (!m_element.renderer()) {
    254             LOG(Media, "MediaElementSession::canControlControlsManager - returning FALSE: No renderer");
     258            LOG(Media, "MediaElementSession::canShowControlsManager - returning FALSE: No renderer");
    255259            return false;
    256260        }
    257261
    258262        if (m_element.document().isMediaDocument()) {
    259             LOG(Media, "MediaElementSession::canControlControlsManager - returning TRUE: Is media document");
     263            LOG(Media, "MediaElementSession::canShowControlsManager - returning TRUE: Is media document");
    260264            return true;
    261265        }
    262266
    263267        if (!m_element.hasVideo()) {
    264             LOG(Media, "MediaElementSession::canControlControlsManager - returning FALSE: No video");
     268            LOG(Media, "MediaElementSession::canShowControlsManager - returning FALSE: No video");
    265269            return false;
    266270        }
    267271
    268         if (isElementLargeEnoughForMainContent(m_element)) {
    269             LOG(Media, "MediaElementSession::canControlControlsManager - returning TRUE: Is main content");
     272        if (isLargeEnoughForMainContent()) {
     273            LOG(Media, "MediaElementSession::canShowControlsManager - returning TRUE: Is main content");
    270274            return true;
    271275        }
    272276    }
    273277
    274     LOG(Media, "MediaElementSession::canControlControlsManager - returning FALSE: No user gesture");
     278    LOG(Media, "MediaElementSession::canShowControlsManager - returning FALSE: No user gesture");
    275279    return false;
     280}
     281
     282bool MediaElementSession::isLargeEnoughForMainContent() const
     283{
     284    return isElementLargeEnoughForMainContent(m_element);
     285}
     286
     287double MediaElementSession::mostRecentUserInteractionTime() const
     288{
     289    return m_mostRecentUserInteractionTime;
     290}
     291
     292bool MediaElementSession::wantsToObserveViewportVisibilityForMediaControls() const
     293{
     294    return isLargeEnoughForMainContent();
     295}
     296
     297bool MediaElementSession::wantsToObserveViewportVisibilityForAutoplay() const
     298{
     299    return hasBehaviorRestriction(InvisibleAutoplayNotPermitted) || hasBehaviorRestriction(OverrideUserGestureRequirementForMainContent);
    276300}
    277301
     
    503527}
    504528
     529void MediaElementSession::resetPlaybackSessionState()
     530{
     531    m_mostRecentUserInteractionTime = 0;
     532    addBehaviorRestriction(RequireUserGestureToControlControlsManager | RequirePlaybackToControlControlsManager);
     533}
     534
    505535bool MediaElementSession::allowsPictureInPicture(const HTMLMediaElement& element) const
    506536{
  • trunk/Source/WebCore/html/MediaElementSession.h

    r205011 r205412  
    5555    bool pageAllowsPlaybackAfterResuming(const HTMLMediaElement&) const;
    5656
    57     bool canControlControlsManager() const override;
    58 
    5957#if ENABLE(WIRELESS_PLAYBACK_TARGET)
    6058    void showPlaybackTargetPicker(const HTMLMediaElement&);
     
    7876
    7977    void mediaEngineUpdated(const HTMLMediaElement&);
     78
     79    void resetPlaybackSessionState() override;
    8080
    8181    // Restrictions to modify default behaviors.
     
    112112    HTMLMediaElement& element() const { return m_element; }
    113113
     114    bool wantsToObserveViewportVisibilityForMediaControls() const;
     115    bool wantsToObserveViewportVisibilityForAutoplay() const;
     116    bool canShowControlsManager() const;
     117    bool isLargeEnoughForMainContent() const;
     118    double mostRecentUserInteractionTime() const;
     119
    114120private:
    115121
     
    141147#endif
    142148
     149    double m_mostRecentUserInteractionTime { 0 };
     150
    143151    mutable bool m_isMainContent { false };
    144152    Timer m_mainContentCheckTimer;
  • trunk/Source/WebCore/platform/audio/PlatformMediaSession.h

    r203982 r205412  
    146146    bool shouldOverrideBackgroundLoadingRestriction() const;
    147147
    148     virtual bool canControlControlsManager() const { return false; }
    149148    virtual bool canPlayToWirelessPlaybackTarget() const { return false; }
    150149    virtual bool isPlayingToWirelessPlaybackTarget() const { return m_isPlayingToWirelessPlaybackTarget; }
     
    167166
    168167    void scheduleClientDataBufferingCheck();
     168    virtual void resetPlaybackSessionState() { }
    169169
    170170protected:
  • trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.cpp

    r203982 r205412  
    281281}
    282282
    283 PlatformMediaSession* PlatformMediaSessionManager::currentSessionMatching(std::function<bool(const PlatformMediaSession &)> filter)
    284 {
     283Vector<PlatformMediaSession*> PlatformMediaSessionManager::currentSessionsMatching(std::function<bool(const PlatformMediaSession &)> filter)
     284{
     285    Vector<PlatformMediaSession*> matchingSessions;
    285286    for (auto& session : m_sessions) {
    286287        if (filter(*session))
    287             return session;
    288     }
    289     return nullptr;
     288            matchingSessions.append(session);
     289    }
     290    return matchingSessions;
    290291}
    291292   
  • trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.h

    r203982 r205412  
    9393    PlatformMediaSession* currentSession() const;
    9494
    95     PlatformMediaSession* currentSessionMatching(std::function<bool(const PlatformMediaSession&)>);
     95    Vector<PlatformMediaSession*> currentSessionsMatching(std::function<bool(const PlatformMediaSession&)>);
    9696
    9797    void sessionIsPlayingToWirelessPlaybackTargetChanged(PlatformMediaSession&);
  • trunk/Source/WebCore/platform/cocoa/WebPlaybackSessionModelMediaElement.mm

    r205365 r205412  
    6868    m_isListening = false;
    6969
     70    if (m_mediaElement)
     71        m_mediaElement->resetPlaybackSessionState();
     72
    7073    m_mediaElement = mediaElement;
    7174
  • trunk/Source/WebKit2/ChangeLog

    r205411 r205412  
     12016-09-03  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        Refactor the heuristic for showing media controls to take all media sessions into account
     4        https://bugs.webkit.org/show_bug.cgi?id=161503
     5        <rdar://problem/28033783>
     6
     7        Reviewed by Darin Adler.
     8
     9        Adds an SPI testing hook for sending the element ID of the currently controlled video element from the web
     10        process to the UI process. See VideoControlsManager.mm in Tools/TestWebKitAPI/ for usage.
     11
     12        * UIProcess/API/Cocoa/WKWebView.mm:
     13        (-[WKWebView _requestControlledElementID]):
     14        (-[WKWebView _handleControlledElementIDResponse:]):
     15        (-[WKWebView _hasActiveVideoForControlsManager]): Deleted.
     16        * UIProcess/API/Cocoa/WKWebViewPrivate.h:
     17        * UIProcess/Cocoa/WebPlaybackSessionManagerProxy.h:
     18        * UIProcess/Cocoa/WebPlaybackSessionManagerProxy.messages.in:
     19        * UIProcess/Cocoa/WebPlaybackSessionManagerProxy.mm:
     20        (WebKit::WebPlaybackSessionManagerProxy::handleControlledElementIDResponse):
     21        (WebKit::WebPlaybackSessionManagerProxy::requestControlledElementID):
     22        * UIProcess/PageClient.h:
     23        * UIProcess/WebPageProxy.cpp:
     24        (WebKit::WebPageProxy::requestControlledElementID):
     25        (WebKit::WebPageProxy::handleControlledElementIDResponse):
     26        * UIProcess/WebPageProxy.h:
     27        * UIProcess/mac/PageClientImpl.h:
     28        * UIProcess/mac/PageClientImpl.mm:
     29        (WebKit::PageClientImpl::handleControlledElementIDResponse):
     30        * WebProcess/cocoa/WebPlaybackSessionManager.h:
     31        * WebProcess/cocoa/WebPlaybackSessionManager.messages.in:
     32        * WebProcess/cocoa/WebPlaybackSessionManager.mm:
     33        (WebKit::WebPlaybackSessionManager::handleControlledElementIDRequest):
     34
    1352016-09-03  Darin Adler  <darin@apple.com>
    236
  • trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebView.mm

    r205371 r205412  
    45544554    return _page && _page->hasActiveVideoForControlsManager();
    45554555}
     4556
     4557- (void)_requestControlledElementID
     4558{
     4559    if (_page)
     4560        _page->requestControlledElementID();
     4561}
     4562
     4563- (void)_handleControlledElementIDResponse:(NSString *)identifier
     4564{
     4565    // Overridden by subclasses.
     4566}
    45564567#endif // PLATFORM(MAC)
    45574568
  • trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebViewPrivate.h

    r204728 r205412  
    272272#if !TARGET_OS_IPHONE
    273273@property (nonatomic, readonly) BOOL _hasActiveVideoForControlsManager WK_API_AVAILABLE(macosx(WK_MAC_TBA));
     274- (void)_requestControlledElementID WK_API_AVAILABLE(macosx(WK_MAC_TBA));
     275- (void)_handleControlledElementIDResponse:(NSString *)identifier WK_API_AVAILABLE(macosx(WK_MAC_TBA));
    274276#endif
    275277
  • trunk/Source/WebKit2/UIProcess/Cocoa/WebPlaybackSessionManagerProxy.h

    r205365 r205412  
    147147
    148148    PlatformWebPlaybackSessionInterface* controlsManagerInterface();
     149    void requestControlledElementID();
    149150
    150151private:
     
    177178    void setDuration(uint64_t contextId, double duration);
    178179    void setRate(uint64_t contextId, bool isPlaying, double rate);
     180    void handleControlledElementIDResponse(uint64_t, String) const;
    179181
    180182    // Messages to WebPlaybackSessionManager
  • trunk/Source/WebKit2/UIProcess/Cocoa/WebPlaybackSessionManagerProxy.messages.in

    r199593 r205412  
    3636    SetUpPlaybackControlsManagerWithID(uint64_t contextId)
    3737    ClearPlaybackControlsManager()
     38
     39    HandleControlledElementIDResponse(uint64_t contextId, String id)
    3840}
    3941#endif
  • trunk/Source/WebKit2/UIProcess/Cocoa/WebPlaybackSessionManagerProxy.mm

    r205365 r205412  
    395395}
    396396
     397
     398void WebPlaybackSessionManagerProxy::handleControlledElementIDResponse(uint64_t contextId, String identifier) const
     399{
     400#if PLATFORM(MAC)
     401    if (contextId == m_controlsManagerContextId)
     402        m_page->handleControlledElementIDResponse(identifier);
     403#else
     404    UNUSED_PARAM(contextId);
     405    UNUSED_PARAM(identifier);
     406#endif
     407}
     408
     409
    397410#pragma mark Messages to WebPlaybackSessionManager
    398411
     
    455468{
    456469    m_page->send(Messages::WebPlaybackSessionManager::SelectLegibleMediaOption(contextId, index), m_page->pageID());
     470}
     471
     472void WebPlaybackSessionManagerProxy::requestControlledElementID()
     473{
     474    if (m_controlsManagerContextId)
     475        m_page->send(Messages::WebPlaybackSessionManager::HandleControlledElementIDRequest(m_controlsManagerContextId), m_page->pageID());
    457476}
    458477
  • trunk/Source/WebKit2/UIProcess/PageClient.h

    r204013 r205412  
    248248    virtual void recommendedScrollbarStyleDidChange(WebCore::ScrollbarStyle) = 0;
    249249    virtual void removeNavigationGestureSnapshot() = 0;
     250    virtual void handleControlledElementIDResponse(const String&) = 0;
    250251
    251252    virtual CGRect boundsOfLayerInLayerBackedWindowCoordinates(CALayer *) const = 0;
  • trunk/Source/WebKit2/UIProcess/WebPageProxy.cpp

    r205369 r205412  
    62996299}
    63006300
     6301void WebPageProxy::requestControlledElementID() const
     6302{
     6303#if ENABLE(VIDEO_PRESENTATION_MODE)
     6304    if (m_playbackSessionManager)
     6305        m_playbackSessionManager->requestControlledElementID();
     6306#endif
     6307}
     6308
     6309void WebPageProxy::handleControlledElementIDResponse(const String& identifier) const
     6310{
     6311    m_pageClient.handleControlledElementIDResponse(identifier);
     6312}
     6313
    63016314bool WebPageProxy::isPlayingVideoInEnhancedFullscreen() const
    63026315{
  • trunk/Source/WebKit2/UIProcess/WebPageProxy.h

    r204996 r205412  
    10321032    void videoControlsManagerDidChange();
    10331033    bool hasActiveVideoForControlsManager() const;
     1034    void requestControlledElementID() const;
     1035    void handleControlledElementIDResponse(const String&) const;
    10341036    bool isPlayingVideoInEnhancedFullscreen() const;
    10351037#endif
  • trunk/Source/WebKit2/UIProcess/mac/PageClientImpl.h

    r204013 r205412  
    207207    void didSameDocumentNavigationForMainFrame(SameDocumentNavigationType) override;
    208208    void removeNavigationGestureSnapshot() override;
     209    void handleControlledElementIDResponse(const String&) override;
    209210
    210211    void didPerformImmediateActionHitTest(const WebHitTestResultData&, bool contentPreventsDefault, API::Object*) override;
  • trunk/Source/WebKit2/UIProcess/mac/PageClientImpl.mm

    r204013 r205412  
    753753}
    754754
     755void PageClientImpl::handleControlledElementIDResponse(const String& identifier)
     756{
     757#if WK_API_ENABLED
     758    [m_webView _handleControlledElementIDResponse:nsStringFromWebCoreString(identifier)];
     759#endif
     760}
     761
    755762void PageClientImpl::didChangeBackgroundColor()
    756763{
  • trunk/Source/WebKit2/WebProcess/cocoa/WebPlaybackSessionManager.h

    r205365 r205412  
    144144    void selectAudioMediaOption(uint64_t contextId, uint64_t index);
    145145    void selectLegibleMediaOption(uint64_t contextId, uint64_t index);
     146    void handleControlledElementIDRequest(uint64_t contextId);
    146147
    147148    WebPage* m_page;
  • trunk/Source/WebKit2/WebProcess/cocoa/WebPlaybackSessionManager.messages.in

    r199593 r205412  
    3636    SelectAudioMediaOption(uint64_t contextId, uint64_t index)
    3737    SelectLegibleMediaOption(uint64_t contextId, uint64_t index)
     38    HandleControlledElementIDRequest(uint64_t contextId)
    3839}
    3940#endif
  • trunk/Source/WebKit2/WebProcess/cocoa/WebPlaybackSessionManager.mm

    r205365 r205412  
    416416}
    417417
     418void WebPlaybackSessionManager::handleControlledElementIDRequest(uint64_t contextId)
     419{
     420    auto element = ensureModel(contextId).mediaElement();
     421    if (element)
     422        m_page->send(Messages::WebPlaybackSessionManagerProxy::HandleControlledElementIDResponse(contextId, element->getIdAttribute()));
     423}
     424
    418425} // namespace WebKit
    419426
  • trunk/Tools/ChangeLog

    r205399 r205412  
     12016-09-03  Wenson Hsieh  <wenson_hsieh@apple.com>
     2
     3        Refactor the heuristic for showing media controls to take all media sessions into account
     4        https://bugs.webkit.org/show_bug.cgi?id=161503
     5        <rdar://problem/28033783>
     6
     7        Reviewed by Darin Adler.
     8
     9        Adds new unit tests verifying the behavior of media playback controls when scrolling another video into view.
     10        Please see the WebCore ChangeLog for more details about this change. Also refactors existing
     11        VideoControlsManager tests by folding duplicated setup and testing logic into helper methods to make the unit
     12        tests more readable.
     13
     14        * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
     15        * TestWebKitAPI/Tests/WebKit2Cocoa/VideoControlsManager.mm:
     16        (-[MessageHandler initWithMessage:handler:]):
     17        (-[MessageHandler userContentController:didReceiveScriptMessage:]):
     18        (-[VideoControlsManagerTestWebView performAfterLoading:]):
     19        (-[VideoControlsManagerTestWebView loadTestPageNamed:]):
     20        (-[VideoControlsManagerTestWebView loadTestPageNamed:andExpectControlsManager:afterReceivingMessage:]):
     21        (-[VideoControlsManagerTestWebView performAfterReceivingMessage:action:]):
     22        (-[VideoControlsManagerTestWebView controlledElementID]):
     23        (-[VideoControlsManagerTestWebView _handleControlledElementIDResponse:]):
     24        (TestWebKitAPI::setUpWebViewForTestingVideoControlsManager):
     25        (TestWebKitAPI::TEST):
     26        (-[MediaPlaybackMessageHandler initWithWKWebView:finalMessageString:]): Deleted.
     27        (-[MediaPlaybackMessageHandler userContentController:didReceiveScriptMessage:]): Deleted.
     28        (-[OnLoadMessageHandler initWithWKWebView:handler:]): Deleted.
     29        (-[OnLoadMessageHandler userContentController:didReceiveScriptMessage:]): Deleted.
     30        (-[WKWebView performAfterLoading:]): Deleted.
     31        * TestWebKitAPI/Tests/WebKit2Cocoa/large-video-playing-scroll-away.html: Added.
     32        * TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-autoplaying-click-to-pause.html: Added.
     33        * TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-autoplaying-scroll-to-video.html: Added.
     34        * TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-paused-video-hides-controls.html: Added.
     35        * TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-playing-muted-video-hides-controls.html: Added.
     36        * TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-playing-video-keeps-controls.html: Added.
     37
    1382016-09-03  Youenn Fablet  <youenn@apple.com>
    239
  • trunk/Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj

    r205329 r205412  
    6565                2E1DFDEF1D42A6F200714A00 /* large-videos-with-audio-autoplay.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E1DFDEE1D42A6EB00714A00 /* large-videos-with-audio-autoplay.html */; };
    6666                2E1DFDF11D42E1E400714A00 /* large-video-seek-to-beginning-and-play-after-ending.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E1DFDF01D42E14400714A00 /* large-video-seek-to-beginning-and-play-after-ending.html */; };
     67                2E691AEA1D78B53600129407 /* large-videos-paused-video-hides-controls.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E691AE81D78B52B00129407 /* large-videos-paused-video-hides-controls.html */; };
     68                2E691AEB1D78B53600129407 /* large-videos-playing-video-keeps-controls.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E691AE91D78B52B00129407 /* large-videos-playing-video-keeps-controls.html */; };
     69                2E691AED1D78B91000129407 /* large-videos-playing-muted-video-hides-controls.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E691AEC1D78B90200129407 /* large-videos-playing-muted-video-hides-controls.html */; };
     70                2E691AEF1D79DE8400129407 /* large-videos-autoplaying-click-to-pause.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E691AEE1D79DE6F00129407 /* large-videos-autoplaying-click-to-pause.html */; };
     71                2E691AF11D79E51A00129407 /* large-videos-autoplaying-scroll-to-video.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E691AF01D79E51400129407 /* large-videos-autoplaying-scroll-to-video.html */; };
     72                2E691AF31D79E75E00129407 /* large-video-playing-scroll-away.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E691AF21D79E75400129407 /* large-video-playing-scroll-away.html */; };
    6773                2E7765CD16C4D80A00BA2BB1 /* mainIOS.mm in Sources */ = {isa = PBXBuildFile; fileRef = 2E7765CC16C4D80A00BA2BB1 /* mainIOS.mm */; };
    6874                2E7765CF16C4D81100BA2BB1 /* mainMac.mm in Sources */ = {isa = PBXBuildFile; fileRef = 2E7765CE16C4D81100BA2BB1 /* mainMac.mm */; };
     
    521527                        dstSubfolderSpec = 7;
    522528                        files = (
     529                                2E691AF31D79E75E00129407 /* large-video-playing-scroll-away.html in Copy Resources */,
     530                                2E691AF11D79E51A00129407 /* large-videos-autoplaying-scroll-to-video.html in Copy Resources */,
     531                                2E691AEF1D79DE8400129407 /* large-videos-autoplaying-click-to-pause.html in Copy Resources */,
     532                                2E691AED1D78B91000129407 /* large-videos-playing-muted-video-hides-controls.html in Copy Resources */,
     533                                2E691AEA1D78B53600129407 /* large-videos-paused-video-hides-controls.html in Copy Resources */,
     534                                2E691AEB1D78B53600129407 /* large-videos-playing-video-keeps-controls.html in Copy Resources */,
    523535                                1A9E52C913E65EF4006917F5 /* 18-characters.html in Copy Resources */,
    524536                                379028B914FAC24C007E6B43 /* acceptsFirstMouse.html in Copy Resources */,
     
    744756                2E1DFDEE1D42A6EB00714A00 /* large-videos-with-audio-autoplay.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-videos-with-audio-autoplay.html"; sourceTree = "<group>"; };
    745757                2E1DFDF01D42E14400714A00 /* large-video-seek-to-beginning-and-play-after-ending.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-video-seek-to-beginning-and-play-after-ending.html"; sourceTree = "<group>"; };
     758                2E691AE81D78B52B00129407 /* large-videos-paused-video-hides-controls.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-videos-paused-video-hides-controls.html"; sourceTree = "<group>"; };
     759                2E691AE91D78B52B00129407 /* large-videos-playing-video-keeps-controls.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-videos-playing-video-keeps-controls.html"; sourceTree = "<group>"; };
     760                2E691AEC1D78B90200129407 /* large-videos-playing-muted-video-hides-controls.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-videos-playing-muted-video-hides-controls.html"; sourceTree = "<group>"; };
     761                2E691AEE1D79DE6F00129407 /* large-videos-autoplaying-click-to-pause.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-videos-autoplaying-click-to-pause.html"; sourceTree = "<group>"; };
     762                2E691AF01D79E51400129407 /* large-videos-autoplaying-scroll-to-video.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-videos-autoplaying-scroll-to-video.html"; sourceTree = "<group>"; };
     763                2E691AF21D79E75400129407 /* large-video-playing-scroll-away.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-video-playing-scroll-away.html"; sourceTree = "<group>"; };
    746764                2E7765CC16C4D80A00BA2BB1 /* mainIOS.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = mainIOS.mm; sourceTree = "<group>"; };
    747765                2E7765CE16C4D81100BA2BB1 /* mainMac.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = mainMac.mm; sourceTree = "<group>"; };
     
    14011419                        isa = PBXGroup;
    14021420                        children = (
     1421                                2E691AF21D79E75400129407 /* large-video-playing-scroll-away.html */,
     1422                                2E691AF01D79E51400129407 /* large-videos-autoplaying-scroll-to-video.html */,
     1423                                2E691AEE1D79DE6F00129407 /* large-videos-autoplaying-click-to-pause.html */,
     1424                                2E691AEC1D78B90200129407 /* large-videos-playing-muted-video-hides-controls.html */,
     1425                                2E691AE81D78B52B00129407 /* large-videos-paused-video-hides-controls.html */,
     1426                                2E691AE91D78B52B00129407 /* large-videos-playing-video-keeps-controls.html */,
    14031427                                5C9E593E1D3EB1DE00E3C62E /* ApplicationCache.db */,
    14041428                                5C9E593F1D3EB1DE00E3C62E /* ApplicationCache.db-shm */,
  • trunk/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/VideoControlsManager.mm

    r204989 r205412  
    3838#if WK_API_ENABLED && PLATFORM(MAC) && __MAC_OS_X_VERSION_MIN_REQUIRED >= 101200
    3939
    40 static bool testedControlsManagerAfterPlaying;
    41 static bool receivedScriptMessage;
    42 
    43 @interface MediaPlaybackMessageHandler : NSObject <WKScriptMessageHandler> {
    44     RetainPtr<WKWebView> _webView;
    45 }
    46 
    47 @property (nonatomic) BOOL expectedToHaveControlsManager;
    48 @property (nonatomic, retain) NSString *finalMessageString;
    49 
    50 - (instancetype)initWithWKWebView:(WKWebView*)webView finalMessageString:(NSString *)finalMessageString;
    51 @end
    52 
    53 @implementation MediaPlaybackMessageHandler
    54 
    55 - (instancetype)initWithWKWebView:(WKWebView*)webView finalMessageString:(NSString *)finalMessageString
     40@interface WKWebView (WKWebViewAdditions)
     41
     42- (void)_interactWithMediaControlsForTesting;
     43
     44@end
     45
     46@interface MessageHandler : NSObject <WKScriptMessageHandler> {
     47    dispatch_block_t _handler;
     48    NSString *_message;
     49}
     50
     51- (instancetype)initWithMessage:(NSString *)message handler:(dispatch_block_t)handler;
     52
     53@end
     54
     55@implementation MessageHandler
     56
     57- (instancetype)initWithMessage:(NSString *)message handler:(dispatch_block_t)handler
    5658{
    5759    if (!(self = [super init]))
    5860        return nil;
    5961
    60     _webView = webView;
    61     _finalMessageString = finalMessageString;
     62    _handler = [handler copy];
     63    _message = message;
    6264
    6365    return self;
     
    6668- (void)userContentController:(WKUserContentController *)userContentController didReceiveScriptMessage:(WKScriptMessage *)message
    6769{
    68     receivedScriptMessage = true;
    69 
    70     NSString *bodyString = (NSString *)[message body];
    71     if ([bodyString isEqualToString:self.finalMessageString]) {
    72         BOOL hasControlsManager = [_webView _hasActiveVideoForControlsManager];
    73         if (self.expectedToHaveControlsManager)
    74             EXPECT_TRUE(hasControlsManager);
    75         else
    76             EXPECT_FALSE(hasControlsManager);
    77         testedControlsManagerAfterPlaying = true;
    78     }
    79 }
    80 @end
    81 
    82 @interface OnLoadMessageHandler : NSObject <WKScriptMessageHandler> {
    83     RetainPtr<WKWebView> _webView;
    84 }
    85 
    86 @property (nonatomic, strong) dispatch_block_t onloadHandler;
    87 
    88 - (instancetype)initWithWKWebView:(WKWebView*)webView handler:(dispatch_block_t)handler;
    89 @end
    90 
    91 @implementation OnLoadMessageHandler
    92 
    93 - (instancetype)initWithWKWebView:(WKWebView*)webView handler:(dispatch_block_t)handler
    94 {
    95     if (!(self = [super init]))
    96         return nil;
    97 
    98     _webView = webView;
    99     _onloadHandler = handler;
    100 
    101     return self;
    102 }
    103 
    104 - (void)userContentController:(WKUserContentController *)userContentController didReceiveScriptMessage:(WKScriptMessage *)message
    105 {
    106     if (![(NSString *)[message body] isEqualToString:@"loaded"])
    107         return;
    108 
    109     if (_onloadHandler)
    110         _onloadHandler();
    111 
    112     _onloadHandler = nil;
    113 }
    114 @end
    115 
    116 @interface WKWebView (WKWebViewAdditions)
    117 
    118 - (void)_interactWithMediaControlsForTesting;
    119 
    120 @end
    121 
    122 @interface WKWebView (TestingAdditions)
     70    if ([(NSString *)[message body] isEqualToString:_message] && _handler)
     71        _handler();
     72}
     73
     74@end
     75
     76@interface VideoControlsManagerTestWebView : WKWebView
    12377
    12478- (void)mouseDownAtPoint:(NSPoint)point;
    12579- (void)performAfterLoading:(dispatch_block_t)actions;
    126 
    127 @end
    128 
    129 @implementation WKWebView (TestingAdditions)
     80- (void)loadTestPageNamed:(NSString *)pageName;
     81- (void)loadTestPageNamed:(NSString *)pageName andExpectControlsManager:(BOOL)expectControlsManager afterReceivingMessage:(NSString *)message;
     82- (void)performAfterReceivingMessage:(NSString *)message action:(dispatch_block_t)action;
     83
     84@property (nonatomic, readonly) NSString *controlledElementID;
     85
     86@end
     87
     88@implementation VideoControlsManagerTestWebView {
     89    bool _isDoneQueryingControlledElementID;
     90    NSString *_controlledElementID;
     91}
    13092
    13193- (void)mouseDownAtPoint:(NSPoint)point {
     
    13496
    13597- (void)performAfterLoading:(dispatch_block_t)actions {
    136     OnLoadMessageHandler *handler = [[OnLoadMessageHandler alloc] initWithWKWebView:self handler:actions];
     98    MessageHandler *handler = [[MessageHandler alloc] initWithMessage:@"loaded" handler:actions];
    13799    NSString *onloadScript = @"window.onload = function() { window.webkit.messageHandlers.onloadHandler.postMessage('loaded'); }";
    138100    WKUserScript *script = [[WKUserScript alloc] initWithSource:onloadScript injectionTime:WKUserScriptInjectionTimeAtDocumentStart forMainFrameOnly:NO];
    139     [[[self configuration] userContentController] addUserScript:script];
    140     [[[self configuration] userContentController] addScriptMessageHandler:handler name:@"onloadHandler"];
     101
     102    WKUserContentController* contentController = [[self configuration] userContentController];
     103    [contentController addUserScript:script];
     104    [contentController addScriptMessageHandler:handler name:@"onloadHandler"];
     105}
     106
     107- (void)loadTestPageNamed:(NSString *)pageName
     108{
     109    NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:pageName withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
     110    [self loadRequest:request];
     111}
     112
     113- (void)loadTestPageNamed:(NSString *)pageName andExpectControlsManager:(BOOL)expectControlsManager afterReceivingMessage:(NSString *)message
     114{
     115    __block bool doneWaiting = false;
     116    [self performAfterReceivingMessage:message action:^ {
     117        BOOL hasVideoForControlsManager = [self _hasActiveVideoForControlsManager];
     118        if (expectControlsManager)
     119            EXPECT_TRUE(hasVideoForControlsManager);
     120        else
     121            EXPECT_FALSE(hasVideoForControlsManager);
     122        doneWaiting = true;
     123    }];
     124
     125    [self loadTestPageNamed:pageName];
     126    TestWebKitAPI::Util::run(&doneWaiting);
     127}
     128
     129- (void)performAfterReceivingMessage:(NSString *)message action:(dispatch_block_t)action
     130{
     131    RetainPtr<MessageHandler> handler = adoptNS([[MessageHandler alloc] initWithMessage:message handler:action]);
     132    [[[self configuration] userContentController] addScriptMessageHandler:handler.get() name:@"playingHandler"];
     133}
     134
     135- (NSString *)controlledElementID
     136{
     137    _isDoneQueryingControlledElementID = false;
     138    [self _requestControlledElementID];
     139    TestWebKitAPI::Util::run(&_isDoneQueryingControlledElementID);
     140    return _controlledElementID;
     141}
     142
     143- (void)_handleControlledElementIDResponse:(NSString *)identifier
     144{
     145    _controlledElementID = [identifier copy];
     146    _isDoneQueryingControlledElementID = true;
    141147}
    142148
     
    145151namespace TestWebKitAPI {
    146152
    147 TEST(VideoControlsManager, VideoControlsManagerSingleLargeVideo)
     153RetainPtr<VideoControlsManagerTestWebView*> setUpWebViewForTestingVideoControlsManager(NSRect frame)
    148154{
    149155    RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    150156    configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    151     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
    152     RetainPtr<MediaPlaybackMessageHandler> handler = adoptNS([[MediaPlaybackMessageHandler alloc] initWithWKWebView:webView.get() finalMessageString:@"playing"]);
    153     [[configuration userContentController] addScriptMessageHandler:handler.get() name:@"playingHandler"];
    154 
     157    RetainPtr<VideoControlsManagerTestWebView> webView = adoptNS([[VideoControlsManagerTestWebView alloc] initWithFrame:frame configuration:configuration.get()]);
    155158    RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    156159    [[window contentView] addSubview:webView.get()];
    157160
     161    return webView;
     162}
     163
     164TEST(VideoControlsManager, VideoControlsManagerSingleLargeVideo)
     165{
     166    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 100, 100));
     167
    158168    // A large video with audio should have a controls manager even if it is played via script like this video.
    159169    // So the expectation is YES.
    160     [handler setExpectedToHaveControlsManager:YES];
    161     NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"large-video-with-audio" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
    162     [webView loadRequest:request];
    163 
    164     TestWebKitAPI::Util::run(&testedControlsManagerAfterPlaying);
    165     TestWebKitAPI::Util::run(&receivedScriptMessage);
     170    [webView loadTestPageNamed:@"large-video-with-audio" andExpectControlsManager:YES afterReceivingMessage:@"playing"];
    166171}
    167172
    168173TEST(VideoControlsManager, VideoControlsManagerSingleSmallVideo)
    169174{
    170     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    171     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    172     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
    173     RetainPtr<MediaPlaybackMessageHandler> handler = adoptNS([[MediaPlaybackMessageHandler alloc] initWithWKWebView:webView.get() finalMessageString:@"playing"]);
    174     [[configuration userContentController] addScriptMessageHandler:handler.get() name:@"playingHandler"];
    175 
    176     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    177     [[window contentView] addSubview:webView.get()];
     175    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 100, 100));
    178176
    179177    // A small video will not have a controls manager unless it started playing because of a user gesture. Since this
    180178    // video is started with a script, the expectation is NO.
    181     [handler setExpectedToHaveControlsManager:NO];
    182     NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"video-with-audio" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
    183     [webView loadRequest:request];
    184 
    185     TestWebKitAPI::Util::run(&testedControlsManagerAfterPlaying);
    186     TestWebKitAPI::Util::run(&receivedScriptMessage);
     179    [webView loadTestPageNamed:@"video-with-audio" andExpectControlsManager:NO afterReceivingMessage:@"playing"];
    187180}
    188181
    189182TEST(VideoControlsManager, VideoControlsManagerMultipleVideosWithAudio)
    190183{
    191     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    192     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    193     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
     184    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 100, 100));
    194185
    195186    __block BOOL didShowMediaControls;
    196187    __block bool isDoneLoading = false;
    197188
    198     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    199     [[window contentView] addSubview:webView.get()];
    200     RetainPtr<OnLoadMessageHandler> onloadHandler = adoptNS([[OnLoadMessageHandler alloc] initWithWKWebView:webView.get() handler:^() {
     189    [webView performAfterLoading:^ {
    201190        didShowMediaControls = [webView _hasActiveVideoForControlsManager];
    202191        isDoneLoading = true;
    203     }]);
    204     [[configuration userContentController] addScriptMessageHandler:onloadHandler.get() name:@"onloadHandler"];
    205 
    206     NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"large-videos-with-audio" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
    207     [webView loadRequest:request];
     192    }];
     193
     194    [webView loadTestPageNamed:@"large-videos-with-audio"];
    208195
    209196    TestWebKitAPI::Util::run(&isDoneLoading);
     
    213200TEST(VideoControlsManager, VideoControlsManagerMultipleVideosWithAudioAndAutoplay)
    214201{
    215     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    216     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    217     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
     202    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 100, 100));
    218203
    219204    __block BOOL didShowMediaControls;
    220205    __block bool isDoneLoading = false;
    221206
    222     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    223     [[window contentView] addSubview:webView.get()];
    224     RetainPtr<OnLoadMessageHandler> onloadHandler = adoptNS([[OnLoadMessageHandler alloc] initWithWKWebView:webView.get() handler:^() {
     207    [webView performAfterLoading:^ {
    225208        didShowMediaControls = [webView _hasActiveVideoForControlsManager];
    226209        isDoneLoading = true;
    227     }]);
    228     [[configuration userContentController] addScriptMessageHandler:onloadHandler.get() name:@"onloadHandler"];
    229 
    230     NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"large-videos-with-audio-autoplay" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
    231     [webView loadRequest:request];
     210    }];
     211
     212    [webView loadTestPageNamed:@"large-videos-with-audio-autoplay"];
    232213
    233214    TestWebKitAPI::Util::run(&isDoneLoading);
     
    235216}
    236217
     218TEST(VideoControlsManager, VideoControlsManagerMultipleVideosScrollPausedVideoOutOfView)
     219{
     220    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 500, 500));
     221
     222    [webView loadTestPageNamed:@"large-videos-paused-video-hides-controls" andExpectControlsManager:NO afterReceivingMessage:@"paused"];
     223}
     224
     225TEST(VideoControlsManager, VideoControlsManagerMultipleVideosScrollPlayingVideoWithSoundOutOfView)
     226{
     227    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 500, 500));
     228
     229    [webView loadTestPageNamed:@"large-videos-playing-video-keeps-controls" andExpectControlsManager:YES afterReceivingMessage:@"playing"];
     230}
     231
     232TEST(VideoControlsManager, VideoControlsManagerMultipleVideosScrollPlayingMutedVideoOutOfView)
     233{
     234    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 500, 500));
     235
     236    [webView loadTestPageNamed:@"large-videos-playing-muted-video-hides-controls" andExpectControlsManager:NO afterReceivingMessage:@"playing"];
     237}
     238
     239TEST(VideoControlsManager, VideoControlsManagerMultipleVideosShowControlsForLastInteractedVideo)
     240{
     241    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 800, 600));
     242    NSPoint clickPoint = NSMakePoint(400, 300);
     243
     244    [webView performAfterLoading:^ {
     245        [webView mouseDownAtPoint:clickPoint];
     246    }];
     247
     248    __block bool firstVideoPaused = false;
     249    __block bool secondVideoPaused = false;
     250    [webView performAfterReceivingMessage:@"paused" action:^ {
     251        NSString *controlledElementID = [webView controlledElementID];
     252        if (firstVideoPaused) {
     253            secondVideoPaused = true;
     254            EXPECT_TRUE([controlledElementID isEqualToString:@"second"]);
     255        } else {
     256            [webView mouseDownAtPoint:clickPoint];
     257            EXPECT_TRUE([controlledElementID isEqualToString:@"first"]);
     258        }
     259        firstVideoPaused = true;
     260    }];
     261
     262    [webView loadTestPageNamed:@"large-videos-autoplaying-click-to-pause"];
     263    TestWebKitAPI::Util::run(&secondVideoPaused);
     264}
     265
     266TEST(VideoControlsManager, VideoControlsManagerMultipleVideosSwitchControlledVideoWhenScrolling)
     267{
     268    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 800, 600));
     269
     270    __block bool scrolledSecondVideoIntoView = false;
     271    [webView loadTestPageNamed:@"large-videos-autoplaying-scroll-to-video"];
     272    [webView performAfterReceivingMessage:@"scrolled" action:^ {
     273        scrolledSecondVideoIntoView = true;
     274    }];
     275
     276    TestWebKitAPI::Util::run(&scrolledSecondVideoIntoView);
     277    EXPECT_TRUE([[webView controlledElementID] isEqualToString:@"second"]);
     278}
     279
     280TEST(VideoControlsManager, VideoControlsManagerMultipleVideosScrollOnlyLargeVideoOutOfView)
     281{
     282    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 500, 500));
     283
     284    [webView loadTestPageNamed:@"large-video-playing-scroll-away" andExpectControlsManager:YES afterReceivingMessage:@"scrolled"];
     285}
     286
    237287TEST(VideoControlsManager, VideoControlsManagerSingleSmallAutoplayingVideo)
    238288{
    239     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    240     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    241     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
    242     RetainPtr<MediaPlaybackMessageHandler> playbackHandler = adoptNS([[MediaPlaybackMessageHandler alloc] initWithWKWebView:webView.get() finalMessageString:@"paused"]);
    243     [[configuration userContentController] addScriptMessageHandler:playbackHandler.get() name:@"playingHandler"];
    244 
    245     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    246     [[window contentView] addSubview:webView.get()];
    247 
    248     RetainPtr<OnLoadMessageHandler> onloadHandler = adoptNS([[OnLoadMessageHandler alloc] initWithWKWebView:webView.get() handler:^() {
     289    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 100, 100));
     290
     291    [webView performAfterLoading:^ {
    249292        [webView mouseDownAtPoint:NSMakePoint(50, 50)];
    250     }]);
    251     [[configuration userContentController] addScriptMessageHandler:onloadHandler.get() name:@"onloadHandler"];
    252 
    253     // A small video should have a controls manager after the first user gesture, which includes pausing the video. The expectation is YES.
    254     [playbackHandler setExpectedToHaveControlsManager:YES];
    255     NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"autoplaying-video-with-audio" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
    256     [webView loadRequest:request];
    257 
    258     TestWebKitAPI::Util::run(&testedControlsManagerAfterPlaying);
    259     TestWebKitAPI::Util::run(&receivedScriptMessage);
     293    }];
     294
     295    [webView loadTestPageNamed:@"autoplaying-video-with-audio" andExpectControlsManager:YES afterReceivingMessage:@"paused"];
    260296}
    261297
    262298TEST(VideoControlsManager, VideoControlsManagerLargeAutoplayingVideoSeeksAfterEnding)
    263299{
    264     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    265     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    266     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
    267     RetainPtr<MediaPlaybackMessageHandler> handler = adoptNS([[MediaPlaybackMessageHandler alloc] initWithWKWebView:webView.get() finalMessageString:@"ended"]);
    268     [[configuration userContentController] addScriptMessageHandler:handler.get() name:@"playingHandler"];
    269 
    270     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    271     [[window contentView] addSubview:webView.get()];
     300    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 100, 100));
    272301
    273302    // Since the video has ended, the expectation is NO even if the page programmatically seeks to the beginning.
    274     [handler setExpectedToHaveControlsManager:NO];
    275     NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"large-video-seek-after-ending" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
    276     [webView loadRequest:request];
    277 
    278     TestWebKitAPI::Util::run(&testedControlsManagerAfterPlaying);
    279     TestWebKitAPI::Util::run(&receivedScriptMessage);
     303    [webView loadTestPageNamed:@"large-video-seek-after-ending" andExpectControlsManager:NO afterReceivingMessage:@"ended"];
    280304}
    281305
    282306TEST(VideoControlsManager, VideoControlsManagerLargeAutoplayingVideoSeeksAndPlaysAfterEnding)
    283307{
    284     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    285     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    286     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
    287     RetainPtr<MediaPlaybackMessageHandler> handler = adoptNS([[MediaPlaybackMessageHandler alloc] initWithWKWebView:webView.get() finalMessageString:@"replaying"]);
    288     [[configuration userContentController] addScriptMessageHandler:handler.get() name:@"playingHandler"];
    289 
    290     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    291     [[window contentView] addSubview:webView.get()];
     308    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 100, 100));
    292309
    293310    // Since the video is still playing, the expectation is YES even if the video has ended once.
    294     [handler setExpectedToHaveControlsManager:YES];
    295     NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"large-video-seek-to-beginning-and-play-after-ending" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
    296     [webView loadRequest:request];
    297 
    298     TestWebKitAPI::Util::run(&testedControlsManagerAfterPlaying);
    299     TestWebKitAPI::Util::run(&receivedScriptMessage);
     311    [webView loadTestPageNamed:@"large-video-seek-to-beginning-and-play-after-ending" andExpectControlsManager:YES afterReceivingMessage:@"replaying"];
    300312}
    301313
    302314TEST(VideoControlsManager, VideoControlsManagerLargeAutoplayingVideoAfterSeekingToEnd)
    303315{
    304     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    305     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    306     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
    307     RetainPtr<MediaPlaybackMessageHandler> handler = adoptNS([[MediaPlaybackMessageHandler alloc] initWithWKWebView:webView.get() finalMessageString:@"ended"]);
    308     [[configuration userContentController] addScriptMessageHandler:handler.get() name:@"playingHandler"];
    309 
    310     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    311     [[window contentView] addSubview:webView.get()];
    312 
    313     RetainPtr<OnLoadMessageHandler> onloadHandler = adoptNS([[OnLoadMessageHandler alloc] initWithWKWebView:webView.get() handler:^() {
     316    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 100, 100));
     317
     318    [webView performAfterLoading:^ {
    314319        [webView mouseDownAtPoint:NSMakePoint(50, 50)];
    315     }]);
    316     [[configuration userContentController] addScriptMessageHandler:onloadHandler.get() name:@"onloadHandler"];
     320    }];
    317321
    318322    // We expect there to be media controls, since this is a user gestured seek to the end.
    319323    // This is akin to seeking to the end by scrubbing in the controls.
    320     [handler setExpectedToHaveControlsManager:YES];
    321     NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"large-video-hides-controls-after-seek-to-end" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
    322     [webView loadRequest:request];
    323 
    324     TestWebKitAPI::Util::run(&testedControlsManagerAfterPlaying);
    325     TestWebKitAPI::Util::run(&receivedScriptMessage);
     324    [webView loadTestPageNamed:@"large-video-hides-controls-after-seek-to-end" andExpectControlsManager:YES afterReceivingMessage:@"ended"];
    326325}
    327326
    328327TEST(VideoControlsManager, VideoControlsManagerSingleLargeVideoWithoutAudio)
    329328{
    330     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    331     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    332     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
    333     RetainPtr<MediaPlaybackMessageHandler> handler = adoptNS([[MediaPlaybackMessageHandler alloc] initWithWKWebView:webView.get() finalMessageString:@"playing"]);
    334     [[configuration userContentController] addScriptMessageHandler:handler.get() name:@"playingHandler"];
    335 
    336     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    337     [[window contentView] addSubview:webView.get()];
     329    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 100, 100));
    338330
    339331    // A large video with no audio will not have a controls manager unless it started playing because of a user gesture. Since this
    340332    // video is started with a script, the expectation is NO.
    341     [handler setExpectedToHaveControlsManager:NO];
    342     NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"large-video-without-audio" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
    343     [webView loadRequest:request];
    344 
    345     TestWebKitAPI::Util::run(&testedControlsManagerAfterPlaying);
    346     TestWebKitAPI::Util::run(&receivedScriptMessage);
     333    [webView loadTestPageNamed:@"large-video-without-audio" andExpectControlsManager:NO afterReceivingMessage:@"playing"];
    347334}
    348335
    349336TEST(VideoControlsManager, VideoControlsManagerAudioElementStartedWithScript)
    350337{
    351     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    352     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    353     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
    354     RetainPtr<MediaPlaybackMessageHandler> handler = adoptNS([[MediaPlaybackMessageHandler alloc] initWithWKWebView:webView.get() finalMessageString:@"playing"]);
    355     [[configuration userContentController] addScriptMessageHandler:handler.get() name:@"playingHandler"];
    356 
    357     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    358     [[window contentView] addSubview:webView.get()];
     338    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 100, 100));
    359339
    360340    // An audio element MUST be started with a user gesture in order to have a controls manager, so the expectation is NO.
    361     [handler setExpectedToHaveControlsManager:NO];
    362     NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"audio-only" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
    363     [webView loadRequest:request];
    364 
    365     TestWebKitAPI::Util::run(&testedControlsManagerAfterPlaying);
    366     TestWebKitAPI::Util::run(&receivedScriptMessage);
     341    [webView loadTestPageNamed:@"audio-only" andExpectControlsManager:NO afterReceivingMessage:@"playing"];
    367342}
    368343
    369344TEST(VideoControlsManager, VideoControlsManagerTearsDownMediaControlsOnDealloc)
    370345{
    371     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    372     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    373     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
    374 
    375     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    376     [[window contentView] addSubview:webView.get()];
     346    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 100, 100));
    377347
    378348    NSURL *urlOfVideo = [[NSBundle mainBundle] URLForResource:@"video-with-audio" withExtension:@"mp4" subdirectory:@"TestWebKitAPI.resources"];
     
    380350
    381351    __block bool finishedTest = false;
    382     [webView performAfterLoading:^()
    383     {
     352    [webView performAfterLoading:^ {
    384353        // Verify that we tear down the media controls properly, such that we don't crash when the web view is released.
    385354        if ([webView respondsToSelector:@selector(_interactWithMediaControlsForTesting)])
     
    395364TEST(VideoControlsManager, VideoControlsManagerSmallVideoInMediaDocument)
    396365{
    397     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    398     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    399     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
    400    
    401     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    402     [[window contentView] addSubview:webView.get()];
    403    
     366    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 100, 100));
     367
    404368    __block bool finishedLoad = false;
    405     [webView performAfterLoading:^()
    406     {
     369    [webView performAfterLoading:^ {
    407370        finishedLoad = true;
    408371    }];
     
    419382TEST(VideoControlsManager, VideoControlsManagerLongSkinnyVideoInWideMainFrame)
    420383{
    421     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    422     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    423     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 1600, 800) configuration:configuration.get()]);
    424     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    425     [[window contentView] addSubview:webView.get()];
    426 
    427     RetainPtr<MediaPlaybackMessageHandler> handler = adoptNS([[MediaPlaybackMessageHandler alloc] initWithWKWebView:webView.get() finalMessageString:@"playing"]);
    428     [[configuration userContentController] addScriptMessageHandler:handler.get() name:@"playingHandler"];
    429     [handler setExpectedToHaveControlsManager:NO];
    430 
    431     NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"skinny-autoplaying-video-with-audio" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
    432     [webView loadRequest:request];
    433 
    434     TestWebKitAPI::Util::run(&testedControlsManagerAfterPlaying);
    435     TestWebKitAPI::Util::run(&receivedScriptMessage);
     384    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 1600, 800));
     385
     386    [webView loadTestPageNamed:@"skinny-autoplaying-video-with-audio" andExpectControlsManager:NO afterReceivingMessage:@"playing"];
    436387}
    437388
    438389TEST(VideoControlsManager, VideoControlsManagerFullSizeVideoInWideMainFrame)
    439390{
    440     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
    441     configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
    442     RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 1600, 800) configuration:configuration.get()]);
    443     RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
    444     [[window contentView] addSubview:webView.get()];
    445 
    446     RetainPtr<MediaPlaybackMessageHandler> handler = adoptNS([[MediaPlaybackMessageHandler alloc] initWithWKWebView:webView.get() finalMessageString:@"playing"]);
    447     [[configuration userContentController] addScriptMessageHandler:handler.get() name:@"playingHandler"];
    448     [handler setExpectedToHaveControlsManager:YES];
    449 
    450     NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"full-size-autoplaying-video-with-audio" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
    451     [webView loadRequest:request];
    452 
    453     TestWebKitAPI::Util::run(&testedControlsManagerAfterPlaying);
    454     TestWebKitAPI::Util::run(&receivedScriptMessage);
     391    RetainPtr<VideoControlsManagerTestWebView*> webView = setUpWebViewForTestingVideoControlsManager(NSMakeRect(0, 0, 1600, 800));
     392
     393    [webView loadTestPageNamed:@"full-size-autoplaying-video-with-audio" andExpectControlsManager:YES afterReceivingMessage:@"playing"];
    455394}
    456395
Note: See TracChangeset for help on using the changeset viewer.