⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 280776 in webkit


Ignore:
Timestamp:
Aug 9, 2021, 8:15:19 AM (5 years ago)
Author:
jer.noble@apple.com
Message:

[Cocoa|GPU] platformLayer() not always added to remotedly hosted context; black video while playing
​https://bugs.webkit.org/show_bug.cgi?id=228873
<rdar://81271107>

Reviewed by Eric Carlson.

Source/WebCore:

Coalesce calls to renderingModeChanged() by setting a flag to call renderingModeChanged() in
the next run loop, and call into this coalescing method, setNeedsRenderingModeChanged(),
whenever the MediaPlayerPrivateAVFoundation subclasses create or destroy a video layer
or software painting context. This ensures that the MediaPlayerClient is always notified
when the rendering mode changes, regardless of why that change occurred.

Drive-by fix: replace the call in createVideoLayer() to "callOnMainThread()"
with "ensureOnMainThread()". The call to "callOnMainThread()" was added to avoid a
deadlock in WebKitLegacy, and "ensureOnMainThread()" has the correct behavior there, but
will be synchronous in WebKit.

  • platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp:

(WebCore::MediaPlayerPrivateAVFoundation::setUpVideoRendering):
(WebCore::MediaPlayerPrivateAVFoundation::setNeedsRenderingModeChanged):
(WebCore::MediaPlayerPrivateAVFoundation::renderingModeChanged):
(WebCore::MediaPlayerPrivateAVFoundation::prepareForRendering):

  • platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
  • platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.cpp:

(WebCore::MediaPlayerPrivateAVFoundationCF::destroyVideoLayer):
(WebCore::AVFWrapper::createAVCFVideoLayer):
(WebCore::AVFWrapper::createImageGenerator):
(WebCore::AVFWrapper::destroyImageGenerator):

  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:

(WebCore::MediaPlayerPrivateAVFoundationObjC::createVideoLayer):
(WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerLayer):
(WebCore::MediaPlayerPrivateAVFoundationObjC::destroyVideoLayer):
(WebCore::MediaPlayerPrivateAVFoundationObjC::createVideoOutput):
(WebCore::MediaPlayerPrivateAVFoundationObjC::destroyVideoOutput):

Source/WebKit:

Currently, the RemoteMediaPlayerProxy will insert its MediaPlayer's platformLayer() into the
remotely hosted context when it receives a "firstVideoFrameAvailable()" notification.
However this is problematic for two reasons: AVPlayerLayer may not become "ready for
playback" when it is not in an active CALayer heirarchy, and for
MediaPlayerPrivateAVFoundation, firstVideoFrameAvailable() is only fired during
updateStates(), which is not guaranteed to occur unless another state change occurs.

Instead, reparent the MediaPlayer's platformLayer() when we receive a
mediaPlayerRenderingModeChanged() notification, which notifies the client whenever a
platformLayer() is created or destroyed.

  • GPUProcess/media/RemoteMediaPlayerProxy.cpp:
  • GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm:

(WebKit::RemoteMediaPlayerProxy::mediaPlayerFirstVideoFrameAvailable):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged):

Location:
trunk/Source
Files:
8 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/WebCore/ChangeLog

    r280775 r280776  
     12021-08-09  Jer Noble  <jer.noble@apple.com>
     2
     3        [Cocoa|GPU] platformLayer() not always added to remotedly hosted context; black video while playing
     4        https://bugs.webkit.org/show_bug.cgi?id=228873
     5        <rdar://81271107>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Coalesce calls to renderingModeChanged() by setting a flag to call renderingModeChanged() in
     10        the next run loop, and call into this coalescing method, setNeedsRenderingModeChanged(),
     11        whenever the MediaPlayerPrivateAVFoundation subclasses create or destroy a video layer
     12        or software painting context. This ensures that the MediaPlayerClient is always notified
     13        when the rendering mode changes, regardless of why that change occurred.
     14
     15        Drive-by fix: replace the call in createVideoLayer() to "callOnMainThread()"
     16        with "ensureOnMainThread()". The call to "callOnMainThread()" was added to avoid a
     17        deadlock in WebKitLegacy, and "ensureOnMainThread()" has the correct behavior there, but
     18        will be synchronous in WebKit.
     19
     20        * platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp:
     21        (WebCore::MediaPlayerPrivateAVFoundation::setUpVideoRendering):
     22        (WebCore::MediaPlayerPrivateAVFoundation::setNeedsRenderingModeChanged):
     23        (WebCore::MediaPlayerPrivateAVFoundation::renderingModeChanged):
     24        (WebCore::MediaPlayerPrivateAVFoundation::prepareForRendering):
     25        * platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
     26        * platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.cpp:
     27        (WebCore::MediaPlayerPrivateAVFoundationCF::destroyVideoLayer):
     28        (WebCore::AVFWrapper::createAVCFVideoLayer):
     29        (WebCore::AVFWrapper::createImageGenerator):
     30        (WebCore::AVFWrapper::destroyImageGenerator):
     31        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     32        (WebCore::MediaPlayerPrivateAVFoundationObjC::createVideoLayer):
     33        (WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerLayer):
     34        (WebCore::MediaPlayerPrivateAVFoundationObjC::destroyVideoLayer):
     35        (WebCore::MediaPlayerPrivateAVFoundationObjC::createVideoOutput):
     36        (WebCore::MediaPlayerPrivateAVFoundationObjC::destroyVideoOutput):
     37
    1382021-08-09  Alicia Boya García  <aboya@igalia.com>
    239

  • trunk/Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp

    r280723 r280776  
    143143    // If using a movie layer, inform the client so the compositing tree is updated.
    144144    if (currentMode == MediaRenderingToLayer || preferredMode == MediaRenderingToLayer)
    145         m_player->renderingModeChanged();
     145        setNeedsRenderingModeChanged();
     146}
     147
     148void MediaPlayerPrivateAVFoundation::setNeedsRenderingModeChanged()
     149{
     150    if (m_needsRenderingModeChanged)
     151        return;
     152    m_needsRenderingModeChanged = true;
     153
     154    queueTaskOnEventLoop([weakThis = makeWeakPtr(*this)] {
     155        if (weakThis)
     156            weakThis->renderingModeChanged();
     157    });
     158}
     159
     160void MediaPlayerPrivateAVFoundation::renderingModeChanged()
     161{
     162    ASSERT(m_needsRenderingModeChanged);
     163    m_needsRenderingModeChanged = false;
     164    m_player->renderingModeChanged();
    146165}
    147166
    … …  
    444463
    445464    if (currentRenderingMode() == MediaRenderingToLayer || preferredRenderingMode() == MediaRenderingToLayer)
    446         m_player->renderingModeChanged();
     465        setNeedsRenderingModeChanged();
    447466}
    448467

  • trunk/Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h

    r280624 r280776  
    321321    const URL& resolvedURL() const { return m_resolvedURL; }
    322322
     323    void setNeedsRenderingModeChanged();
     324    void renderingModeChanged();
     325
    323326private:
    324327    MediaPlayer* m_player;
    … …  
    369372    bool m_shouldMaintainAspectRatio;
    370373    bool m_seeking;
     374    bool m_needsRenderingModeChanged { false };
    371375};
    372376

  • trunk/Source/WebCore/platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.cpp

    r280624 r280776  
    443443        return;
    444444
    445     if (m_avfWrapper)
    446         m_avfWrapper->createImageGenerator();
     445    if (!m_avfWrapper)
     446        return;
     447
     448    m_avfWrapper->createImageGenerator();
     449    setNeedsRenderingModeChanged();
    447450}
    448451
    … …  
    450453{
    451454    ASSERT(isMainThread());
    452     if (m_avfWrapper)
    453         m_avfWrapper->destroyImageGenerator();
     455    if (!m_avfWrapper)
     456        return;
     457
     458    m_avfWrapper->destroyImageGenerator();
     459    setNeedsRenderingModeChanged();
    454460}
    455461
    … …  
    459465    ASSERT(supportsAcceleratedRendering());
    460466
    461     if (m_avfWrapper)
    462         m_avfWrapper->createAVCFVideoLayer();
     467    if (!m_avfWrapper)
     468        return;
     469
     470    m_avfWrapper->createAVCFVideoLayer();
     471    setNeedsRenderingModeChanged();
    463472}
    464473
    … …  
    467476    ASSERT(isMainThread());
    468477    LOG(Media, "MediaPlayerPrivateAVFoundationCF::destroyVideoLayer(%p) - destroying %p", this, videoLayer(m_avfWrapper));
    469     if (m_avfWrapper)
    470         m_avfWrapper->destroyVideoLayer();
     478    if (!m_avfWrapper)
     479        return;
     480
     481    m_avfWrapper->destroyVideoLayer();
     482    setNeedsRenderingModeChanged();
    471483}
    472484

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm

    r280766 r280776  
    607607        return;
    608608
    609     callOnMainThread([this, weakThis = makeWeakPtr(*this)] {
     609    ensureOnMainThread([this, weakThis = makeWeakPtr(*this)] {
    610610        if (!weakThis)
    611611            return;
    … …  
    620620        if (!m_videoOutput)
    621621            createVideoOutput();
    622 
    623         player()->renderingModeChanged();
    624622    });
    625623}
    … …  
    646644        [m_videoLayer setPIPModeEnabled:(player()->fullscreenMode() & MediaPlayer::VideoFullscreenModePictureInPicture)];
    647645#endif
     646
     647    setNeedsRenderingModeChanged();
    648648}
    649649
    … …  
    660660
    661661    m_videoLayer = nil;
     662
     663    setNeedsRenderingModeChanged();
    662664}
    663665
    … …  
    24332435
    24342436    [m_avPlayerItem.get() addOutput:m_videoOutput.get()];
     2437
     2438    setNeedsRenderingModeChanged();
    24352439}
    24362440
    … …  
    24462450
    24472451    m_videoOutput = 0;
     2452
     2453    setNeedsRenderingModeChanged();
    24482454}
    24492455

  • trunk/Source/WebKit/ChangeLog

    r280774 r280776  
     12021-08-09  Jer Noble  <jer.noble@apple.com>
     2
     3        [Cocoa|GPU] platformLayer() not always added to remotedly hosted context; black video while playing
     4        https://bugs.webkit.org/show_bug.cgi?id=228873
     5        <rdar://81271107>
     6
     7        Reviewed by Eric Carlson.
     8
     9        Currently, the RemoteMediaPlayerProxy will insert its MediaPlayer's platformLayer() into the
     10        remotely hosted context when it receives a "firstVideoFrameAvailable()" notification.
     11        However this is problematic for two reasons: AVPlayerLayer may not become "ready for
     12        playback" when it is not in an active CALayer heirarchy, and for
     13        MediaPlayerPrivateAVFoundation, firstVideoFrameAvailable() is only fired during
     14        updateStates(), which is not guaranteed to occur unless another state change occurs.
     15
     16        Instead, reparent the MediaPlayer's platformLayer() when we receive a
     17        mediaPlayerRenderingModeChanged() notification, which notifies the client whenever a
     18        platformLayer() is created or destroyed.
     19
     20        * GPUProcess/media/RemoteMediaPlayerProxy.cpp:
     21        * GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm:
     22        (WebKit::RemoteMediaPlayerProxy::mediaPlayerFirstVideoFrameAvailable):
     23        (WebKit::RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged):
     24
    1252021-08-09  Imanol Fernandez  <ifernandez@igalia.com>
    226

  • trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp

    r280308 r280776  
    519519}
    520520
     521#if !PLATFORM(COCOA)
    521522void RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged()
    522523{
    523524    m_webProcessConnection->send(Messages::MediaPlayerPrivateRemote::RenderingModeChanged(), m_id);
    524525}
     526#endif
    525527
    526528TrackPrivateRemoteIdentifier RemoteMediaPlayerProxy::addRemoteAudioTrackProxy(WebCore::AudioTrackPrivate& track)

  • trunk/Source/WebKit/GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm

    r279221 r280776  
    6767{
    6868    // Initially the size of the platformLayer may be 0x0 because we do not provide mediaPlayerContentBoxRect() in this class.
    69     m_inlineLayerHostingContext->setRootLayer(m_player->platformLayer());
    7069    setVideoInlineSizeIfPossible(*m_inlineLayerHostingContext, m_videoInlineSize);
    7170    m_webProcessConnection->send(Messages::MediaPlayerPrivateRemote::FirstVideoFrameAvailable(), m_id);
    7271}
    7372
     73void RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged()
     74{
     75    m_inlineLayerHostingContext->setRootLayer(m_player->platformLayer());
     76    m_webProcessConnection->send(Messages::MediaPlayerPrivateRemote::RenderingModeChanged(), m_id);
     77}
    7478void RemoteMediaPlayerProxy::setVideoInlineSizeFenced(const WebCore::FloatSize& size, const WTF::MachSendRight& machSendRight)
    7579{
Note: See TracChangeset for help on using the changeset viewer.