⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 280998 in webkit


Ignore:
Timestamp:
Aug 12, 2021, 5:04:20 PM (5 years ago)
Author:
Alan Coon
Message:

Cherry-pick r280776. rdar://problem/81861548

[Cocoa|GPU] platformLayer() not always added to remotedly hosted context; black video while playing
​https://bugs.webkit.org/show_bug.cgi?id=228873
<rdar://81271107>

Reviewed by Eric Carlson.

Source/WebCore:

Coalesce calls to renderingModeChanged() by setting a flag to call renderingModeChanged() in
the next run loop, and call into this coalescing method, setNeedsRenderingModeChanged(),
whenever the MediaPlayerPrivateAVFoundation subclasses create or destroy a video layer
or software painting context. This ensures that the MediaPlayerClient is always notified
when the rendering mode changes, regardless of why that change occurred.

Drive-by fix: replace the call in createVideoLayer() to "callOnMainThread()"
with "ensureOnMainThread()". The call to "callOnMainThread()" was added to avoid a
deadlock in WebKitLegacy, and "ensureOnMainThread()" has the correct behavior there, but
will be synchronous in WebKit.

  • platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp: (WebCore::MediaPlayerPrivateAVFoundation::setUpVideoRendering): (WebCore::MediaPlayerPrivateAVFoundation::setNeedsRenderingModeChanged): (WebCore::MediaPlayerPrivateAVFoundation::renderingModeChanged): (WebCore::MediaPlayerPrivateAVFoundation::prepareForRendering):
  • platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
  • platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.cpp: (WebCore::MediaPlayerPrivateAVFoundationCF::destroyVideoLayer): (WebCore::AVFWrapper::createAVCFVideoLayer): (WebCore::AVFWrapper::createImageGenerator): (WebCore::AVFWrapper::destroyImageGenerator):
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm: (WebCore::MediaPlayerPrivateAVFoundationObjC::createVideoLayer): (WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerLayer): (WebCore::MediaPlayerPrivateAVFoundationObjC::destroyVideoLayer): (WebCore::MediaPlayerPrivateAVFoundationObjC::createVideoOutput): (WebCore::MediaPlayerPrivateAVFoundationObjC::destroyVideoOutput):

Source/WebKit:

Currently, the RemoteMediaPlayerProxy will insert its MediaPlayer's platformLayer() into the
remotely hosted context when it receives a "firstVideoFrameAvailable()" notification.
However this is problematic for two reasons: AVPlayerLayer may not become "ready for
playback" when it is not in an active CALayer heirarchy, and for
MediaPlayerPrivateAVFoundation, firstVideoFrameAvailable() is only fired during
updateStates(), which is not guaranteed to occur unless another state change occurs.

Instead, reparent the MediaPlayer's platformLayer() when we receive a
mediaPlayerRenderingModeChanged() notification, which notifies the client whenever a
platformLayer() is created or destroyed.

  • GPUProcess/media/RemoteMediaPlayerProxy.cpp:
  • GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm: (WebKit::RemoteMediaPlayerProxy::mediaPlayerFirstVideoFrameAvailable): (WebKit::RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged):

git-svn-id: ​https://svn.webkit.org/repository/webkit/trunk@280776 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Location:
branches/safari-612.1.27.0.6-branch/Source
Files:
8 edited

Legend:

Unmodified
Added
Removed

  • branches/safari-612.1.27.0.6-branch/Source/WebCore/ChangeLog

    r280857 r280998  
     12021-08-12  Russell Epstein  <repstein@apple.com>
     2
     3        Cherry-pick r280776. rdar://problem/81861548
     4
     5    [Cocoa|GPU] platformLayer() not always added to remotedly hosted context; black video while playing
     6    https://bugs.webkit.org/show_bug.cgi?id=228873
     7    <rdar://81271107>
     8   
     9    Reviewed by Eric Carlson.
     10   
     11    Source/WebCore:
     12   
     13    Coalesce calls to renderingModeChanged() by setting a flag to call renderingModeChanged() in
     14    the next run loop, and call into this coalescing method, setNeedsRenderingModeChanged(),
     15    whenever the MediaPlayerPrivateAVFoundation subclasses create or destroy a video layer
     16    or software painting context. This ensures that the MediaPlayerClient is always notified
     17    when the rendering mode changes, regardless of why that change occurred.
     18   
     19    Drive-by fix: replace the call in createVideoLayer() to "callOnMainThread()"
     20    with "ensureOnMainThread()". The call to "callOnMainThread()" was added to avoid a
     21    deadlock in WebKitLegacy, and "ensureOnMainThread()" has the correct behavior there, but
     22    will be synchronous in WebKit.
     23   
     24    * platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp:
     25    (WebCore::MediaPlayerPrivateAVFoundation::setUpVideoRendering):
     26    (WebCore::MediaPlayerPrivateAVFoundation::setNeedsRenderingModeChanged):
     27    (WebCore::MediaPlayerPrivateAVFoundation::renderingModeChanged):
     28    (WebCore::MediaPlayerPrivateAVFoundation::prepareForRendering):
     29    * platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
     30    * platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.cpp:
     31    (WebCore::MediaPlayerPrivateAVFoundationCF::destroyVideoLayer):
     32    (WebCore::AVFWrapper::createAVCFVideoLayer):
     33    (WebCore::AVFWrapper::createImageGenerator):
     34    (WebCore::AVFWrapper::destroyImageGenerator):
     35    * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     36    (WebCore::MediaPlayerPrivateAVFoundationObjC::createVideoLayer):
     37    (WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerLayer):
     38    (WebCore::MediaPlayerPrivateAVFoundationObjC::destroyVideoLayer):
     39    (WebCore::MediaPlayerPrivateAVFoundationObjC::createVideoOutput):
     40    (WebCore::MediaPlayerPrivateAVFoundationObjC::destroyVideoOutput):
     41   
     42    Source/WebKit:
     43   
     44    Currently, the RemoteMediaPlayerProxy will insert its MediaPlayer's platformLayer() into the
     45    remotely hosted context when it receives a "firstVideoFrameAvailable()" notification.
     46    However this is problematic for two reasons: AVPlayerLayer may not become "ready for
     47    playback" when it is not in an active CALayer heirarchy, and for
     48    MediaPlayerPrivateAVFoundation, firstVideoFrameAvailable() is only fired during
     49    updateStates(), which is not guaranteed to occur unless another state change occurs.
     50   
     51    Instead, reparent the MediaPlayer's platformLayer() when we receive a
     52    mediaPlayerRenderingModeChanged() notification, which notifies the client whenever a
     53    platformLayer() is created or destroyed.
     54   
     55    * GPUProcess/media/RemoteMediaPlayerProxy.cpp:
     56    * GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm:
     57    (WebKit::RemoteMediaPlayerProxy::mediaPlayerFirstVideoFrameAvailable):
     58    (WebKit::RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged):
     59   
     60   
     61    git-svn-id: https://svn.webkit.org/repository/webkit/trunk@280776 268f45cc-cd09-0410-ab3c-d52691b4dbfc
     62
     63    2021-08-09  Jer Noble  <jer.noble@apple.com>
     64
     65            [Cocoa|GPU] platformLayer() not always added to remotedly hosted context; black video while playing
     66            https://bugs.webkit.org/show_bug.cgi?id=228873
     67            <rdar://81271107>
     68
     69            Reviewed by Eric Carlson.
     70
     71            Coalesce calls to renderingModeChanged() by setting a flag to call renderingModeChanged() in
     72            the next run loop, and call into this coalescing method, setNeedsRenderingModeChanged(),
     73            whenever the MediaPlayerPrivateAVFoundation subclasses create or destroy a video layer
     74            or software painting context. This ensures that the MediaPlayerClient is always notified
     75            when the rendering mode changes, regardless of why that change occurred.
     76
     77            Drive-by fix: replace the call in createVideoLayer() to "callOnMainThread()"
     78            with "ensureOnMainThread()". The call to "callOnMainThread()" was added to avoid a
     79            deadlock in WebKitLegacy, and "ensureOnMainThread()" has the correct behavior there, but
     80            will be synchronous in WebKit.
     81
     82            * platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp:
     83            (WebCore::MediaPlayerPrivateAVFoundation::setUpVideoRendering):
     84            (WebCore::MediaPlayerPrivateAVFoundation::setNeedsRenderingModeChanged):
     85            (WebCore::MediaPlayerPrivateAVFoundation::renderingModeChanged):
     86            (WebCore::MediaPlayerPrivateAVFoundation::prepareForRendering):
     87            * platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
     88            * platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.cpp:
     89            (WebCore::MediaPlayerPrivateAVFoundationCF::destroyVideoLayer):
     90            (WebCore::AVFWrapper::createAVCFVideoLayer):
     91            (WebCore::AVFWrapper::createImageGenerator):
     92            (WebCore::AVFWrapper::destroyImageGenerator):
     93            * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     94            (WebCore::MediaPlayerPrivateAVFoundationObjC::createVideoLayer):
     95            (WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerLayer):
     96            (WebCore::MediaPlayerPrivateAVFoundationObjC::destroyVideoLayer):
     97            (WebCore::MediaPlayerPrivateAVFoundationObjC::createVideoOutput):
     98            (WebCore::MediaPlayerPrivateAVFoundationObjC::destroyVideoOutput):
     99
    11002021-08-10  Russell Epstein  <repstein@apple.com>
    2101

  • branches/safari-612.1.27.0.6-branch/Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp

    r280786 r280998  
    143143    // If using a movie layer, inform the client so the compositing tree is updated.
    144144    if (currentMode == MediaRenderingToLayer || preferredMode == MediaRenderingToLayer)
    145         m_player->renderingModeChanged();
     145        setNeedsRenderingModeChanged();
     146}
     147
     148void MediaPlayerPrivateAVFoundation::setNeedsRenderingModeChanged()
     149{
     150    if (m_needsRenderingModeChanged)
     151        return;
     152    m_needsRenderingModeChanged = true;
     153
     154    queueTaskOnEventLoop([weakThis = makeWeakPtr(*this)] {
     155        if (weakThis)
     156            weakThis->renderingModeChanged();
     157    });
     158}
     159
     160void MediaPlayerPrivateAVFoundation::renderingModeChanged()
     161{
     162    ASSERT(m_needsRenderingModeChanged);
     163    m_needsRenderingModeChanged = false;
     164    m_player->renderingModeChanged();
    146165}
    147166
    … …  
    444463
    445464    if (currentRenderingMode() == MediaRenderingToLayer || preferredRenderingMode() == MediaRenderingToLayer)
    446         m_player->renderingModeChanged();
     465        setNeedsRenderingModeChanged();
    447466}
    448467

  • branches/safari-612.1.27.0.6-branch/Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h

    r280785 r280998  
    321321    const URL& resolvedURL() const { return m_resolvedURL; }
    322322
     323    void setNeedsRenderingModeChanged();
     324    void renderingModeChanged();
     325
    323326private:
    324327    MediaPlayer* m_player;
    … …  
    369372    bool m_shouldMaintainAspectRatio;
    370373    bool m_seeking;
     374    bool m_needsRenderingModeChanged { false };
    371375};
    372376

  • branches/safari-612.1.27.0.6-branch/Source/WebCore/platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.cpp

    r280785 r280998  
    443443        return;
    444444
    445     if (m_avfWrapper)
    446         m_avfWrapper->createImageGenerator();
     445    if (!m_avfWrapper)
     446        return;
     447
     448    m_avfWrapper->createImageGenerator();
     449    setNeedsRenderingModeChanged();
    447450}
    448451
    … …  
    450453{
    451454    ASSERT(isMainThread());
    452     if (m_avfWrapper)
    453         m_avfWrapper->destroyImageGenerator();
     455    if (!m_avfWrapper)
     456        return;
     457
     458    m_avfWrapper->destroyImageGenerator();
     459    setNeedsRenderingModeChanged();
    454460}
    455461
    … …  
    459465    ASSERT(supportsAcceleratedRendering());
    460466
    461     if (m_avfWrapper)
    462         m_avfWrapper->createAVCFVideoLayer();
     467    if (!m_avfWrapper)
     468        return;
     469
     470    m_avfWrapper->createAVCFVideoLayer();
     471    setNeedsRenderingModeChanged();
    463472}
    464473
    … …  
    467476    ASSERT(isMainThread());
    468477    LOG(Media, "MediaPlayerPrivateAVFoundationCF::destroyVideoLayer(%p) - destroying %p", this, videoLayer(m_avfWrapper));
    469     if (m_avfWrapper)
    470         m_avfWrapper->destroyVideoLayer();
     478    if (!m_avfWrapper)
     479        return;
     480
     481    m_avfWrapper->destroyVideoLayer();
     482    setNeedsRenderingModeChanged();
    471483}
    472484

  • branches/safari-612.1.27.0.6-branch/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm

    r280786 r280998  
    607607        return;
    608608
    609     callOnMainThread([this, weakThis = makeWeakPtr(*this)] {
     609    ensureOnMainThread([this, weakThis = makeWeakPtr(*this)] {
    610610        if (!weakThis)
    611611            return;
    … …  
    620620        if (!m_videoOutput)
    621621            createVideoOutput();
    622 
    623         player()->renderingModeChanged();
    624622    });
    625623}
    … …  
    646644        [m_videoLayer setPIPModeEnabled:(player()->fullscreenMode() & MediaPlayer::VideoFullscreenModePictureInPicture)];
    647645#endif
     646
     647    setNeedsRenderingModeChanged();
    648648}
    649649
    … …  
    660660
    661661    m_videoLayer = nil;
     662
     663    setNeedsRenderingModeChanged();
    662664}
    663665
    … …  
    24302432
    24312433    [m_avPlayerItem.get() addOutput:m_videoOutput.get()];
     2434
     2435    setNeedsRenderingModeChanged();
    24322436}
    24332437
    … …  
    24432447
    24442448    m_videoOutput = 0;
     2449
     2450    setNeedsRenderingModeChanged();
    24452451}
    24462452

  • branches/safari-612.1.27.0.6-branch/Source/WebKit/ChangeLog

    r280788 r280998  
     12021-08-12  Russell Epstein  <repstein@apple.com>
     2
     3        Cherry-pick r280776. rdar://problem/81861548
     4
     5    [Cocoa|GPU] platformLayer() not always added to remotedly hosted context; black video while playing
     6    https://bugs.webkit.org/show_bug.cgi?id=228873
     7    <rdar://81271107>
     8   
     9    Reviewed by Eric Carlson.
     10   
     11    Source/WebCore:
     12   
     13    Coalesce calls to renderingModeChanged() by setting a flag to call renderingModeChanged() in
     14    the next run loop, and call into this coalescing method, setNeedsRenderingModeChanged(),
     15    whenever the MediaPlayerPrivateAVFoundation subclasses create or destroy a video layer
     16    or software painting context. This ensures that the MediaPlayerClient is always notified
     17    when the rendering mode changes, regardless of why that change occurred.
     18   
     19    Drive-by fix: replace the call in createVideoLayer() to "callOnMainThread()"
     20    with "ensureOnMainThread()". The call to "callOnMainThread()" was added to avoid a
     21    deadlock in WebKitLegacy, and "ensureOnMainThread()" has the correct behavior there, but
     22    will be synchronous in WebKit.
     23   
     24    * platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp:
     25    (WebCore::MediaPlayerPrivateAVFoundation::setUpVideoRendering):
     26    (WebCore::MediaPlayerPrivateAVFoundation::setNeedsRenderingModeChanged):
     27    (WebCore::MediaPlayerPrivateAVFoundation::renderingModeChanged):
     28    (WebCore::MediaPlayerPrivateAVFoundation::prepareForRendering):
     29    * platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
     30    * platform/graphics/avfoundation/cf/MediaPlayerPrivateAVFoundationCF.cpp:
     31    (WebCore::MediaPlayerPrivateAVFoundationCF::destroyVideoLayer):
     32    (WebCore::AVFWrapper::createAVCFVideoLayer):
     33    (WebCore::AVFWrapper::createImageGenerator):
     34    (WebCore::AVFWrapper::destroyImageGenerator):
     35    * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     36    (WebCore::MediaPlayerPrivateAVFoundationObjC::createVideoLayer):
     37    (WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerLayer):
     38    (WebCore::MediaPlayerPrivateAVFoundationObjC::destroyVideoLayer):
     39    (WebCore::MediaPlayerPrivateAVFoundationObjC::createVideoOutput):
     40    (WebCore::MediaPlayerPrivateAVFoundationObjC::destroyVideoOutput):
     41   
     42    Source/WebKit:
     43   
     44    Currently, the RemoteMediaPlayerProxy will insert its MediaPlayer's platformLayer() into the
     45    remotely hosted context when it receives a "firstVideoFrameAvailable()" notification.
     46    However this is problematic for two reasons: AVPlayerLayer may not become "ready for
     47    playback" when it is not in an active CALayer heirarchy, and for
     48    MediaPlayerPrivateAVFoundation, firstVideoFrameAvailable() is only fired during
     49    updateStates(), which is not guaranteed to occur unless another state change occurs.
     50   
     51    Instead, reparent the MediaPlayer's platformLayer() when we receive a
     52    mediaPlayerRenderingModeChanged() notification, which notifies the client whenever a
     53    platformLayer() is created or destroyed.
     54   
     55    * GPUProcess/media/RemoteMediaPlayerProxy.cpp:
     56    * GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm:
     57    (WebKit::RemoteMediaPlayerProxy::mediaPlayerFirstVideoFrameAvailable):
     58    (WebKit::RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged):
     59   
     60   
     61    git-svn-id: https://svn.webkit.org/repository/webkit/trunk@280776 268f45cc-cd09-0410-ab3c-d52691b4dbfc
     62
     63    2021-08-09  Jer Noble  <jer.noble@apple.com>
     64
     65            [Cocoa|GPU] platformLayer() not always added to remotedly hosted context; black video while playing
     66            https://bugs.webkit.org/show_bug.cgi?id=228873
     67            <rdar://81271107>
     68
     69            Reviewed by Eric Carlson.
     70
     71            Currently, the RemoteMediaPlayerProxy will insert its MediaPlayer's platformLayer() into the
     72            remotely hosted context when it receives a "firstVideoFrameAvailable()" notification.
     73            However this is problematic for two reasons: AVPlayerLayer may not become "ready for
     74            playback" when it is not in an active CALayer heirarchy, and for
     75            MediaPlayerPrivateAVFoundation, firstVideoFrameAvailable() is only fired during
     76            updateStates(), which is not guaranteed to occur unless another state change occurs.
     77
     78            Instead, reparent the MediaPlayer's platformLayer() when we receive a
     79            mediaPlayerRenderingModeChanged() notification, which notifies the client whenever a
     80            platformLayer() is created or destroyed.
     81
     82            * GPUProcess/media/RemoteMediaPlayerProxy.cpp:
     83            * GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm:
     84            (WebKit::RemoteMediaPlayerProxy::mediaPlayerFirstVideoFrameAvailable):
     85            (WebKit::RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged):
     86
    1872021-08-09  Russell Epstein  <repstein@apple.com>
    288

  • branches/safari-612.1.27.0.6-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp

    r280308 r280998  
    519519}
    520520
     521#if !PLATFORM(COCOA)
    521522void RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged()
    522523{
    523524    m_webProcessConnection->send(Messages::MediaPlayerPrivateRemote::RenderingModeChanged(), m_id);
    524525}
     526#endif
    525527
    526528TrackPrivateRemoteIdentifier RemoteMediaPlayerProxy::addRemoteAudioTrackProxy(WebCore::AudioTrackPrivate& track)

  • branches/safari-612.1.27.0.6-branch/Source/WebKit/GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm

    r279221 r280998  
    6767{
    6868    // Initially the size of the platformLayer may be 0x0 because we do not provide mediaPlayerContentBoxRect() in this class.
    69     m_inlineLayerHostingContext->setRootLayer(m_player->platformLayer());
    7069    setVideoInlineSizeIfPossible(*m_inlineLayerHostingContext, m_videoInlineSize);
    7170    m_webProcessConnection->send(Messages::MediaPlayerPrivateRemote::FirstVideoFrameAvailable(), m_id);
    7271}
    7372
     73void RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged()
     74{
     75    m_inlineLayerHostingContext->setRootLayer(m_player->platformLayer());
     76    m_webProcessConnection->send(Messages::MediaPlayerPrivateRemote::RenderingModeChanged(), m_id);
     77}
    7478void RemoteMediaPlayerProxy::setVideoInlineSizeFenced(const WebCore::FloatSize& size, const WTF::MachSendRight& machSendRight)
    7579{
Note: See TracChangeset for help on using the changeset viewer.