⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 284834 in webkit


Ignore:
Timestamp:
Oct 25, 2021, 2:51:10 PM (5 years ago)
Author:
Alan Coon
Message:

Cherry-pick r284102. rdar://problem/84630200

WebGL video texture upload is very slow due to excessive transfer of the video pixel buffer
​https://bugs.webkit.org/show_bug.cgi?id=231425

Patch by Kimmo Kinnunen <​kkinnunen@apple.com> on 2021-10-13
Reviewed by Youenn Fablet.

Typical WebGL content requests the videos to be uploaded to a texture
once per render loop update, even though the video has not changed.
The video pixel buffer is slow to transfer across IPC. It should be transferred
only when it has changed.

MediaPlayerPrivateAVFoundationObjC and MediaPlayerPrivateMediaSourceAVFObjC hold
the last requested pixel buffer ref. They will update it or discard it only
when pixelBufferForCurrentTime() is called.

Cache the pixel buffer to MediaPlayerPrivateRemote (GPUP side proxy)
Cache the pixel buffer to RemoteMediaPlayerProxy (WebP side proxy)

The caches increase the total memory use only during the duration
of the call, since after the ref has been updated, it refers
to the one always held in the original objects (MediaPlayerPrivateAVFoundationObjC
and MediaPlayerPrivateMediaSourceAVFObjC)

No new tests, fixes a perf regression wrt GPUP media.

  • GPUProcess/media/RemoteMediaPlayerProxy.cpp: (WebKit::RemoteMediaPlayerProxy::invalidate):
  • GPUProcess/media/RemoteMediaPlayerProxy.h:
  • GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
  • GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm: (WebKit::RemoteMediaPlayerProxy::nativeImageForCurrentTime): (WebKit::RemoteMediaPlayerProxy::pixelBufferForCurrentTimeIfChanged): (WebKit::RemoteMediaPlayerProxy::pixelBufferForCurrentTime): Deleted.
  • WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
  • WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm: (WebKit::MediaPlayerPrivateRemote::pixelBufferForCurrentTime):

git-svn-id: ​https://svn.webkit.org/repository/webkit/trunk@284102 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Location:
branches/safari-612-branch/Source/WebKit
Files:
7 edited

Legend:

Unmodified
Added
Removed

  • branches/safari-612-branch/Source/WebKit/ChangeLog

    r284831 r284834  
     12021-10-25  Null  <null@apple.com>
     2
     3        Cherry-pick r284102. rdar://problem/84630200
     4
     5    WebGL video texture upload is very slow due to excessive transfer of the video pixel buffer
     6    https://bugs.webkit.org/show_bug.cgi?id=231425
     7   
     8    Patch by Kimmo Kinnunen <kkinnunen@apple.com> on 2021-10-13
     9    Reviewed by Youenn Fablet.
     10   
     11    Typical WebGL content requests the videos to be uploaded to a texture
     12    once per render loop update, even though the video has not changed.
     13    The video pixel buffer is slow to transfer across IPC. It should be transferred
     14    only when it has changed.
     15   
     16    MediaPlayerPrivateAVFoundationObjC and MediaPlayerPrivateMediaSourceAVFObjC hold
     17    the last requested pixel buffer ref. They will update it or discard it only
     18    when pixelBufferForCurrentTime() is called.
     19   
     20    Cache the pixel buffer to MediaPlayerPrivateRemote (GPUP side proxy)
     21    Cache the pixel buffer to RemoteMediaPlayerProxy (WebP side proxy)
     22   
     23    The caches increase the total memory use only during the duration
     24    of the call, since after the ref has been updated, it refers
     25    to the one always held in the original objects (MediaPlayerPrivateAVFoundationObjC
     26    and MediaPlayerPrivateMediaSourceAVFObjC)
     27   
     28    No new tests, fixes a perf regression wrt GPUP media.
     29   
     30    * GPUProcess/media/RemoteMediaPlayerProxy.cpp:
     31    (WebKit::RemoteMediaPlayerProxy::invalidate):
     32    * GPUProcess/media/RemoteMediaPlayerProxy.h:
     33    * GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
     34    * GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm:
     35    (WebKit::RemoteMediaPlayerProxy::nativeImageForCurrentTime):
     36    (WebKit::RemoteMediaPlayerProxy::pixelBufferForCurrentTimeIfChanged):
     37    (WebKit::RemoteMediaPlayerProxy::pixelBufferForCurrentTime): Deleted.
     38    * WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
     39    * WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm:
     40    (WebKit::MediaPlayerPrivateRemote::pixelBufferForCurrentTime):
     41   
     42    git-svn-id: https://svn.webkit.org/repository/webkit/trunk@284102 268f45cc-cd09-0410-ab3c-d52691b4dbfc
     43
     44    2021-10-13  Kimmo Kinnunen  <kkinnunen@apple.com>
     45
     46            WebGL video texture upload is very slow due to excessive transfer of the video pixel buffer
     47            https://bugs.webkit.org/show_bug.cgi?id=231425
     48
     49            Reviewed by Youenn Fablet.
     50
     51            Typical WebGL content requests the videos to be uploaded to a texture
     52            once per render loop update, even though the video has not changed.
     53            The video pixel buffer is slow to transfer across IPC. It should be transferred
     54            only when it has changed.
     55
     56            MediaPlayerPrivateAVFoundationObjC and MediaPlayerPrivateMediaSourceAVFObjC hold
     57            the last requested pixel buffer ref. They will update it or discard it only
     58            when pixelBufferForCurrentTime() is called.
     59
     60            Cache the pixel buffer to MediaPlayerPrivateRemote (GPUP side proxy)
     61            Cache the pixel buffer to RemoteMediaPlayerProxy (WebP side proxy)
     62
     63            The caches increase the total memory use only during the duration
     64            of the call, since after the ref has been updated, it refers
     65            to the one always held in the original objects (MediaPlayerPrivateAVFoundationObjC
     66            and MediaPlayerPrivateMediaSourceAVFObjC)
     67
     68            No new tests, fixes a perf regression wrt GPUP media.
     69
     70            * GPUProcess/media/RemoteMediaPlayerProxy.cpp:
     71            (WebKit::RemoteMediaPlayerProxy::invalidate):
     72            * GPUProcess/media/RemoteMediaPlayerProxy.h:
     73            * GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
     74            * GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm:
     75            (WebKit::RemoteMediaPlayerProxy::nativeImageForCurrentTime):
     76            (WebKit::RemoteMediaPlayerProxy::pixelBufferForCurrentTimeIfChanged):
     77            (WebKit::RemoteMediaPlayerProxy::pixelBufferForCurrentTime): Deleted.
     78            * WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
     79            * WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm:
     80            (WebKit::MediaPlayerPrivateRemote::pixelBufferForCurrentTime):
     81
    1822021-10-25  Null  <null@apple.com>
    283

  • branches/safari-612-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp

    r282945 r284834  
    105105    }
    106106    m_renderingResourcesRequest = { };
     107#if USE(AVFOUNDATION)
     108    m_pixelBufferForCurrentTime = nullptr;
     109#endif
    107110}
    108111

  • branches/safari-612-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h

    r281227 r284834  
    312312#endif
    313313#if USE(AVFOUNDATION)
    314     void pixelBufferForCurrentTime(CompletionHandler<void(RetainPtr<CVPixelBufferRef>&&)>&&);
     314    void pixelBufferForCurrentTimeIfChanged(CompletionHandler<void(std::optional<RetainPtr<CVPixelBufferRef>>&&)>&&);
    315315#endif
    316316
    … …  
    367367
    368368    bool m_observingTimeChanges { false };
    369 
     369#if USE(AVFOUNDATION)
     370    RetainPtr<CVPixelBufferRef> m_pixelBufferForCurrentTime;
     371#endif
    370372#if !RELEASE_LOG_DISABLED
    371373    const Logger& m_logger;

  • branches/safari-612-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in

    r281227 r284834  
    127127#endif
    128128#if USE(AVFOUNDATION)
    129     PixelBufferForCurrentTime() -> (RetainPtr<CVPixelBufferRef> pixelBuffer) Synchronous
     129    PixelBufferForCurrentTimeIfChanged() -> (std::optional<RetainPtr<CVPixelBufferRef>> pixelBuffer) Synchronous
    130130#endif
    131131

  • branches/safari-612-branch/Source/WebKit/GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm

    r280776 r284834  
    112112}
    113113
    114 void RemoteMediaPlayerProxy::pixelBufferForCurrentTime(CompletionHandler<void(RetainPtr<CVPixelBufferRef>&&)>&& completionHandler)
     114#if USE(AVFOUNDATION)
     115void RemoteMediaPlayerProxy::pixelBufferForCurrentTimeIfChanged(CompletionHandler<void(std::optional<RetainPtr<CVPixelBufferRef>>&&)>&& completionHandler)
    115116{
    116     RetainPtr<CVPixelBufferRef> result;
     117    std::optional<RetainPtr<CVPixelBufferRef>> result;
     118    RetainPtr<CVPixelBufferRef> pixelBuffer;
    117119    if (m_player)
    118         result = m_player->pixelBufferForCurrentTime();
     120        pixelBuffer = m_player->pixelBufferForCurrentTime();
     121    if (m_pixelBufferForCurrentTime != pixelBuffer) {
     122        result = pixelBuffer;
     123        m_pixelBufferForCurrentTime = WTFMove(pixelBuffer);
     124    }
    119125    completionHandler(WTFMove(result));
    120126}
     127#endif
    121128
    122129} // namespace WebKit

  • branches/safari-612-branch/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h

    r282945 r284834  
    446446    bool m_timeIsProgressing { false };
    447447    bool m_renderingCanBeAccelerated { false };
     448#if USE(AVFOUNDATION)
     449    RetainPtr<CVPixelBufferRef> m_pixelBufferForCurrentTime;
     450#endif
    448451};
    449452

  • branches/safari-612-branch/Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm

    r281157 r284834  
    6767}
    6868
     69#if USE(AVFOUNDATION)
    6970RetainPtr<CVPixelBufferRef> MediaPlayerPrivateRemote::pixelBufferForCurrentTime()
    7071{
    71 
    72     RetainPtr<CVPixelBufferRef> result;
    73     if (!connection().sendSync(Messages::RemoteMediaPlayerProxy::PixelBufferForCurrentTime(), Messages::RemoteMediaPlayerProxy::PixelBufferForCurrentTime::Reply(result), m_id))
     72    std::optional<RetainPtr<CVPixelBufferRef>> result;
     73    if (!connection().sendSync(Messages::RemoteMediaPlayerProxy::PixelBufferForCurrentTimeIfChanged(), Messages::RemoteMediaPlayerProxy::PixelBufferForCurrentTimeIfChanged::Reply(result), m_id))
    7474        return nullptr;
    75     return result;
     75    if (result)
     76        m_pixelBufferForCurrentTime = WTFMove(*result);
     77    return m_pixelBufferForCurrentTime;
    7678}
     79#endif
    7780
    7881} // namespace WebKit
Note: See TracChangeset for help on using the changeset viewer.