⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 283238 in webkit


Ignore:
Timestamp:
Sep 29, 2021, 11:21:58 AM (5 years ago)
Author:
commit-queue@webkit.org
Message:

MediaStream canvas.captureStream() fails for WebGL
​https://bugs.webkit.org/show_bug.cgi?id=230613
<rdar://82307293>

Patch by Kimmo Kinnunen <​kkinnunen@apple.com> on 2021-09-29
Reviewed by Youenn Fablet.

.:

Add a manual test to interact with media stream
captureStream() behavior until the implementation is
testable.

  • ManualTests/mediastream/mediastream-canvas-to-video.html: Added.

Source/WebCore:

Remove previous incorrect behavior of turning on "preservesDrawingBuffer"
and sampling the drawing buffer. The drawing buffer is not what should
ever be touched, rather it's the display buffer that should be available.

Implement sampling of WebGL display buffer for
media stream capture. The upon sampling, obtain a shared
reference to the
display buffer, passes it to media sample consuming process and mark
the WebGL context display buffer non-eligible for recycling as
new drawing buffer.

Currently the implementation is displayed correctly for
WebGL contents that is premultiplied. Unpremultiplied
content is fixed in the associated bugs.

Note: there are bugs preventing robust use of the API,
listed in the associated bugs. As such, testing is not expanded
in this bug.

Tested by
ManualTests/mediastream/mediastream-canvas-to-video.html (added)

​https://webrtc.github.io/samples/src/content/capture/canvas-video/
(Refresh the page or hide the window until MediaPlayer initialization
flakyness lets rendering happen)

The test LayoutTests/fast/mediastream/captureStream/canvas3d.html
cannot yet be used to test the implementation as the rest of the
captureStream() and playback implementation is so flaky.

  • Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp:

(WebCore::CanvasCaptureMediaStreamTrack::Source::canvasChanged):

  • html/HTMLCanvasElement.cpp:

(WebCore::HTMLCanvasElement::toMediaSample):

  • html/canvas/WebGLRenderingContextBase.cpp:

(WebCore::WebGLRenderingContextBase::paintCompositedResultsToMediaSample):

  • html/canvas/WebGLRenderingContextBase.h:

(WebCore::WebGLRenderingContextBase::isPreservingDrawingBuffer const):

  • platform/graphics/GraphicsContextGL.cpp:
  • platform/graphics/GraphicsContextGL.h:
  • platform/graphics/RemoteGraphicsContextGLProxyBase.cpp:

(WebCore::RemoteGraphicsContextGLProxyBase::paintCompositedResultsToMediaSample):

  • platform/graphics/RemoteGraphicsContextGLProxyBase.h:
  • platform/graphics/avfoundation/objc/MediaSampleAVFObjC.h:
  • platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm:

(WebCore::MediaSampleAVFObjC::createImageSample):

  • platform/graphics/cocoa/GraphicsContextGLIOSurfaceSwapChain.cpp:

(WebCore::GraphicsContextGLIOSurfaceSwapChain::present):

  • platform/graphics/cocoa/GraphicsContextGLIOSurfaceSwapChain.h:
  • platform/graphics/cocoa/GraphicsContextGLOpenGLCocoa.mm:

(WebCore::GraphicsContextGLOpenGL::paintCompositedResultsToMediaSample):

  • platform/graphics/cocoa/RemoteGraphicsContextGLProxyBaseCocoa.mm:

(WebCore::RemoteGraphicsContextGLProxyBase::paintCompositedResultsToMediaSample):

  • platform/graphics/opengl/GraphicsContextGLOpenGL.cpp:

(WebCore::GraphicsContextGLOpenGL::paintCompositedResultsToMediaSample):

  • platform/graphics/opengl/GraphicsContextGLOpenGL.h:

LayoutTests:

Add new expectations, a warning was removed.
CONSOLE MESSAGE: Turning drawing buffer preservation for the WebGL canvas being captured

  • fast/mediacapturefromelement/CanvasCaptureMediaStream-webgl-events-expected.txt:
  • fast/mediastream/captureStream/canvas3d-expected.txt:
Location:
trunk
Files:
2 added
21 edited

Legend:

Unmodified
Added
Removed

  • trunk/ChangeLog

    r283175 r283238  
     12021-09-29  Kimmo Kinnunen  <kkinnunen@apple.com>
     2
     3        MediaStream canvas.captureStream() fails for WebGL
     4        https://bugs.webkit.org/show_bug.cgi?id=230613
     5        <rdar://82307293>
     6
     7        Reviewed by Youenn Fablet.
     8
     9        Add a manual test to interact with media stream
     10        captureStream() behavior until the implementation is
     11        testable.
     12
     13        * ManualTests/mediastream/mediastream-canvas-to-video.html: Added.
     14
    1152021-09-28  Carlos Garcia Campos  <cgarcia@igalia.com>
    216

  • trunk/LayoutTests/ChangeLog

    r283235 r283238  
     12021-09-29  Kimmo Kinnunen  <kkinnunen@apple.com>
     2
     3        MediaStream canvas.captureStream() fails for WebGL
     4        https://bugs.webkit.org/show_bug.cgi?id=230613
     5        <rdar://82307293>
     6
     7        Reviewed by Youenn Fablet.
     8
     9        Add new expectations, a warning was removed.
     10        CONSOLE MESSAGE: Turning drawing buffer preservation for the WebGL canvas being captured
     11
     12        * fast/mediacapturefromelement/CanvasCaptureMediaStream-webgl-events-expected.txt:
     13        * fast/mediastream/captureStream/canvas3d-expected.txt:
     14
    1152021-09-29  Youenn Fablet  <youenn@apple.com>
    216

  • trunk/LayoutTests/fast/mediacapturefromelement/CanvasCaptureMediaStream-webgl-events-expected.txt

    r270059 r283238  
    1 CONSOLE MESSAGE: Turning drawing buffer preservation for the WebGL canvas being captured
    21Exercises the potential events on CanvasCaptureMediaStream.
    32

  • trunk/LayoutTests/fast/mediastream/captureStream/canvas3d-expected.txt

    r267644 r283238  
    1 CONSOLE MESSAGE: Turning drawing buffer preservation for the WebGL canvas being captured
    21
    32

  • trunk/Source/WebCore/ChangeLog

    r283237 r283238  
     12021-09-29  Kimmo Kinnunen  <kkinnunen@apple.com>
     2
     3        MediaStream canvas.captureStream() fails for WebGL
     4        https://bugs.webkit.org/show_bug.cgi?id=230613
     5        <rdar://82307293>
     6
     7        Reviewed by Youenn Fablet.
     8
     9        Remove previous incorrect behavior of turning on "preservesDrawingBuffer"
     10        and sampling the drawing buffer. The drawing buffer is not what should
     11        ever be touched, rather it's the display buffer that should be available.
     12
     13        Implement sampling of WebGL display buffer for
     14        media stream capture. The upon sampling, obtain a shared
     15        reference to the
     16        display buffer, passes it to media sample consuming process and mark
     17        the WebGL context display buffer non-eligible for recycling as
     18        new drawing buffer.
     19
     20        Currently the implementation is displayed correctly for
     21        WebGL contents that is premultiplied. Unpremultiplied
     22        content is fixed in the associated bugs.
     23
     24        Note: there are bugs preventing robust use of the API,
     25        listed in the associated bugs. As such, testing is not expanded
     26        in this bug.
     27
     28        Tested by
     29        ManualTests/mediastream/mediastream-canvas-to-video.html (added)
     30
     31        https://webrtc.github.io/samples/src/content/capture/canvas-video/
     32        (Refresh the page or hide the window until MediaPlayer initialization
     33        flakyness lets rendering happen)
     34
     35        The test LayoutTests/fast/mediastream/captureStream/canvas3d.html
     36        cannot yet be used to test the implementation as the rest of the
     37        captureStream() and playback implementation is so flaky.
     38
     39        * Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp:
     40        (WebCore::CanvasCaptureMediaStreamTrack::Source::canvasChanged):
     41        * html/HTMLCanvasElement.cpp:
     42        (WebCore::HTMLCanvasElement::toMediaSample):
     43        * html/canvas/WebGLRenderingContextBase.cpp:
     44        (WebCore::WebGLRenderingContextBase::paintCompositedResultsToMediaSample):
     45        * html/canvas/WebGLRenderingContextBase.h:
     46        (WebCore::WebGLRenderingContextBase::isPreservingDrawingBuffer const):
     47        * platform/graphics/GraphicsContextGL.cpp:
     48        * platform/graphics/GraphicsContextGL.h:
     49        * platform/graphics/RemoteGraphicsContextGLProxyBase.cpp:
     50        (WebCore::RemoteGraphicsContextGLProxyBase::paintCompositedResultsToMediaSample):
     51        * platform/graphics/RemoteGraphicsContextGLProxyBase.h:
     52        * platform/graphics/avfoundation/objc/MediaSampleAVFObjC.h:
     53        * platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm:
     54        (WebCore::MediaSampleAVFObjC::createImageSample):
     55        * platform/graphics/cocoa/GraphicsContextGLIOSurfaceSwapChain.cpp:
     56        (WebCore::GraphicsContextGLIOSurfaceSwapChain::present):
     57        * platform/graphics/cocoa/GraphicsContextGLIOSurfaceSwapChain.h:
     58        * platform/graphics/cocoa/GraphicsContextGLOpenGLCocoa.mm:
     59        (WebCore::GraphicsContextGLOpenGL::paintCompositedResultsToMediaSample):
     60        * platform/graphics/cocoa/RemoteGraphicsContextGLProxyBaseCocoa.mm:
     61        (WebCore::RemoteGraphicsContextGLProxyBase::paintCompositedResultsToMediaSample):
     62        * platform/graphics/opengl/GraphicsContextGLOpenGL.cpp:
     63        (WebCore::GraphicsContextGLOpenGL::paintCompositedResultsToMediaSample):
     64        * platform/graphics/opengl/GraphicsContextGLOpenGL.h:
     65
    1662021-09-29  Jer Noble  <jer.noble@apple.com>
    267

  • trunk/Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp

    r278253 r283238  
    154154    ASSERT_UNUSED(canvas, m_canvas == &canvas);
    155155
    156 #if ENABLE(WEBGL)
    157     // FIXME: We need to preserve drawing buffer as we are currently grabbing frames asynchronously.
    158     // We should instead add an anchor point for both 2d and 3d contexts where canvas will actually paint.
    159     // And call canvas observers from that point.
    160     if (is<WebGLRenderingContextBase>(canvas.renderingContext())) {
    161         auto& context = downcast<WebGLRenderingContextBase>(*canvas.renderingContext());
    162         if (!context.isPreservingDrawingBuffer()) {
    163             canvas.scriptExecutionContext()->addConsoleMessage(MessageSource::JS, MessageLevel::Warning, "Turning drawing buffer preservation for the WebGL canvas being captured"_s);
    164             context.enablePreserveDrawingBuffer();
    165         }
    166     }
    167 #endif
    168 
    169156    // FIXME: We should try to generate the frame at the time the screen is being updated.
    170157    if (m_captureCanvasTimer.isActive())

  • trunk/Source/WebCore/html/HTMLCanvasElement.cpp

    r283049 r283238  
    759759{
    760760#if PLATFORM(COCOA) || USE(GSTREAMER)
     761#if ENABLE(WEBGL)
     762    if (is<WebGLRenderingContextBase>(m_context.get())) {
     763        if (RuntimeEnabledFeatures::sharedFeatures().webAPIStatisticsEnabled())
     764            ResourceLoadObserver::shared().logCanvasRead(document());
     765        return downcast<WebGLRenderingContextBase>(*m_context).paintCompositedResultsToMediaSample();
     766    }
     767#endif
    761768    auto* imageBuffer = buffer();
    762769    if (!imageBuffer)

  • trunk/Source/WebCore/html/canvas/WebGLRenderingContextBase.cpp

    r282860 r283238  
    135135#endif
    136136
     137#if ENABLE(MEDIA_STREAM)
     138#include "MediaSample.h"
     139#endif
     140
    137141namespace WebCore {
    138142
    … …  
    13751379    return m_context->paintRenderingResultsToPixelBuffer();
    13761380}
     1381
     1382#if ENABLE(MEDIA_STREAM)
     1383RefPtr<MediaSample> WebGLRenderingContextBase::paintCompositedResultsToMediaSample()
     1384{
     1385    if (isContextLostOrPending())
     1386        return nullptr;
     1387    return m_context->paintCompositedResultsToMediaSample();
     1388}
     1389#endif
    13771390
    13781391WebGLTexture::TextureExtensionFlag WebGLRenderingContextBase::textureExtensionFlags() const
    … …  
    39513964}
    39523965
    3953 void WebGLRenderingContextBase::enablePreserveDrawingBuffer()
    3954 {
    3955     ASSERT(!m_attributes.preserveDrawingBuffer);
    3956     m_attributes.preserveDrawingBuffer = true;
    3957     // Must send this notification down to the GraphicsContextGL as well.
    3958     m_context->enablePreserveDrawingBuffer();
    3959 }
    3960 
    39613966void WebGLRenderingContextBase::hint(GCGLenum target, GCGLenum mode)
    39623967{

  • trunk/Source/WebCore/html/canvas/WebGLRenderingContextBase.h

    r282192 r283238  
    129129#endif
    130130
     131#if ENABLE(MEDIA_STREAM)
     132class MediaSample;
     133#endif
     134
    131135class WebGLRenderingContextBase : public GraphicsContextGL::Client, public GPUBasedCanvasRenderingContext, private ActivityStateChangeObserver {
    132136    WTF_MAKE_ISO_ALLOCATED(WebGLRenderingContextBase);
    … …  
    234238
    235239    bool isPreservingDrawingBuffer() const { return m_attributes.preserveDrawingBuffer; }
    236     // Concession to canvas capture API, which must dynamically enable
    237     // preserveDrawingBuffer. This can only be called once, when
    238     // isPreservingDrawingBuffer() returns false.
    239     void enablePreserveDrawingBuffer();
    240240
    241241    bool preventBufferClearForInspector() const { return m_preventBufferClearForInspector; }
    … …  
    393393    void paintRenderingResultsToCanvas() final;
    394394    std::optional<PixelBuffer> paintRenderingResultsToPixelBuffer();
     395#if ENABLE(MEDIA_STREAM)
     396    RefPtr<MediaSample> paintCompositedResultsToMediaSample();
     397#endif
    395398
    396399    void removeSharedObject(WebGLSharedObject&);

  • trunk/Source/WebCore/platform/graphics/GraphicsContextGL.cpp

    r281621 r283238  
    363363}
    364364
    365 void GraphicsContextGL::enablePreserveDrawingBuffer()
    366 {
    367     // Canvas capture should not call this unless necessary.
    368     ASSERT(!m_attrs.preserveDrawingBuffer);
    369     m_attrs.preserveDrawingBuffer = true;
    370 }
    371365
    372366bool GraphicsContextGL::computeFormatAndTypeParameters(GCGLenum format, GCGLenum type, unsigned* componentsPerPixel, unsigned* bytesPerComponent)

  • trunk/Source/WebCore/platform/graphics/GraphicsContextGL.h

    r282053 r283238  
    6060class GraphicsContextGLCV;
    6161#endif
    62 
     62#if ENABLE(MEDIA_STREAM)
     63class MediaSample;
     64#endif
    6365
    6466// Base class for graphics context for implementing WebGL rendering model.
    … …  
    12451247    void setContextAttributes(const GraphicsContextGLAttributes& attrs) { m_attrs = attrs; }
    12461248
    1247     // FIXME: The calling site should be fixed, this breaks the WebGL spec
    1248     // wrt preserveDrawingBuffer == false behavior.
    1249     // Concession to Canvas captureStream, which needs to dynamically set
    1250     // preserveDrawingBuffer to true in order to avoid implicit clears.
    1251     // Implementations generally do not support toggling this bit arbitrarily.
    1252     virtual void enablePreserveDrawingBuffer();
    1253 
    12541249    // Support for extensions. Returns a non-null object, though not
    12551250    // all methods it contains may necessarily be supported on the
    … …  
    12951290    virtual std::optional<PixelBuffer> paintRenderingResultsToPixelBuffer() = 0;
    12961291    virtual void paintCompositedResultsToCanvas(ImageBuffer&) = 0;
     1292#if ENABLE(MEDIA_STREAM)
     1293    virtual RefPtr<MediaSample> paintCompositedResultsToMediaSample() = 0;
     1294#endif
    12971295
    12981296    // FIXME: this should be removed. The layer should be marked composited by

  • trunk/Source/WebCore/platform/graphics/RemoteGraphicsContextGLProxyBase.cpp

    r272217 r283238  
    9898}
    9999
    100 void RemoteGraphicsContextGLProxyBase::enablePreserveDrawingBuffer()
    101 {
    102     // Redeclared for export reasons.
    103     // FIXME: The whole function should be removed.
    104     GraphicsContextGL::enablePreserveDrawingBuffer();
    105 }
    106 
    107100bool RemoteGraphicsContextGLProxyBase::supports(const String& name)
    108101{
    … …  
    157150}
    158151#endif
     152
     153#if ENABLE(MEDIA_STREAM) && !PLATFORM(COCOA)
     154RefPtr<MediaSample> RemoteGraphicsContextGLProxyBase::paintCompositedResultsToMediaSample()
     155{
     156    return nullptr;
    159157}
    160158#endif
     159}
     160#endif

  • trunk/Source/WebCore/platform/graphics/RemoteGraphicsContextGLProxyBase.h

    r272217 r283238  
    6868    GCGLbitfield getBuffersToAutoClear() const final;
    6969    void markLayerComposited() final;
    70     void enablePreserveDrawingBuffer() final;
    7170
    7271#if ENABLE(VIDEO) && USE(AVFOUNDATION)
    … …  
    8382    void getnUniformfvEXT(GCGLuint, GCGLint, GCGLsizei, GCGLfloat*) final;
    8483    void getnUniformivEXT(GCGLuint, GCGLint, GCGLsizei, GCGLint*) final;
     84#endif
     85#if ENABLE(MEDIA_STREAM)
     86    RefPtr<MediaSample> paintCompositedResultsToMediaSample() final;
    8587#endif
    8688

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.h

    r280416 r283238  
    3131#include <wtf/Forward.h>
    3232
     33using CVPixelBufferRef = struct __CVBuffer*;
     34
    3335namespace WebCore {
    3436
    … …  
    4143    static Ref<MediaSampleAVFObjC> create(CMSampleBufferRef sample, VideoRotation rotation = VideoRotation::None, bool mirrored = false) { return adoptRef(*new MediaSampleAVFObjC(sample, rotation, mirrored)); }
    4244    static RefPtr<MediaSampleAVFObjC> createImageSample(PixelBuffer&&);
     45    static RefPtr<MediaSampleAVFObjC> createImageSample(RetainPtr<CVPixelBufferRef>&&, VideoRotation, bool mirrored);
    4346
    4447    WEBCORE_EXPORT static void setAsDisplayImmediately(MediaSample&);

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm

    r282755 r283238  
    9696    }
    9797    ASSERT_UNUSED(status, !status);
    98 
     98    return createImageSample(WTFMove(cvPixelBuffer), VideoRotation::None, false);
     99}
     100
     101RefPtr<MediaSampleAVFObjC> MediaSampleAVFObjC::createImageSample(RetainPtr<CVPixelBufferRef>&& pixelBuffer, VideoRotation rotation, bool mirrored)
     102{
    99103    CMVideoFormatDescriptionRef formatDescriptionRaw = nullptr;
    100     status = PAL::CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, cvPixelBuffer.get(), &formatDescriptionRaw);
    101     ASSERT(!status);
     104    auto status = PAL::CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, pixelBuffer.get(), &formatDescriptionRaw);
     105    if (status || !formatDescriptionRaw) {
     106        ASSERT_NOT_REACHED();
     107        return nullptr;
     108    }
    102109    auto formatDescription = adoptCF(formatDescriptionRaw);
    103110
    104111    CMSampleTimingInfo sampleTimingInformation = { PAL::kCMTimeInvalid, PAL::kCMTimeInvalid, PAL::kCMTimeInvalid };
    105112    CMSampleBufferRef sampleBufferRaw = nullptr;
    106     status = PAL::CMSampleBufferCreateReadyWithImageBuffer(kCFAllocatorDefault, cvPixelBuffer.get(), formatDescription.get(), &sampleTimingInformation, &sampleBufferRaw);
    107     ASSERT(!status);
     113    status = PAL::CMSampleBufferCreateReadyWithImageBuffer(kCFAllocatorDefault, pixelBuffer.get(), formatDescription.get(), &sampleTimingInformation, &sampleBufferRaw);
     114    if (status || !sampleBufferRaw) {
     115        ASSERT_NOT_REACHED();
     116        return nullptr;
     117    }
    108118    auto sampleBuffer = adoptCF(sampleBufferRaw);
    109 
    110119    CFArrayRef attachmentsArray = PAL::CMSampleBufferGetSampleAttachmentsArray(sampleBuffer.get(), true);
    111120    for (CFIndex i = 0, count = CFArrayGetCount(attachmentsArray); i < count; ++i) {
    … …  
    113122        CFDictionarySetValue(attachments, PAL::kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanTrue);
    114123    }
    115     return create(sampleBuffer.get());
     124    return create(sampleBuffer.get(), rotation, mirrored);
    116125}
    117126

  • trunk/Source/WebCore/platform/graphics/cocoa/GraphicsContextGLIOSurfaceSwapChain.cpp

    r273080 r283238  
    3838}
    3939
     40void GraphicsContextGLIOSurfaceSwapChain::markDisplayBufferInUse()
     41{
     42    m_displayBufferInUse = true;
     43}
     44
    4045GraphicsContextGLIOSurfaceSwapChain::Buffer GraphicsContextGLIOSurfaceSwapChain::recycleBuffer()
    4146{
    … …  
    5762{
    5863    m_spareBuffer = std::exchange(m_displayBuffer, WTFMove(buffer));
     64    if (m_displayBufferInUse) {
     65        m_displayBufferInUse = false;
     66        m_spareBuffer.surface.reset();
     67    }
    5968}
    6069

  • trunk/Source/WebCore/platform/graphics/cocoa/GraphicsContextGLIOSurfaceSwapChain.h

    r273080 r283238  
    6868    virtual const Buffer& displayBuffer() const;
    6969
     70    // Marks the current display buffer to be in use and not eligble to recycling.
     71    void markDisplayBufferInUse();
     72
    7073    // Detaches the client and returns the current contents buffer metadata handle.
    7174    // The if multiple buffers have been submitted, recycleBuffer must have been called before calling
    … …  
    7679    Buffer m_displayBuffer;
    7780    Buffer m_spareBuffer;
     81    bool m_displayBufferInUse = false;
    7882};
    7983

  • trunk/Source/WebCore/platform/graphics/cocoa/GraphicsContextGLOpenGLCocoa.mm

    r282132 r283238  
    2929#import "GraphicsContextGLOpenGL.h"
    3030
     31#import "CVUtilities.h"
    3132#import "ExtensionsGLANGLE.h"
    3233#import "GraphicsContextGLANGLEUtilities.h"
    … …  
    4950#if ENABLE(VIDEO) && USE(AVFOUNDATION)
    5051#include "GraphicsContextGLCV.h"
     52#endif
     53
     54#if ENABLE(MEDIA_STREAM)
     55#include "MediaSampleAVFObjC.h"
    5156#endif
    5257
    … …  
    879884}
    880885
     886#if ENABLE(MEDIA_STREAM)
     887RefPtr<MediaSample> GraphicsContextGLOpenGL::paintCompositedResultsToMediaSample()
     888{
     889    auto &displayBuffer = m_swapChain->displayBuffer();
     890    if (!displayBuffer.surface || !displayBuffer.handle)
     891        return nullptr;
     892    if (displayBuffer.surface->size() != getInternalFramebufferSize())
     893        return nullptr;
     894    m_swapChain->markDisplayBufferInUse();
     895    auto pixelBuffer = createCVPixelBuffer(displayBuffer.surface->surface());
     896    if (!pixelBuffer)
     897        return nullptr;
     898    return MediaSampleAVFObjC::createImageSample(WTFMove(*pixelBuffer), MediaSampleAVFObjC::VideoRotation::UpsideDown, true);
     899}
     900#endif
    881901}
    882902

  • trunk/Source/WebCore/platform/graphics/cocoa/RemoteGraphicsContextGLProxyBaseCocoa.mm

    r270587 r283238  
    3333#import <wtf/BlockObjCExceptions.h>
    3434
     35#if ENABLE(MEDIA_STREAM)
     36#import "CVUtilities.h"
     37#import "MediaSampleAVFObjC.h"
     38#endif
     39
    3540namespace WebCore {
    3641
    … …  
    6469}
    6570
     71#if ENABLE(MEDIA_STREAM)
     72RefPtr<MediaSample> RemoteGraphicsContextGLProxyBase::paintCompositedResultsToMediaSample()
     73{
     74    auto& sc = platformSwapChain();
     75    auto& displayBuffer = sc.displayBuffer();
     76    if (!displayBuffer.surface)
     77        return nullptr;
     78    sc.markDisplayBufferInUse();
     79    auto pixelBuffer = createCVPixelBuffer(displayBuffer.surface->surface());
     80    if (!pixelBuffer)
     81        return nullptr;
     82    return MediaSampleAVFObjC::createImageSample(WTFMove(*pixelBuffer), MediaSampleAVFObjC::VideoRotation::UpsideDown, true);
    6683}
    6784#endif
     85
     86}
     87#endif

  • trunk/Source/WebCore/platform/graphics/opengl/GraphicsContextGLOpenGL.cpp

    r281621 r283238  
    3535#include "MediaPlayerPrivate.h"
    3636#include "PixelBuffer.h"
     37#include <memory>
    3738#include <wtf/UniqueArray.h>
    3839
    … …  
    4142#endif
    4243
    43 #include <memory>
     44#if !PLATFORM(COCOA) && ENABLE(MEDIA_STREAM)
     45#include "MediaSample.h"
     46#endif
    4447
    4548namespace WebCore {
    … …  
    7174{
    7275    return m_buffersToAutoClear;
    73 }
    74 
    75 void GraphicsContextGLOpenGL::enablePreserveDrawingBuffer()
    76 {
    77     GraphicsContextGL::enablePreserveDrawingBuffer();
    78     // After dynamically transitioning to preserveDrawingBuffer:true
    79     // for canvas capture, clear out any buffer auto-clearing state.
    80     m_buffersToAutoClear = 0;
    8176}
    8277
    … …  
    171166}
    172167
     168#if !PLATFORM(COCOA) && ENABLE(MEDIA_STREAM)
     169RefPtr<MediaSample> GraphicsContextGLOpenGL::paintCompositedResultsToMediaSample()
     170{
     171    return nullptr;
     172}
     173#endif
     174
    173175std::optional<PixelBuffer> GraphicsContextGLOpenGL::readRenderingResultsForPainting()
    174176{

  • trunk/Source/WebCore/platform/graphics/opengl/GraphicsContextGLOpenGL.h

    r281621 r283238  
    461461    void setBuffersToAutoClear(GCGLbitfield) final;
    462462    GCGLbitfield getBuffersToAutoClear() const final;
    463     void enablePreserveDrawingBuffer() final;
    464463
    465464    void dispatchContextChangedNotification();
    … …  
    468467    std::optional<PixelBuffer> paintRenderingResultsToPixelBuffer() final;
    469468    void paintCompositedResultsToCanvas(ImageBuffer&) final;
     469#if ENABLE(MEDIA_STREAM)
     470    RefPtr<MediaSample> paintCompositedResultsToMediaSample() final;
     471#endif
    470472
    471473    std::optional<PixelBuffer> readRenderingResultsForPainting();
Note: See TracChangeset for help on using the changeset viewer.