Changeset 225638 in webkit


Ignore:
Timestamp:
Dec 7, 2017 11:56:25 AM (6 years ago)
Author:
jer.noble@apple.com
Message:

Creating a second AVPlayerItemVideoOutput causes flakey failures
​https://bugs.webkit.org/show_bug.cgi?id=177119
<rdar://problem/34507977>

Reviewed by Dean Jackson.

Source/WebCore:

Rather than having two concurrent AVPlayerItemVideoOutputs when painting to OpenGL,
use only one, but with the OpenGL compatibility flags enabled, for both 3D and 2D
canvas painting.

Of course, once this is fixed, the test breaks again because VideoTextureCopierCV is
stomping on top of some WebGL state set by the test. Rather than expand the
VideoTextureCopierCV's GraphicsStateSaver, use a totally separate (but shared) GL context to
implement the texture copy, so that its own state will not affect the state of the WebGL
context.

This necessitates adding a new createShared() factory for GraphicsContext3D. Since this is only
used on Cocoa platforms (for now), make it platform specific, and modify the other implementations
only so far as to throw assertions if somehow a shared context pointer is passed into their
constructors.

Drive-by fixes:

  • These changes exposed some test failures, which were off-by-1-or-2 errors in the color

output of videos painted into WebGL contexts. The Y' parameter needs to be scaled by 16/255
for video-range conversions and the CbCr vales by 128/255 (rather than just 0.5). Make the
color transforms a full 4x4 matrix and do the scaling in the matrix rather than tracked by a
separate uniform.

  • There was a slight error in WebCoreDecompressionSession that could cause seeks to never

complete because incoming frames would be skipped if their startTime was in the past
(rather than their startTime + duration being in the past).

  • The matrix name comparison should use CFEqual rather than pointer equality to test.
  • iOS currently has a bug where the red and green color channels will be reversed when

uploading BGRA pixel formatted data to a texture; work around this bug for now.

  • platform/graphics/GraphicsContext3D.h:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:

(WebCore::MediaPlayerPrivateAVFoundationObjC::destroyContextVideoRenderer):
(WebCore::MediaPlayerPrivateAVFoundationObjC::hasAvailableVideoFrame const):
(WebCore::MediaPlayerPrivateAVFoundationObjC::updateLastPixelBuffer):
(WebCore::MediaPlayerPrivateAVFoundationObjC::videoOutputHasAvailableFrame):
(WebCore::MediaPlayerPrivateAVFoundationObjC::updateLastImage):
(WebCore::MediaPlayerPrivateAVFoundationObjC::paintWithVideoOutput):
(WebCore::MediaPlayerPrivateAVFoundationObjC::copyVideoTextureToPlatformTexture):
(WebCore::MediaPlayerPrivateAVFoundationObjC::nativeImageForCurrentTime):
(WebCore::MediaPlayerPrivateAVFoundationObjC::waitForVideoOutputMediaDataWillChange):

  • platform/graphics/cairo/GraphicsContext3DCairo.cpp:

(WebCore::GraphicsContext3D::GraphicsContext3D):

  • platform/graphics/cocoa/GraphicsContext3DCocoa.mm:

(WebCore::GraphicsContext3D::createShared):
(WebCore::GraphicsContext3D::GraphicsContext3D):

  • platform/graphics/cv/VideoTextureCopierCV.cpp:

(WebCore::VideoTextureCopierCV::VideoTextureCopierCV):
(WebCore::VideoTextureCopierCV::~VideoTextureCopierCV):
(WebCore::transferFunctionFromString):
(WebCore::YCbCrToRGBMatrixForRangeAndTransferFunction):
(WebCore::VideoTextureCopierCV::initializeUVContextObjects):
(WebCore::VideoTextureCopierCV::copyImageToPlatformTexture):
(WebCore::VideoTextureCopierCV::copyVideoTextureToPlatformTexture):

  • platform/graphics/cv/VideoTextureCopierCV.h:

(WebCore::VideoTextureCopierCV::context):

  • platform/graphics/opengl/GraphicsContext3DOpenGLES.cpp:

(WebCore::GraphicsContext3D::GraphicsContext3D):

LayoutTests:

  • platform/mac/TestExpectations:
Location:
trunk
Files:
13 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r225637 r225638  
     12017-12-07  Jer Noble  <jer.noble@apple.com>
     2
     3        Creating a second AVPlayerItemVideoOutput causes flakey failures
     4        https://bugs.webkit.org/show_bug.cgi?id=177119
     5        <rdar://problem/34507977>
     6
     7        Reviewed by Dean Jackson.
     8
     9        * platform/mac/TestExpectations:
     10
    1112017-12-07  Jer Noble  <jer.noble@apple.com>
    212

  • trunk/LayoutTests/platform/mac/TestExpectations

    r225637 r225638  
    17261726[ HighSierra+ ] http/tests/websocket/tests/hybi/deflate-frame-parameter.html [ Failure ]
    17271727
    1728 # <rdar://problem/34507977>
    1729 webkit.org/b/177119 webgl/1.0.2/conformance/textures/tex-image-and-sub-image-2d-with-video.html [ Pass Failure ]
    1730 
    17311728# Release logging not supported
    17321729[ ElCapitan ] inspector/console/webcore-logging.html [ Skip ]

  • trunk/Source/WebCore/ChangeLog

    r225637 r225638  
     12017-12-07  Jer Noble  <jer.noble@apple.com>
     2
     3        Creating a second AVPlayerItemVideoOutput causes flakey failures
     4        https://bugs.webkit.org/show_bug.cgi?id=177119
     5        <rdar://problem/34507977>
     6
     7        Reviewed by Dean Jackson.
     8
     9        Rather than having two concurrent AVPlayerItemVideoOutputs when painting to OpenGL,
     10        use only one, but with the OpenGL compatibility flags enabled, for both 3D and 2D
     11        canvas painting.
     12
     13        Of course, once this is fixed, the test breaks again because VideoTextureCopierCV is
     14        stomping on top of some WebGL state set by the test. Rather than expand the
     15        VideoTextureCopierCV's GraphicsStateSaver, use a totally separate (but shared) GL context to
     16        implement the texture copy, so that its own state will not affect the state of the WebGL
     17        context.
     18
     19        This necessitates adding a new createShared() factory for GraphicsContext3D. Since this is only
     20        used on Cocoa platforms (for now), make it platform specific, and modify the other implementations
     21        only so far as to throw assertions if somehow a shared context pointer is passed into their
     22        constructors.
     23
     24        Drive-by fixes:
     25
     26        - These changes exposed some test failures, which were off-by-1-or-2 errors in the color
     27        output of videos painted into WebGL contexts. The Y' parameter needs to be scaled by 16/255
     28        for video-range conversions and the CbCr vales by 128/255 (rather than just 0.5). Make the
     29        color transforms a full 4x4 matrix and do the scaling in the matrix rather than tracked by a
     30        separate uniform.
     31
     32        - There was a slight error in WebCoreDecompressionSession that could cause seeks to never
     33        complete because incoming frames would be skipped if their startTime was in the past
     34        (rather than their startTime + duration being in the past).
     35
     36        - The matrix name comparison should use CFEqual rather than pointer equality to test.
     37
     38        - iOS currently has a bug where the red and green color channels will be reversed when
     39        uploading BGRA pixel formatted data to a texture; work around this bug for now.
     40
     41        * platform/graphics/GraphicsContext3D.h:
     42        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
     43        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     44        (WebCore::MediaPlayerPrivateAVFoundationObjC::destroyContextVideoRenderer):
     45        (WebCore::MediaPlayerPrivateAVFoundationObjC::hasAvailableVideoFrame const):
     46        (WebCore::MediaPlayerPrivateAVFoundationObjC::updateLastPixelBuffer):
     47        (WebCore::MediaPlayerPrivateAVFoundationObjC::videoOutputHasAvailableFrame):
     48        (WebCore::MediaPlayerPrivateAVFoundationObjC::updateLastImage):
     49        (WebCore::MediaPlayerPrivateAVFoundationObjC::paintWithVideoOutput):
     50        (WebCore::MediaPlayerPrivateAVFoundationObjC::copyVideoTextureToPlatformTexture):
     51        (WebCore::MediaPlayerPrivateAVFoundationObjC::nativeImageForCurrentTime):
     52        (WebCore::MediaPlayerPrivateAVFoundationObjC::waitForVideoOutputMediaDataWillChange):
     53        * platform/graphics/cairo/GraphicsContext3DCairo.cpp:
     54        (WebCore::GraphicsContext3D::GraphicsContext3D):
     55        * platform/graphics/cocoa/GraphicsContext3DCocoa.mm:
     56        (WebCore::GraphicsContext3D::createShared):
     57        (WebCore::GraphicsContext3D::GraphicsContext3D):
     58        * platform/graphics/cv/VideoTextureCopierCV.cpp:
     59        (WebCore::VideoTextureCopierCV::VideoTextureCopierCV):
     60        (WebCore::VideoTextureCopierCV::~VideoTextureCopierCV):
     61        (WebCore::transferFunctionFromString):
     62        (WebCore::YCbCrToRGBMatrixForRangeAndTransferFunction):
     63        (WebCore::VideoTextureCopierCV::initializeUVContextObjects):
     64        (WebCore::VideoTextureCopierCV::copyImageToPlatformTexture):
     65        (WebCore::VideoTextureCopierCV::copyVideoTextureToPlatformTexture):
     66        * platform/graphics/cv/VideoTextureCopierCV.h:
     67        (WebCore::VideoTextureCopierCV::context):
     68        * platform/graphics/opengl/GraphicsContext3DOpenGLES.cpp:
     69        (WebCore::GraphicsContext3D::GraphicsContext3D):
     70
    1712017-12-07  Jer Noble  <jer.noble@apple.com>
    272

  • trunk/Source/WebCore/platform/graphics/GraphicsContext3D.h

    r225300 r225638  
    746746
    747747#if PLATFORM(COCOA)
     748    static Ref<GraphicsContext3D> createShared(GraphicsContext3D& sharedContext);
     749#endif
     750
     751#if PLATFORM(COCOA)
    748752    PlatformGraphicsContext3D platformGraphicsContext3D() const { return m_contextObj; }
    749753    Platform3DObject platformTexture() const { return m_texture; }
    … …  
    12881292
    12891293private:
    1290     GraphicsContext3D(GraphicsContext3DAttributes, HostWindow*, RenderStyle = RenderOffscreen);
     1294    GraphicsContext3D(GraphicsContext3DAttributes, HostWindow*, RenderStyle = RenderOffscreen, GraphicsContext3D* sharedContext = nullptr);
    12911295
    12921296    // Helper for packImageData/extractImageData/extractTextureData which implement packing of pixel

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h

    r224297 r225638  
    5959typedef struct CGImage *CGImageRef;
    6060typedef struct __CVBuffer *CVPixelBufferRef;
    61 #if PLATFORM(IOS)
    62 typedef struct  __CVOpenGLESTextureCache *CVOpenGLESTextureCacheRef;
    63 #else
    64 typedef struct __CVOpenGLTextureCache* CVOpenGLTextureCacheRef;
    65 #endif
    6661
    6762namespace WebCore {
    … …  
    266261    void createVideoOutput();
    267262    void destroyVideoOutput();
    268     RetainPtr<CVPixelBufferRef> createPixelBuffer();
     263    bool updateLastPixelBuffer();
    269264    void updateLastImage();
    270265    bool videoOutputHasAvailableFrame();
    … …  
    272267    NativeImagePtr nativeImageForCurrentTime() override;
    273268    void waitForVideoOutputMediaDataWillChange();
    274 
    275     void createOpenGLVideoOutput();
    276     void destroyOpenGLVideoOutput();
    277     void updateLastOpenGLImage();
    278269
    279270    bool copyVideoTextureToPlatformTexture(GraphicsContext3D*, Platform3DObject, GC3Denum target, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY) override;
    … …  
    364355    RetainPtr<AVPlayerItemVideoOutput> m_videoOutput;
    365356    RetainPtr<WebCoreAVFPullDelegate> m_videoOutputDelegate;
     357    RetainPtr<CVPixelBufferRef> m_lastPixelBuffer;
    366358    RetainPtr<CGImageRef> m_lastImage;
    367359    dispatch_semaphore_t m_videoOutputSemaphore;
    368 
    369     RetainPtr<AVPlayerItemVideoOutput> m_openGLVideoOutput;
    370     std::unique_ptr<TextureCacheCV> m_textureCache;
    371360    std::unique_ptr<VideoTextureCopierCV> m_videoTextureCopier;
    372     RetainPtr<CVPixelBufferRef> m_lastOpenGLImage;
    373361#endif
    374362

  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm

    r225618 r225638  
    649649#if HAVE(AVFOUNDATION_VIDEO_OUTPUT)
    650650    destroyVideoOutput();
    651     destroyOpenGLVideoOutput();
    652651#endif
    653652    destroyImageGenerator();
    … …  
    753752    if (currentRenderingMode() == MediaRenderingToLayer)
    754753        return m_cachedIsReadyForDisplay;
     754
     755    if (m_videoOutput && (m_lastPixelBuffer || [m_videoOutput hasNewPixelBufferForItemTime:[m_avPlayerItem currentTime]]))
     756        return true;
    755757
    756758    return m_videoFrameHasDrawn;
    … …  
    22932295}
    22942296
    2295 RetainPtr<CVPixelBufferRef> MediaPlayerPrivateAVFoundationObjC::createPixelBuffer()
     2297bool MediaPlayerPrivateAVFoundationObjC::updateLastPixelBuffer()
    22962298{
    22972299    if (!m_videoOutput)
    … …  
    23022304
    23032305    if (![m_videoOutput.get() hasNewPixelBufferForItemTime:currentTime])
    2304         return 0;
    2305 
    2306     return adoptCF([m_videoOutput.get() copyPixelBufferForItemTime:currentTime itemTimeForDisplay:nil]);
     2306        return false;
     2307
     2308    m_lastPixelBuffer = adoptCF([m_videoOutput.get() copyPixelBufferForItemTime:currentTime itemTimeForDisplay:nil]);
     2309    m_lastImage = nullptr;
     2310    return true;
    23072311}
    23082312
    … …  
    23232327void MediaPlayerPrivateAVFoundationObjC::updateLastImage()
    23242328{
    2325     RetainPtr<CVPixelBufferRef> pixelBuffer = createPixelBuffer();
    2326 
    23272329    // Calls to copyPixelBufferForItemTime:itemTimeForDisplay: may return nil if the pixel buffer
    23282330    // for the requested time has already been retrieved. In this case, the last valid image (if any)
    23292331    // should be displayed.
    2330     if (!pixelBuffer)
     2332    if (!updateLastPixelBuffer() && (m_lastImage || !m_lastPixelBuffer))
    23312333        return;
    23322334
    … …  
    23442346#endif
    23452347
    2346     m_lastImage = m_pixelBufferConformer->createImageFromPixelBuffer(pixelBuffer.get());
     2348    m_lastImage = m_pixelBufferConformer->createImageFromPixelBuffer(m_lastPixelBuffer.get());
    23472349
    23482350#if !RELEASE_LOG_DISABLED
    … …  
    23822384}
    23832385
    2384 void MediaPlayerPrivateAVFoundationObjC::createOpenGLVideoOutput()
    2385 {
    2386     INFO_LOG(LOGIDENTIFIER);
    2387 
    2388     if (!m_avPlayerItem || m_openGLVideoOutput)
    2389         return;
    2390 
    2391 #if PLATFORM(IOS)
    2392     NSDictionary* attributes = @{(NSString *)kCVPixelBufferIOSurfaceOpenGLESFBOCompatibilityKey: @YES};
    2393 #else
    2394     NSDictionary* attributes = @{(NSString *)kCVPixelBufferIOSurfaceOpenGLFBOCompatibilityKey: @YES};
    2395 #endif
    2396     m_openGLVideoOutput = adoptNS([[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:attributes]);
    2397     ASSERT(m_openGLVideoOutput);
    2398 
    2399     [m_avPlayerItem.get() addOutput:m_openGLVideoOutput.get()];
    2400 }
    2401 
    2402 void MediaPlayerPrivateAVFoundationObjC::destroyOpenGLVideoOutput()
    2403 {
    2404     if (!m_openGLVideoOutput)
    2405         return;
    2406 
    2407     INFO_LOG(LOGIDENTIFIER);
    2408 
    2409     if (m_avPlayerItem)
    2410         [m_avPlayerItem.get() removeOutput:m_openGLVideoOutput.get()];
    2411 
    2412     m_openGLVideoOutput = 0;
    2413 }
    2414 
    2415 void MediaPlayerPrivateAVFoundationObjC::updateLastOpenGLImage()
    2416 {
    2417     if (!m_openGLVideoOutput)
    2418         return;
    2419 
    2420     CMTime currentTime = [m_openGLVideoOutput itemTimeForHostTime:CACurrentMediaTime()];
    2421     if (![m_openGLVideoOutput hasNewPixelBufferForItemTime:currentTime])
    2422         return;
    2423 
    2424     m_lastOpenGLImage = adoptCF([m_openGLVideoOutput copyPixelBufferForItemTime:currentTime itemTimeForDisplay:nil]);
    2425 }
    2426 
    24272386bool MediaPlayerPrivateAVFoundationObjC::copyVideoTextureToPlatformTexture(GraphicsContext3D* context, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY)
    24282387{
    24292388    ASSERT(context);
    24302389
    2431     if (!m_openGLVideoOutput)
    2432         createOpenGLVideoOutput();
    2433 
    2434     updateLastOpenGLImage();
    2435 
    2436     if (!m_lastOpenGLImage)
     2390    updateLastPixelBuffer();
     2391    if (!m_lastPixelBuffer)
    24372392        return false;
    24382393
    2439     size_t width = CVPixelBufferGetWidth(m_lastOpenGLImage.get());
    2440     size_t height = CVPixelBufferGetHeight(m_lastOpenGLImage.get());
    2441 
    2442     if (!m_textureCache) {
    2443         m_textureCache = TextureCacheCV::create(*context);
    2444         if (!m_textureCache)
    2445             return false;
    2446     }
    2447 
    2448     RetainPtr<CVOpenGLTextureRef> videoTexture = m_textureCache->textureFromImage(m_lastOpenGLImage.get(), outputTarget, level, internalFormat, format, type);
     2394    size_t width = CVPixelBufferGetWidth(m_lastPixelBuffer.get());
     2395    size_t height = CVPixelBufferGetHeight(m_lastPixelBuffer.get());
    24492396
    24502397    if (!m_videoTextureCopier)
    24512398        m_videoTextureCopier = std::make_unique<VideoTextureCopierCV>(*context);
    24522399
    2453     return m_videoTextureCopier->copyVideoTextureToPlatformTexture(videoTexture.get(), width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY);
     2400    return m_videoTextureCopier->copyImageToPlatformTexture(m_lastPixelBuffer.get(), width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY);
    24542401}
    24552402

  • trunk/Source/WebCore/platform/graphics/cairo/GraphicsContext3DCairo.cpp

    r225428 r225638  
    114114}
    115115
    116 GraphicsContext3D::GraphicsContext3D(GraphicsContext3DAttributes attributes, HostWindow*, GraphicsContext3D::RenderStyle renderStyle)
     116GraphicsContext3D::GraphicsContext3D(GraphicsContext3DAttributes attributes, HostWindow*, GraphicsContext3D::RenderStyle renderStyle, GraphicsContext3D* sharedContext)
    117117    : m_attrs(attributes)
    118118{
     119    ASSERT_UNUSED(sharedContext, !sharedContext);
    119120#if USE(TEXTURE_MAPPER)
    120121    m_texmapLayer = std::make_unique<TextureMapperGC3DPlatformLayer>(*this, renderStyle);

  • trunk/Source/WebCore/platform/graphics/cocoa/GraphicsContext3DCocoa.mm

    r223707 r225638  
    387387}
    388388
    389 GraphicsContext3D::GraphicsContext3D(GraphicsContext3DAttributes attrs, HostWindow*, GraphicsContext3D::RenderStyle)
     389Ref<GraphicsContext3D> GraphicsContext3D::createShared(GraphicsContext3D& sharedContext)
     390{
     391    auto context = adoptRef(*new GraphicsContext3D(sharedContext.getContextAttributes(), nullptr, sharedContext.m_renderStyle, &sharedContext));
     392
     393    manager().addContext(context.ptr());
     394
     395    return context;
     396}
     397
     398GraphicsContext3D::GraphicsContext3D(GraphicsContext3DAttributes attrs, HostWindow*, GraphicsContext3D::RenderStyle, GraphicsContext3D* sharedContext)
    390399    : m_attrs(attrs)
    391400#if PLATFORM(IOS)
    … …  
    396405#if PLATFORM(IOS)
    397406    EAGLRenderingAPI api = m_attrs.useGLES3 ? kEAGLRenderingAPIOpenGLES3 : kEAGLRenderingAPIOpenGLES2;
    398     m_contextObj = [[EAGLContext alloc] initWithAPI:api];
     407    if (!sharedContext)
     408        m_contextObj = [[EAGLContext alloc] initWithAPI:api];
     409    else
     410        m_contextObj = [[EAGLContext alloc] initWithAPI:api sharegroup:sharedContext->m_contextObj.sharegroup];
    399411    makeContextCurrent();
    400412#else
    … …  
    443455        return;
    444456
    445     CGLError err = CGLCreateContext(pixelFormatObj, 0, &m_contextObj);
     457    CGLError err = CGLCreateContext(pixelFormatObj, sharedContext ? sharedContext->m_contextObj : nullptr, &m_contextObj);
    446458    GLint abortOnBlacklist = 0;
    447459#if PLATFORM(MAC)

  • trunk/Source/WebCore/platform/graphics/cocoa/WebCoreDecompressionSession.mm

    r223476 r225638  
    186186
    187187    auto currentTime = CMTimebaseGetTime(m_timebase.get());
    188     auto presentationTimeStamp = CMSampleBufferGetPresentationTimeStamp(sample);
    189     if (CMTimeCompare(presentationTimeStamp, currentTime) >= 0)
     188    auto presentationStartTime = CMSampleBufferGetPresentationTimeStamp(sample);
     189    auto duration = CMSampleBufferGetDuration(sample);
     190    auto presentationEndTime = CMTimeAdd(presentationStartTime, duration);
     191    if (CMTimeCompare(presentationEndTime, currentTime) >= 0)
    190192        return true;
    191193

  • trunk/Source/WebCore/platform/graphics/cv/VideoTextureCopierCV.cpp

    r224207 r225638  
    9797static TransferFunction transferFunctionFromString(CFStringRef string)
    9898{
    99     if (string == kCVImageBufferYCbCrMatrix_ITU_R_709_2)
     99    if (CFEqual(string, kCVImageBufferYCbCrMatrix_ITU_R_709_2))
    100100        return TransferFunction::kITU_R_709_2;
    101     if (string == kCVImageBufferYCbCrMatrix_ITU_R_601_4)
     101    if (CFEqual(string, kCVImageBufferYCbCrMatrix_ITU_R_601_4))
    102102        return TransferFunction::kITU_R_601_4;
    103     if (string == kCVImageBufferYCbCrMatrix_SMPTE_240M_1995)
     103    if (CFEqual(string, kCVImageBufferYCbCrMatrix_SMPTE_240M_1995))
    104104        return TransferFunction::kSMPTE_240M_1995;
    105     if (canLoad_CoreVideo_kCVImageBufferYCbCrMatrix_DCI_P3() && string == kCVImageBufferYCbCrMatrix_DCI_P3)
     105    if (canLoad_CoreVideo_kCVImageBufferYCbCrMatrix_DCI_P3() && CFEqual(string, kCVImageBufferYCbCrMatrix_DCI_P3))
    106106        return TransferFunction::kDCI_P3;
    107     if (canLoad_CoreVideo_kCVImageBufferYCbCrMatrix_P3_D65() && string == kCVImageBufferYCbCrMatrix_P3_D65)
     107    if (canLoad_CoreVideo_kCVImageBufferYCbCrMatrix_P3_D65() && CFEqual(string, kCVImageBufferYCbCrMatrix_P3_D65))
    108108        return TransferFunction::kP3_D65;
    109     if (canLoad_CoreVideo_kCVImageBufferYCbCrMatrix_ITU_R_2020() && string == kCVImageBufferYCbCrMatrix_ITU_R_2020)
     109    if (canLoad_CoreVideo_kCVImageBufferYCbCrMatrix_ITU_R_2020() && CFEqual(string, kCVImageBufferYCbCrMatrix_ITU_R_2020))
    110110        return TransferFunction::kITU_R_2020;
    111111    return TransferFunction::Unknown;
    … …  
    123123        // https://www.itu.int/rec/R-REC-BT.601
    124124        matrices.get().emplace(MapKey(PixelRange::Video, TransferFunction::kITU_R_601_4), Vector<GLfloat>({
    125             1.164383562f,  0.0f,           1.596026786f,
    126             1.164383562f, -0.3917622901f, -0.8129676472f,
    127             1.164383562f,  2.017232143f,   0.0f,
     125            1.164383562f,  0.0f,           1.596026786f,  -0.874202218f,
     126            1.164383562f, -0.3917622901f, -0.8129676472f,  0.531667823f,
     127            1.164383562f,  2.017232143f,   0.0f,          -1.085630790f,
     128            0.0f,          0.0f,           0.0f,           1.0f,
    128129        }));
    129130        matrices.get().emplace(MapKey({PixelRange::Full, TransferFunction::kITU_R_601_4}), Vector<GLfloat>({
    130             1.000000000f,  0.0f,           1.4075196850f,
    131             1.000000000f, -0.3454911535f, -0.7169478464f,
    132             1.000000000f,  1.7789763780f,  0.0f,
     131            1.000000000f,  0.0f,           1.4075196850f, -1.012571428f,
     132            1.000000000f, -0.3454911535f, -0.7169478464f,  0.533302714f,
     133            1.000000000f,  1.7789763780f,  0.0f,          -0.892976378f,
     134            0.0f,          0.0f,           0.0f,           1.0f,
    133135        }));
    134136        // Matrices are derived from the components in the ITU R.709 rev 2 specification
    135137        // https://www.itu.int/rec/R-REC-BT.709-2-199510-S
    136138        matrices.get().emplace(MapKey({PixelRange::Video, TransferFunction::kITU_R_709_2}), Vector<GLfloat>({
    137             1.164383562f,  0.0f,           1.792741071f,
    138             1.164383562f, -0.2132486143f, -0.5329093286f,
    139             1.164383562f,  2.112401786f,   0.0f,
     139            1.164383562f,  0.0f,           1.792741071f,  -0.972945075f,
     140            1.164383562f, -0.2132486143f, -0.5329093286f,  0.301482665f,
     141            1.164383562f,  2.112401786f,   0.0f,          -1.133402218f,
     142            0.0f,          0.0f,           0.0f,           1.0f,
    140143        }));
    141144        matrices.get().emplace(MapKey({PixelRange::Full, TransferFunction::kITU_R_709_2}), Vector<GLfloat>({
    142             1.000000000f,  0.0f,           1.5810000000f,
    143             1.000000000f, -0.1880617701f, -0.4699672819f,
    144             1.000000000f,  1.8629055118f,  0.0f,
     145            1.000000000f,  0.0f,           1.5810000000f, -0.793600000f,
     146            1.000000000f, -0.1880617701f, -0.4699672819f,  0.330304779f,
     147            1.000000000f,  1.8629055118f,  0.0f,          -0.935105512f,
     148            0.0f,          0.0f,           0.0f,           1.0f,
    145149        }));
    146150    });
    … …  
    163167
    164168VideoTextureCopierCV::VideoTextureCopierCV(GraphicsContext3D& context)
    165     : m_context(context)
     169    : m_sharedContext(context)
     170    , m_context(GraphicsContext3D::createShared(context))
    166171    , m_framebuffer(context.createFramebuffer())
    167172{
    … …  
    322327    fragmentShaderSource.appendLiteral("uniform int u_premultiply;\n");
    323328    fragmentShaderSource.appendLiteral("uniform vec2 u_textureDimensions;\n");
     329    fragmentShaderSource.appendLiteral("uniform int u_swapColorChannels;\n");
    324330    fragmentShaderSource.appendLiteral("void main() {\n");
    325331    fragmentShaderSource.appendLiteral("    vec2 texPos = vec2(v_texturePosition.x * u_textureDimensions.x, v_texturePosition.y * u_textureDimensions.y);\n");
    … …  
    329335    fragmentShaderSource.appendLiteral("    vec4 color = texture2DRect(u_texture, texPos);\n");
    330336#endif
     337    fragmentShaderSource.appendLiteral("    if (u_swapColorChannels == 1) {\n");
     338    fragmentShaderSource.appendLiteral("        color.rgba = color.bgra;\n");
     339    fragmentShaderSource.appendLiteral("    }\n");
    331340    fragmentShaderSource.appendLiteral("    if (u_premultiply == 1) {\n");
    332341    fragmentShaderSource.appendLiteral("        gl_FragColor = vec4(color.r * color.a, color.g * color.a, color.b * color.a, color.a);\n");
    … …  
    366375    m_textureDimensionsUniformLocation = m_context->getUniformLocation(m_program, ASCIILiteral("u_textureDimensions"));
    367376    m_flipYUniformLocation = m_context->getUniformLocation(m_program, ASCIILiteral("u_flipY"));
     377    m_swapColorChannelsUniformLocation = m_context->getUniformLocation(m_program, ASCIILiteral("u_swapColorChannels"));
    368378    m_premultiplyUniformLocation = m_context->getUniformLocation(m_program, ASCIILiteral("u_premultiply"));
    369379    m_positionAttributeLocation = m_context->getAttribLocationDirect(m_program, ASCIILiteral("a_position"));
    … …  
    434444        "uniform SAMPLERTYPE u_yTexture;\n"
    435445        "uniform SAMPLERTYPE u_uvTexture;\n"
    436         "uniform mat3 u_colorMatrix;\n"
     446        "uniform mat4 u_colorMatrix;\n"
    437447        "varying vec2 v_yTextureCoordinate;\n"
    438448        "varying vec2 v_uvTextureCoordinate;\n"
    439449        "void main() {\n"
    440         "    vec3 yuv;\n"
     450        "    vec4 yuv;\n"
    441451        "    yuv.r = TEXTUREFUNC(u_yTexture, v_yTextureCoordinate).r;\n"
    442         "    yuv.gb = TEXTUREFUNC(u_uvTexture, v_uvTextureCoordinate).rg - vec2(0.5, 0.5);\n"
    443         "    gl_FragColor = vec4(yuv * u_colorMatrix, 1);\n"
     452        "    yuv.gb = TEXTUREFUNC(u_uvTexture, v_uvTextureCoordinate).rg;\n"
     453        "    yuv.a = 1.0;\n"
     454        "    gl_FragColor = yuv * u_colorMatrix;\n"
    444455        "}\n"
    445456    );
    … …  
    490501    m_context->bindBuffer(GraphicsContext3D::ARRAY_BUFFER, m_yuvVertexBuffer);
    491502    m_context->bufferData(GraphicsContext3D::ARRAY_BUFFER, sizeof(vertices), vertices, GraphicsContext3D::STATIC_DRAW);
     503    m_context->enableVertexAttribArray(m_yuvPositionAttributeLocation);
     504    m_context->vertexAttribPointer(m_yuvPositionAttributeLocation, 2, GraphicsContext3D::FLOAT, false, 0, 0);
    492505
    493506    return true;
    … …  
    502515    }
    503516
    504     if (auto texture = m_textureCache->textureFromImage(image, outputTarget, level, internalFormat, format, type))
    505         return copyVideoTextureToPlatformTexture(texture.get(), width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY);
     517    if (auto texture = m_textureCache->textureFromImage(image, outputTarget, level, internalFormat, format, type)) {
     518        bool swapColorChannels = false;
     519#if PLATFORM(IOS)
     520        // FIXME: Remove this workaround once rdar://problem/35834388 is fixed.
     521        swapColorChannels = CVPixelBufferGetPixelFormatType(image) == kCVPixelFormatType_32BGRA;
     522#endif
     523        return copyVideoTextureToPlatformTexture(texture.get(), width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY, swapColorChannels);
     524    }
    506525
    507526#if USE(IOSURFACE)
    … …  
    527546    }
    528547
    529     GC3DStateSaver stateSaver(m_context.get());
    530 
    531548    if (!m_yuvProgram) {
    532549        if (!initializeUVContextObjects()) {
    … …  
    535552        }
    536553    }
    537 
    538     stateSaver.saveVertexAttribState(m_yuvPositionAttributeLocation);
    539554
    540555    m_context->bindFramebuffer(GraphicsContext3D::FRAMEBUFFER, m_framebuffer);
    … …  
    604619    auto transferFunction = transferFunctionFromString((CFStringRef)CVBufferGetAttachment(image, kCVImageBufferYCbCrMatrixKey, nil));
    605620    auto& colorMatrix = YCbCrToRGBMatrixForRangeAndTransferFunction(range, transferFunction);
    606     m_context->uniformMatrix3fv(m_colorMatrixUniformLocation, 1, GL_FALSE, colorMatrix.data());
     621    m_context->uniformMatrix4fv(m_colorMatrixUniformLocation, 1, GL_FALSE, colorMatrix.data());
    607622
    608623    // Do the actual drawing.
    609     m_context->bindBuffer(GraphicsContext3D::ARRAY_BUFFER, m_yuvVertexBuffer);
    610     m_context->enableVertexAttribArray(m_yuvPositionAttributeLocation);
    611     m_context->vertexAttribPointer(m_yuvPositionAttributeLocation, 2, GraphicsContext3D::FLOAT, false, 0, 0);
    612624    m_context->drawArrays(GraphicsContext3D::TRIANGLES, 0, 6);
     625
     626#if PLATFORM(IOS)
     627    // flush() must be called here in order to re-synchronize the output texture's contents across the
     628    // two EAGL contexts.
     629    m_context->flush();
     630#endif
    613631
    614632    // Clean-up.
    615633    m_context->deleteTexture(yTexture);
    616634    m_context->deleteTexture(uvTexture);
    617     m_context->bindTexture(videoTextureTarget, 0);
    618635
    619636    m_lastSurface = surface;
    … …  
    628645}
    629646
    630 bool VideoTextureCopierCV::copyVideoTextureToPlatformTexture(TextureType inputVideoTexture, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY)
     647bool VideoTextureCopierCV::copyVideoTextureToPlatformTexture(TextureType inputVideoTexture, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY, bool swapColorChannels)
    631648{
    632649    if (!inputVideoTexture)
    … …  
    650667        flipY = !flipY;
    651668
    652     return copyVideoTextureToPlatformTexture(videoTextureName, videoTextureTarget, width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY);
    653 }
    654 
    655 bool VideoTextureCopierCV::copyVideoTextureToPlatformTexture(Platform3DObject videoTextureName, GC3Denum videoTextureTarget, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY)
     669    return copyVideoTextureToPlatformTexture(videoTextureName, videoTextureTarget, width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY, swapColorChannels);
     670}
     671
     672bool VideoTextureCopierCV::copyVideoTextureToPlatformTexture(Platform3DObject videoTextureName, GC3Denum videoTextureTarget, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY, bool swapColorChannels)
    656673{
    657674    LOG(WebGL, "VideoTextureCopierCV::copyVideoTextureToPlatformTexture(%p) - internalFormat: %s, format: %s, type: %s flipY: %s, premultiplyAlpha: %s", this, enumToStringMap()[internalFormat], enumToStringMap()[format], enumToStringMap()[type], flipY ? "true" : "false", premultiplyAlpha ? "true" : "false");
    658 
    659     GC3DStateSaver stateSaver(m_context.get());
    660675
    661676    if (!m_program) {
    … …  
    665680        }
    666681    }
    667 
    668     stateSaver.saveVertexAttribState(m_positionAttributeLocation);
    669682
    670683    m_context->bindFramebuffer(GraphicsContext3D::FRAMEBUFFER, m_framebuffer);
    … …  
    705718
    706719    m_context->uniform1i(m_flipYUniformLocation, flipY);
     720    m_context->uniform1i(m_swapColorChannelsUniformLocation, swapColorChannels);
    707721    m_context->uniform1i(m_premultiplyUniformLocation, premultiplyAlpha);
    708722
    … …  
    713727    m_context->drawArrays(GraphicsContext3D::TRIANGLES, 0, 6);
    714728
     729#if PLATFORM(IOS)
     730    // flush() must be called here in order to re-synchronize the output texture's contents across the
     731    // two EAGL contexts.
     732    m_context->flush();
     733#endif
     734
    715735    // Clean-up.
    716736    m_context->bindTexture(videoTextureTarget, 0);
    … …  
    720740}
    721741
    722 VideoTextureCopierCV::GC3DStateSaver::GC3DStateSaver(GraphicsContext3D& context)
    723     : m_context(context)
    724 {
    725     m_activeTextureUnit = m_context.activeTextureUnit();
    726     m_boundTarget = m_context.currentBoundTarget();
    727     m_boundTexture = m_context.currentBoundTexture();
    728     m_context.getIntegerv(GraphicsContext3D::FRAMEBUFFER_BINDING, &m_framebuffer);
    729     m_context.getIntegerv(GraphicsContext3D::CURRENT_PROGRAM, &m_program);
    730     m_context.getIntegerv(GraphicsContext3D::ARRAY_BUFFER_BINDING, &m_arrayBuffer);
    731     m_context.getIntegerv(GraphicsContext3D::VIEWPORT, m_viewport);
    732 
    733 }
    734 
    735 VideoTextureCopierCV::GC3DStateSaver::~GC3DStateSaver()
    736 {
    737     if (m_vertexAttribEnabled)
    738         m_context.enableVertexAttribArray(m_vertexAttribIndex);
    739     else
    740         m_context.disableVertexAttribArray(m_vertexAttribIndex);
    741 
    742     m_context.bindBuffer(GraphicsContext3D::ARRAY_BUFFER, m_arrayBuffer);
    743     m_context.vertexAttribPointer(m_vertexAttribIndex, m_vertexAttribSize, m_vertexAttribType, m_vertexAttribNormalized, m_vertexAttribStride, m_vertexAttribPointer);
    744 
    745     m_context.activeTexture(m_activeTextureUnit);
    746     m_context.bindTexture(m_boundTarget, m_boundTexture);
    747     m_context.bindFramebuffer(GraphicsContext3D::FRAMEBUFFER, m_framebuffer);
    748     m_context.useProgram(m_program);
    749     m_context.viewport(m_viewport[0], m_viewport[1], m_viewport[2], m_viewport[3]);
    750 }
    751 
    752 void VideoTextureCopierCV::GC3DStateSaver::saveVertexAttribState(GC3Duint index)
    753 {
    754     m_vertexAttribIndex = index;
    755     m_context.getVertexAttribiv(index, GraphicsContext3D::VERTEX_ATTRIB_ARRAY_ENABLED, &m_vertexAttribEnabled);
    756     m_context.getVertexAttribiv(index, GraphicsContext3D::VERTEX_ATTRIB_ARRAY_SIZE, &m_vertexAttribSize);
    757     m_context.getVertexAttribiv(index, GraphicsContext3D::VERTEX_ATTRIB_ARRAY_TYPE, &m_vertexAttribType);
    758     m_context.getVertexAttribiv(index, GraphicsContext3D::VERTEX_ATTRIB_ARRAY_NORMALIZED, &m_vertexAttribNormalized);
    759     m_context.getVertexAttribiv(index, GraphicsContext3D::VERTEX_ATTRIB_ARRAY_STRIDE, &m_vertexAttribStride);
    760     m_vertexAttribPointer = m_context.getVertexAttribOffset(index, GraphicsContext3D::VERTEX_ATTRIB_ARRAY_POINTER);
    761 }
    762 
    763 }
     742}

  • trunk/Source/WebCore/platform/graphics/cv/VideoTextureCopierCV.h

    r224207 r225638  
    5151
    5252    bool copyImageToPlatformTexture(CVPixelBufferRef, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY);
    53     bool copyVideoTextureToPlatformTexture(TextureType, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY);
     53    bool copyVideoTextureToPlatformTexture(TextureType, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY, bool swapColorChannels = false);
    5454
    55     GraphicsContext3D& context() { return m_context.get(); }
     55    GraphicsContext3D& context() { return m_context; }
    5656
    5757private:
    58     bool copyVideoTextureToPlatformTexture(Platform3DObject inputTexture, GC3Denum inputTarget, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY);
    59 
    60     class GC3DStateSaver {
    61     public:
    62         GC3DStateSaver(GraphicsContext3D&);
    63         ~GC3DStateSaver();
    64 
    65         void saveVertexAttribState(GC3Duint index);
    66 
    67     private:
    68         GraphicsContext3D& m_context;
    69         GC3Denum m_activeTextureUnit { 0 };
    70         GC3Denum m_boundTarget { 0 };
    71         GC3Denum m_boundTexture { 0 };
    72         GC3Dint m_framebuffer { 0 };
    73         GC3Dint m_program { 0 };
    74         GC3Dint m_arrayBuffer { 0 };
    75         GC3Dint m_viewport[4] { 0, 0, 0, 0 };
    76 
    77         GC3Duint m_vertexAttribIndex { 0 };
    78         GC3Dint m_vertexAttribEnabled { 0 };
    79         GC3Dint m_vertexAttribSize { 0 };
    80         GC3Dint m_vertexAttribType { 0 };
    81         GC3Dint m_vertexAttribNormalized { 0 };
    82         GC3Dint m_vertexAttribStride { 0 };
    83         GC3Dsizeiptr m_vertexAttribPointer { 0 };
    84     };
     58    bool copyVideoTextureToPlatformTexture(Platform3DObject inputTexture, GC3Denum inputTarget, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY, bool swapColorChannels);
    8559
    8660    bool initializeContextObjects();
    … …  
    9569#endif
    9670
     71    Ref<GraphicsContext3D> m_sharedContext;
    9772    Ref<GraphicsContext3D> m_context;
    9873    std::unique_ptr<TextureCacheCV> m_textureCache;
    … …  
    10378    GC3Dint m_textureDimensionsUniformLocation { -1 };
    10479    GC3Dint m_flipYUniformLocation { -1 };
     80    GC3Dint m_swapColorChannelsUniformLocation { -1 };
    10581    GC3Dint m_premultiplyUniformLocation { -1 };
    10682    GC3Dint m_positionAttributeLocation { -1 };

  • trunk/Source/WebCore/platform/graphics/opengl/GraphicsContext3DOpenGLES.cpp

    r219553 r225638  
    275275}
    276276
    277 GraphicsContext3D::GraphicsContext3D(GraphicsContext3DAttributes attributes, HostWindow*, GraphicsContext3D::RenderStyle renderStyle)
     277GraphicsContext3D::GraphicsContext3D(GraphicsContext3DAttributes attributes, HostWindow*, GraphicsContext3D::RenderStyle renderStyle, GraphicsContext3D* sharedContext)
    278278    : m_attrs(attributes)
    279279    , m_compiler(isGLES2Compliant() ? SH_ESSL_OUTPUT : SH_GLSL_COMPATIBILITY_OUTPUT)
    280280    , m_private(std::make_unique<GraphicsContext3DPrivate>(this, renderStyle))
    281281{
     282    ASSERT_UNUSED(sharedContext, !sharedContext);
    282283    makeContextCurrent();
    283284   
Note: See TracChangeset for help on using the changeset viewer.