Changeset 222197 in webkit


Ignore:
Timestamp:
Sep 18, 2017 7:52:38 PM (7 years ago)
Author:
dino@apple.com
Message:

[WebGL] accelerated texImage2D for video doesn't respect flipY
https://bugs.webkit.org/show_bug.cgi?id=176491
<rdar://problem/33833511>

Reviewed by Jer Noble.

(Take 3 - this was rolled out due to a test failure)

Source/WebCore:

Previously, if UNPACK_FLIP_Y_WEBGL was set to true, we'd either fall
back to software or fail to upload texture data. Fix this by intercepting
the texImage2D call, checking the orientation of the video, and running
a small shader program to flip it if necessary.

While there, implement UNPACK_PREMULTIPLY_ALPHA_WEBGL as well, although
none of our media decoders support video with alpha, so unfortunately
this will have no visible change.

Tests: fast/canvas/webgl/texImage2D-video-flipY-false.html

fast/canvas/webgl/texImage2D-video-flipY-true.html

  • platform/cocoa/CoreVideoSoftLink.cpp: Add link to CVOpenGL(ES)TextureGetCleanTexCoords,

which is used to check the orientation of the source video.

  • platform/cocoa/CoreVideoSoftLink.h:
  • platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:

(WebCore::MediaPlayerPrivateAVFoundationObjC::copyVideoTextureToPlatformTexture): We can
now handle flipped or premultiplied requests.

  • platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:

(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::copyVideoTextureToPlatformTexture): Ditto.

  • platform/graphics/cv/VideoTextureCopierCV.cpp:

(WebCore::VideoTextureCopierCV::VideoTextureCopierCV): Rename readFramebuffer to
simply framebuffer.
(WebCore::VideoTextureCopierCV::~VideoTextureCopierCV): Delete the program and buffer
if they were created.
(WebCore::VideoTextureCopierCV::initializeContextObjects): Sets up the shader program
and the vertex buffer for drawing. Also records the location of the uniforms.
(WebCore::VideoTextureCopierCV::copyVideoTextureToPlatformTexture): Create a new
framebuffer object, and render the video texture into that framebuffer using a
shader that can flip the coordinates.
(WebCore::VideoTextureCopierCV::GC3DStateSaver::GC3DStateSaver): Helper to restore
the state of the user's GraphicsContext3D while we're intercepting calls.
(WebCore::VideoTextureCopierCV::GC3DStateSaver::~GC3DStateSaver):

  • platform/graphics/cv/VideoTextureCopierCV.h:
  • platform/graphics/GraphicsContext3D.h: Add two new entry points, for direct shader

compilation and attribute access. This avoids going through ANGLE.

  • platform/graphics/opengl/GraphicsContext3DOpenGLCommon.cpp:

(WebCore::GraphicsContext3D::compileShader):
(WebCore::GraphicsContext3D::compileShaderDirect):
(WebCore::GraphicsContext3D::getAttribLocationDirect):

LayoutTests:

Test that exercises UNPACK_FLIP_Y_WEBGL for video on the accelerated
path.

  • fast/canvas/webgl/resources/orientation-flipped.mp4: Added.
  • fast/canvas/webgl/resources/orientation-normal.mp4: Added.
  • fast/canvas/webgl/texImage2D-video-flipY-false-expected.txt: Added.
  • fast/canvas/webgl/texImage2D-video-flipY-false.html: Added.
  • fast/canvas/webgl/texImage2D-video-flipY-true-expected.txt: Added.
  • fast/canvas/webgl/texImage2D-video-flipY-true.html: Added.
  • platform/ios/TestExpectations: This test is macOS only.
  • platform/mac/TestExpectations: Mark an existing WebGL test as flakey, while a bug exposed here is investigated.
Location:
trunk
Files:
6 added
12 edited

Legend:

Unmodified
Added
Removed
  • trunk/LayoutTests/ChangeLog

    r222193 r222197  
     12017-09-19  Dean Jackson  <dino@apple.com>
     2
     3        [WebGL] accelerated texImage2D for video doesn't respect flipY
     4        https://bugs.webkit.org/show_bug.cgi?id=176491
     5        <rdar://problem/33833511>
     6
     7        Reviewed by Jer Noble.
     8
     9        (Take 3 - this was rolled out due to a test failure)
     10
     11        Test that exercises UNPACK_FLIP_Y_WEBGL for video on the accelerated
     12        path.
     13
     14        * fast/canvas/webgl/resources/orientation-flipped.mp4: Added.
     15        * fast/canvas/webgl/resources/orientation-normal.mp4: Added.
     16        * fast/canvas/webgl/texImage2D-video-flipY-false-expected.txt: Added.
     17        * fast/canvas/webgl/texImage2D-video-flipY-false.html: Added.
     18        * fast/canvas/webgl/texImage2D-video-flipY-true-expected.txt: Added.
     19        * fast/canvas/webgl/texImage2D-video-flipY-true.html: Added.
     20        * platform/ios/TestExpectations: This test is macOS only.
     21        * platform/mac/TestExpectations: Mark an existing WebGL test as flakey, while
     22          a bug exposed here is investigated.
     23
    1242017-09-18  Ryan Haddad  <ryanhaddad@apple.com>
    225
  • trunk/LayoutTests/platform/ios/TestExpectations

    r222176 r222197  
    29502950webgl/1.0.2/conformance/uniforms/out-of-bounds-uniform-array-access.html [ Skip ]
    29512951
     2952fast/canvas/webgl/texImage2D-video-flipY-false.html [ Skip ]
     2953fast/canvas/webgl/texImage2D-video-flipY-true.html [ Skip ]
     2954
    29522955webkit.org/b/174120 http/tests/loading/resourceLoadStatistics/user-interaction-in-cross-origin-sub-frame.html [ Skip ]
    29532956
  • trunk/LayoutTests/platform/mac/TestExpectations

    r222193 r222197  
    17771777# <rdar://problem/33850189>
    17781778[ HighSierra+ ] http/tests/websocket/tests/hybi/deflate-frame-parameter.html [ Failure ]
     1779
     1780# <rdar://problem/34507977>
     1781webkit.org/b/177119 webgl/1.0.2/conformance/textures/tex-image-and-sub-image-2d-with-video.html [ Pass Failure ]
  • trunk/Source/WebCore/ChangeLog

    r222196 r222197  
     12017-09-19  Dean Jackson  <dino@apple.com>
     2
     3        [WebGL] accelerated texImage2D for video doesn't respect flipY
     4        https://bugs.webkit.org/show_bug.cgi?id=176491
     5        <rdar://problem/33833511>
     6
     7        Reviewed by Jer Noble.
     8
     9        (Take 3 - this was rolled out due to a test failure)
     10
     11        Previously, if UNPACK_FLIP_Y_WEBGL was set to true, we'd either fall
     12        back to software or fail to upload texture data. Fix this by intercepting
     13        the texImage2D call, checking the orientation of the video, and running
     14        a small shader program to flip it if necessary.
     15
     16        While there, implement UNPACK_PREMULTIPLY_ALPHA_WEBGL as well, although
     17        none of our media decoders support video with alpha, so unfortunately
     18        this will have no visible change.
     19
     20        Tests: fast/canvas/webgl/texImage2D-video-flipY-false.html
     21               fast/canvas/webgl/texImage2D-video-flipY-true.html
     22
     23        * platform/cocoa/CoreVideoSoftLink.cpp: Add link to CVOpenGL(ES)TextureGetCleanTexCoords,
     24        which is used to check the orientation of the source video.
     25        * platform/cocoa/CoreVideoSoftLink.h:
     26
     27        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
     28        (WebCore::MediaPlayerPrivateAVFoundationObjC::copyVideoTextureToPlatformTexture): We can
     29        now handle flipped or premultiplied requests.
     30        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
     31        (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::copyVideoTextureToPlatformTexture): Ditto.
     32
     33        * platform/graphics/cv/VideoTextureCopierCV.cpp:
     34        (WebCore::VideoTextureCopierCV::VideoTextureCopierCV): Rename readFramebuffer to
     35        simply framebuffer.
     36        (WebCore::VideoTextureCopierCV::~VideoTextureCopierCV): Delete the program and buffer
     37        if they were created.
     38        (WebCore::VideoTextureCopierCV::initializeContextObjects): Sets up the shader program
     39        and the vertex buffer for drawing. Also records the location of the uniforms.
     40        (WebCore::VideoTextureCopierCV::copyVideoTextureToPlatformTexture): Create a new
     41        framebuffer object, and render the video texture into that framebuffer using a
     42        shader that can flip the coordinates.
     43        (WebCore::VideoTextureCopierCV::GC3DStateSaver::GC3DStateSaver): Helper to restore
     44        the state of the user's GraphicsContext3D while we're intercepting calls.
     45        (WebCore::VideoTextureCopierCV::GC3DStateSaver::~GC3DStateSaver):
     46        * platform/graphics/cv/VideoTextureCopierCV.h:
     47
     48        * platform/graphics/GraphicsContext3D.h: Add two new entry points, for direct shader
     49        compilation and attribute access. This avoids going through ANGLE.
     50        * platform/graphics/opengl/GraphicsContext3DOpenGLCommon.cpp:
     51        (WebCore::GraphicsContext3D::compileShader):
     52        (WebCore::GraphicsContext3D::compileShaderDirect):
     53        (WebCore::GraphicsContext3D::getAttribLocationDirect):
     54
    1552017-09-18  Said Abou-Hallawa  <sabouhallawa@apple.com>
    256
  • trunk/Source/WebCore/platform/cocoa/CoreVideoSoftLink.cpp

    r222069 r222197  
    6262SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVPixelBufferCreate, CVReturn, (CFAllocatorRef allocator, size_t width, size_t height, OSType pixelFormatType, CFDictionaryRef pixelBufferAttributes, CVPixelBufferRef *pixelBufferOut), (allocator, width, height, pixelFormatType, pixelBufferAttributes, pixelBufferOut))
    6363SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVPixelBufferCreateWithBytes, CVReturn, (CFAllocatorRef allocator, size_t width, size_t height, OSType pixelFormatType, void* data, size_t bytesPerRow, void (*releaseCallback)(void*, const void*), void* releasePointer, CFDictionaryRef pixelBufferAttributes, CVPixelBufferRef *pixelBufferOut), (allocator, width, height, pixelFormatType, data, bytesPerRow, releaseCallback, releasePointer, pixelBufferAttributes, pixelBufferOut))
     64SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVOpenGLESTextureGetCleanTexCoords, void, (CVOpenGLESTextureRef image, GLfloat lowerLeft[2], GLfloat lowerRight[2], GLfloat upperLeft[2], GLfloat upperRight[2]), (image, lowerLeft, lowerRight, upperLeft, upperRight))
    6465#else
    6566SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVOpenGLTextureCacheCreate, CVReturn, (CFAllocatorRef allocator, CFDictionaryRef cacheAttributes, CGLContextObj cglContext, CGLPixelFormatObj cglPixelFormat, CFDictionaryRef textureAttributes, CVOpenGLTextureCacheRef* cacheOut), (allocator, cacheAttributes, cglContext, cglPixelFormat, textureAttributes, cacheOut))
     
    6869SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVOpenGLTextureGetTarget, GLenum, (CVOpenGLTextureRef image), (image))
    6970SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVOpenGLTextureGetName, GLuint, (CVOpenGLTextureRef image), (image))
     71SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreVideo, CVOpenGLTextureGetCleanTexCoords, void, (CVOpenGLTextureRef image, GLfloat lowerLeft[2], GLfloat lowerRight[2], GLfloat upperLeft[2], GLfloat upperRight[2]), (image, lowerLeft, lowerRight, upperLeft, upperRight))
    7072SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, CoreVideo, kCVPixelBufferIOSurfaceOpenGLFBOCompatibilityKey, CFStringRef)
    7173#endif
  • trunk/Source/WebCore/platform/cocoa/CoreVideoSoftLink.h

    r222069 r222197  
    8787SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVPixelBufferCreateWithBytes, CVReturn, (CFAllocatorRef allocator, size_t width, size_t height, OSType pixelFormatType, void* data, size_t bytesPerRow, void (*releaseCallback)(void*, const void*), void* releasePointer, CFDictionaryRef pixelBufferAttributes, CVPixelBufferRef *pixelBufferOut), (allocator, width, height, pixelFormatType, data, bytesPerRow, releaseCallback, releasePointer, pixelBufferAttributes, pixelBufferOut))
    8888#define CVPixelBufferCreateWithBytes softLink_CoreVideo_CVPixelBufferCreateWithBytes
     89SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVOpenGLESTextureGetCleanTexCoords, void, (CVOpenGLESTextureRef image, GLfloat lowerLeft[2], GLfloat lowerRight[2], GLfloat upperLeft[2], GLfloat upperRight[2]), (image, lowerLeft, lowerRight, upperLeft, upperRight))
     90#define CVOpenGLESTextureGetCleanTexCoords softLink_CoreVideo_CVOpenGLESTextureGetCleanTexCoords
    8991
    9092SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, CoreVideo, kCVPixelBufferCGBitmapContextCompatibilityKey, CFStringRef)
     
    105107SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVOpenGLTextureGetName, GLuint, (CVOpenGLTextureRef image), (image))
    106108#define CVOpenGLTextureGetName softLink_CoreVideo_CVOpenGLTextureGetName
     109SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVOpenGLTextureGetCleanTexCoords, void, (CVOpenGLTextureRef image, GLfloat lowerLeft[2], GLfloat lowerRight[2], GLfloat upperLeft[2], GLfloat upperRight[2]), (image, lowerLeft, lowerRight, upperLeft, upperRight))
     110#define CVOpenGLTextureGetCleanTexCoords softLink_CoreVideo_CVOpenGLTextureGetCleanTexCoords
    107111
    108112SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, CoreVideo, kCVPixelBufferIOSurfaceOpenGLFBOCompatibilityKey, CFStringRef)
  • trunk/Source/WebCore/platform/graphics/GraphicsContext3D.h

    r222069 r222197  
    765765    void texImage2DDirect(GC3Denum target, GC3Dint level, GC3Denum internalformat, GC3Dsizei width, GC3Dsizei height, GC3Dint border, GC3Denum format, GC3Denum type, const void* pixels);
    766766
     767    // Get an attribute location without checking the name -> mangledname mapping.
     768    int getAttribLocationDirect(Platform3DObject program, const String& name);
     769
     770    // Compile a shader without going through ANGLE.
     771    void compileShaderDirect(Platform3DObject);
     772
    767773    // Helper to texImage2D with pixel==0 case: pixels are initialized to 0.
    768774    // Return true if no GL error is synthesized.
  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm

    r222092 r222197  
    24822482bool MediaPlayerPrivateAVFoundationObjC::copyVideoTextureToPlatformTexture(GraphicsContext3D* context, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY)
    24832483{
    2484     if (flipY || premultiplyAlpha)
    2485         return false;
    2486 
    24872484    ASSERT(context);
    24882485
  • trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm

    r222069 r222197  
    600600bool MediaPlayerPrivateMediaSourceAVFObjC::copyVideoTextureToPlatformTexture(GraphicsContext3D* context, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY)
    601601{
    602     if (flipY || premultiplyAlpha)
    603         return false;
    604 
    605602    // We have been asked to paint into a WebGL canvas, so take that as a signal to create
    606603    // a decompression session, even if that means the native video can't also be displayed
  • trunk/Source/WebCore/platform/graphics/cv/VideoTextureCopierCV.cpp

    r222069 r222197  
    2929#include "Logging.h"
    3030#include <wtf/NeverDestroyed.h>
     31#include <wtf/text/StringBuilder.h>
    3132
    3233#if PLATFORM(IOS)
     
    4041VideoTextureCopierCV::VideoTextureCopierCV(GraphicsContext3D& context)
    4142    : m_context(context)
    42     , m_readFramebuffer(context.createFramebuffer())
     43    , m_framebuffer(context.createFramebuffer())
    4344{
    4445}
     
    4647VideoTextureCopierCV::~VideoTextureCopierCV()
    4748{
    48     m_context->deleteFramebuffer(m_readFramebuffer);
     49    if (m_vertexBuffer)
     50        m_context->deleteProgram(m_vertexBuffer);
     51    if (m_program)
     52        m_context->deleteProgram(m_program);
     53    m_context->deleteFramebuffer(m_framebuffer);
    4954}
    5055
     
    153158#endif
    154159
    155 bool VideoTextureCopierCV::copyVideoTextureToPlatformTexture(TextureType inputTexture, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY)
    156 {
    157     if (flipY || premultiplyAlpha)
    158         return false;
    159 
    160     if (!inputTexture)
    161         return false;
    162 
    163 #if PLATFORM(IOS)
    164     Platform3DObject videoTextureName = CVOpenGLESTextureGetName(inputTexture);
    165     GC3Denum videoTextureTarget = CVOpenGLESTextureGetTarget(inputTexture);
     160bool VideoTextureCopierCV::initializeContextObjects()
     161{
     162    StringBuilder vertexShaderSource;
     163    vertexShaderSource.appendLiteral("attribute vec4 a_position;\n");
     164    vertexShaderSource.appendLiteral("uniform int u_flipY;\n");
     165    vertexShaderSource.appendLiteral("varying vec2 v_texturePosition;\n");
     166    vertexShaderSource.appendLiteral("void main() {\n");
     167    vertexShaderSource.appendLiteral("    v_texturePosition = vec2((a_position.x + 1.0) / 2.0, (a_position.y + 1.0) / 2.0);\n");
     168    vertexShaderSource.appendLiteral("    if (u_flipY == 1) {\n");
     169    vertexShaderSource.appendLiteral("        v_texturePosition.y = 1.0 - v_texturePosition.y;\n");
     170    vertexShaderSource.appendLiteral("    }\n");
     171    vertexShaderSource.appendLiteral("    gl_Position = a_position;\n");
     172    vertexShaderSource.appendLiteral("}\n");
     173
     174    Platform3DObject vertexShader = m_context->createShader(GraphicsContext3D::VERTEX_SHADER);
     175    m_context->shaderSource(vertexShader, vertexShaderSource.toString());
     176    m_context->compileShaderDirect(vertexShader);
     177
     178    GC3Dint value = 0;
     179    m_context->getShaderiv(vertexShader, GraphicsContext3D::COMPILE_STATUS, &value);
     180    if (!value) {
     181        LOG(WebGL, "VideoTextureCopierCV::copyVideoTextureToPlatformTexture(%p) - Vertex shader failed to compile.", this);
     182        m_context->deleteShader(vertexShader);
     183        return false;
     184    }
     185
     186    StringBuilder fragmentShaderSource;
     187
     188#if PLATFORM(IOS)
     189    fragmentShaderSource.appendLiteral("precision mediump float;\n");
     190    fragmentShaderSource.appendLiteral("uniform sampler2D u_texture;\n");
    166191#else
    167     Platform3DObject videoTextureName = CVOpenGLTextureGetName(inputTexture);
    168     GC3Denum videoTextureTarget = CVOpenGLTextureGetTarget(inputTexture);
    169 #endif
    170 
    171     LOG(Media, "VideoTextureCopierCV::copyVideoTextureToPlatformTexture(%p) - internalFormat: %s, format: %s, type: %s", this, enumToStringMap()[internalFormat], enumToStringMap()[format], enumToStringMap()[type]);
    172 
    173     // Save the origial bound texture & framebuffer names so we can re-bind them after copying the video texture.
    174     GC3Dint boundTexture = 0;
    175     GC3Dint boundReadFramebuffer = 0;
    176     m_context->getIntegerv(GraphicsContext3D::TEXTURE_BINDING_2D, &boundTexture);
    177     m_context->getIntegerv(GraphicsContext3D::READ_FRAMEBUFFER_BINDING, &boundReadFramebuffer);
    178 
     192    fragmentShaderSource.appendLiteral("uniform sampler2DRect u_texture;\n");
     193#endif
     194    fragmentShaderSource.appendLiteral("varying vec2 v_texturePosition;\n");
     195    fragmentShaderSource.appendLiteral("uniform int u_premultiply;\n");
     196    fragmentShaderSource.appendLiteral("uniform vec2 u_textureDimensions;\n");
     197    fragmentShaderSource.appendLiteral("void main() {\n");
     198    fragmentShaderSource.appendLiteral("    vec2 texPos = vec2(v_texturePosition.x * u_textureDimensions.x, v_texturePosition.y * u_textureDimensions.y);\n");
     199#if PLATFORM(IOS)
     200    fragmentShaderSource.appendLiteral("    vec4 color = texture2D(u_texture, texPos);\n");
     201#else
     202    fragmentShaderSource.appendLiteral("    vec4 color = texture2DRect(u_texture, texPos);\n");
     203#endif
     204    fragmentShaderSource.appendLiteral("    if (u_premultiply == 1) {\n");
     205    fragmentShaderSource.appendLiteral("        gl_FragColor = vec4(color.r * color.a, color.g * color.a, color.b * color.a, color.a);\n");
     206    fragmentShaderSource.appendLiteral("    } else {\n");
     207    fragmentShaderSource.appendLiteral("        gl_FragColor = color;\n");
     208    fragmentShaderSource.appendLiteral("    }\n");
     209    fragmentShaderSource.appendLiteral("}\n");
     210
     211    Platform3DObject fragmentShader = m_context->createShader(GraphicsContext3D::FRAGMENT_SHADER);
     212    m_context->shaderSource(fragmentShader, fragmentShaderSource.toString());
     213    m_context->compileShaderDirect(fragmentShader);
     214
     215    m_context->getShaderiv(fragmentShader, GraphicsContext3D::COMPILE_STATUS, &value);
     216    if (!value) {
     217        LOG(WebGL, "VideoTextureCopierCV::copyVideoTextureToPlatformTexture(%p) - Fragment shader failed to compile.", this);
     218        m_context->deleteShader(vertexShader);
     219        m_context->deleteShader(fragmentShader);
     220        return false;
     221    }
     222
     223    m_program = m_context->createProgram();
     224    m_context->attachShader(m_program, vertexShader);
     225    m_context->attachShader(m_program, fragmentShader);
     226    m_context->linkProgram(m_program);
     227
     228    m_context->getProgramiv(m_program, GraphicsContext3D::LINK_STATUS, &value);
     229    if (!value) {
     230        LOG(WebGL, "VideoTextureCopierCV::copyVideoTextureToPlatformTexture(%p) - Program failed to link.", this);
     231        m_context->deleteShader(vertexShader);
     232        m_context->deleteShader(fragmentShader);
     233        m_context->deleteProgram(m_program);
     234        m_program = 0;
     235        return false;
     236    }
     237
     238    m_textureUniformLocation = m_context->getUniformLocation(m_program, ASCIILiteral("u_texture"));
     239    m_textureDimensionsUniformLocation = m_context->getUniformLocation(m_program, ASCIILiteral("u_textureDimensions"));
     240    m_flipYUniformLocation = m_context->getUniformLocation(m_program, ASCIILiteral("u_flipY"));
     241    m_premultiplyUniformLocation = m_context->getUniformLocation(m_program, ASCIILiteral("u_premultiply"));
     242    m_positionAttributeLocation = m_context->getAttribLocationDirect(m_program, ASCIILiteral("a_position"));
     243
     244    m_context->detachShader(m_program, vertexShader);
     245    m_context->detachShader(m_program, fragmentShader);
     246    m_context->deleteShader(vertexShader);
     247    m_context->deleteShader(fragmentShader);
     248
     249    LOG(WebGL, "Uniform and Attribute locations: u_texture = %d, u_textureDimensions = %d, u_flipY = %d, u_premultiply = %d, a_position = %d", m_textureUniformLocation, m_textureDimensionsUniformLocation, m_flipYUniformLocation, m_premultiplyUniformLocation, m_positionAttributeLocation);
     250    m_context->enableVertexAttribArray(m_positionAttributeLocation);
     251
     252    m_vertexBuffer = m_context->createBuffer();
     253    float vertices[12] = { -1, -1, 1, -1, 1, 1, 1, 1, -1, 1, -1, -1 };
     254
     255    m_context->bindBuffer(GraphicsContext3D::ARRAY_BUFFER, m_vertexBuffer);
     256    m_context->bufferData(GraphicsContext3D::ARRAY_BUFFER, sizeof(float) * 12, vertices, GraphicsContext3D::STATIC_DRAW);
     257
     258    return true;
     259}
     260
     261bool VideoTextureCopierCV::copyVideoTextureToPlatformTexture(TextureType inputVideoTexture, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY)
     262{
     263    if (!inputVideoTexture)
     264        return false;
     265
     266    GC3DStateSaver stateSaver(&m_context.get());
     267
     268    if (!m_program) {
     269        if (!initializeContextObjects()) {
     270            LOG(WebGL, "VideoTextureCopierCV::copyVideoTextureToPlatformTexture(%p) - Unable to initialize OpenGL context objects.", this);
     271            return false;
     272        }
     273    }
     274
     275    GLfloat lowerLeft[2] = { 0, 0 };
     276    GLfloat lowerRight[2] = { 0, 0 };
     277    GLfloat upperRight[2] = { 0, 0 };
     278    GLfloat upperLeft[2] = { 0, 0 };
     279#if PLATFORM(IOS)
     280    Platform3DObject videoTextureName = CVOpenGLESTextureGetName(inputVideoTexture);
     281    GC3Denum videoTextureTarget = CVOpenGLESTextureGetTarget(inputVideoTexture);
     282    CVOpenGLESTextureGetCleanTexCoords(inputVideoTexture, lowerLeft, lowerRight, upperRight, upperLeft);
     283#else
     284    Platform3DObject videoTextureName = CVOpenGLTextureGetName(inputVideoTexture);
     285    GC3Denum videoTextureTarget = CVOpenGLTextureGetTarget(inputVideoTexture);
     286    CVOpenGLTextureGetCleanTexCoords(inputVideoTexture, lowerLeft, lowerRight, upperRight, upperLeft);
     287#endif
     288
     289    LOG(WebGL, "VideoTextureCopierCV::copyVideoTextureToPlatformTexture(%p) - internalFormat: %s, format: %s, type: %s flipY: %s, premultiplyAlpha: %s", this, enumToStringMap()[internalFormat], enumToStringMap()[format], enumToStringMap()[type], flipY ? "true" : "false", premultiplyAlpha ? "true" : "false");
     290
     291    m_context->bindFramebuffer(GraphicsContext3D::FRAMEBUFFER, m_framebuffer);
     292   
     293    // Allocate memory for the output texture.
     294    m_context->bindTexture(GraphicsContext3D::TEXTURE_2D, outputTexture);
     295    m_context->texParameteri(GraphicsContext3D::TEXTURE_2D, GraphicsContext3D::TEXTURE_MAG_FILTER, GraphicsContext3D::LINEAR);
     296    m_context->texParameteri(GraphicsContext3D::TEXTURE_2D, GraphicsContext3D::TEXTURE_MIN_FILTER, GraphicsContext3D::LINEAR);
     297    m_context->texParameteri(GraphicsContext3D::TEXTURE_2D, GraphicsContext3D::TEXTURE_WRAP_S, GraphicsContext3D::CLAMP_TO_EDGE);
     298    m_context->texParameteri(GraphicsContext3D::TEXTURE_2D, GraphicsContext3D::TEXTURE_WRAP_T, GraphicsContext3D::CLAMP_TO_EDGE);
     299    m_context->texImage2DDirect(GraphicsContext3D::TEXTURE_2D, level, internalFormat, width, height, 0, format, type, nullptr);
     300
     301    m_context->framebufferTexture2D(GraphicsContext3D::FRAMEBUFFER, GraphicsContext3D::COLOR_ATTACHMENT0, GraphicsContext3D::TEXTURE_2D, outputTexture, level);
     302    GC3Denum status = m_context->checkFramebufferStatus(GraphicsContext3D::FRAMEBUFFER);
     303    if (status != GraphicsContext3D::FRAMEBUFFER_COMPLETE) {
     304        LOG(WebGL, "VideoTextureCopierCV::copyVideoTextureToPlatformTexture(%p) - Unable to create framebuffer for outputTexture.", this);
     305        return false;
     306    }
     307
     308    m_context->useProgram(m_program);
     309    m_context->viewport(0, 0, width, height);
     310
     311    // Bind and set up the texture for the video source.
     312    m_context->activeTexture(GraphicsContext3D::TEXTURE0);
    179313    m_context->bindTexture(videoTextureTarget, videoTextureName);
    180     m_context->texParameteri(GraphicsContext3D::TEXTURE_2D, GraphicsContext3D::TEXTURE_MIN_FILTER, GraphicsContext3D::LINEAR);
    181     m_context->texParameterf(GraphicsContext3D::TEXTURE_2D, GraphicsContext3D::TEXTURE_WRAP_S, GraphicsContext3D::CLAMP_TO_EDGE);
    182     m_context->texParameterf(GraphicsContext3D::TEXTURE_2D, GraphicsContext3D::TEXTURE_WRAP_T, GraphicsContext3D::CLAMP_TO_EDGE);
    183 
    184     // Make that framebuffer the read source from which drawing commands will read voxels.
    185     m_context->bindFramebuffer(GraphicsContext3D::READ_FRAMEBUFFER, m_readFramebuffer);
    186 
    187     // Allocate uninitialized memory for the output texture.
     314    m_context->texParameteri(videoTextureTarget, GraphicsContext3D::TEXTURE_MAG_FILTER, GraphicsContext3D::LINEAR);
     315    m_context->texParameteri(videoTextureTarget, GraphicsContext3D::TEXTURE_MIN_FILTER, GraphicsContext3D::LINEAR);
     316    m_context->texParameteri(videoTextureTarget, GraphicsContext3D::TEXTURE_WRAP_S, GraphicsContext3D::CLAMP_TO_EDGE);
     317    m_context->texParameteri(videoTextureTarget, GraphicsContext3D::TEXTURE_WRAP_T, GraphicsContext3D::CLAMP_TO_EDGE);
     318
     319    // Configure the drawing parameters.
     320    m_context->uniform1i(m_textureUniformLocation, 0);
     321#if PLATFORM(IOS)
     322    m_context->uniform2f(m_textureDimensionsUniformLocation, 1, 1);
     323#else
     324    m_context->uniform2f(m_textureDimensionsUniformLocation, width, height);
     325#endif
     326
     327    if (lowerLeft[1] < upperRight[1])
     328        flipY = !flipY;
     329
     330    m_context->uniform1i(m_flipYUniformLocation, flipY);
     331    m_context->uniform1i(m_premultiplyUniformLocation, premultiplyAlpha);
     332
     333    // Do the actual drawing.
     334    m_context->enableVertexAttribArray(m_positionAttributeLocation);
     335    m_context->bindBuffer(GraphicsContext3D::ARRAY_BUFFER, m_vertexBuffer);
     336    m_context->vertexAttribPointer(m_positionAttributeLocation, 2, GraphicsContext3D::FLOAT, false, 0, 0);
     337    m_context->drawArrays(GraphicsContext3D::TRIANGLES, 0, 6);
     338
     339    // Clean-up.
     340    m_context->bindTexture(videoTextureTarget, 0);
    188341    m_context->bindTexture(outputTarget, outputTexture);
    189     m_context->texParameteri(GraphicsContext3D::TEXTURE_2D, GraphicsContext3D::TEXTURE_MIN_FILTER, GraphicsContext3D::LINEAR);
    190     m_context->texParameterf(GraphicsContext3D::TEXTURE_2D, GraphicsContext3D::TEXTURE_WRAP_S, GraphicsContext3D::CLAMP_TO_EDGE);
    191     m_context->texParameterf(GraphicsContext3D::TEXTURE_2D, GraphicsContext3D::TEXTURE_WRAP_T, GraphicsContext3D::CLAMP_TO_EDGE);
    192     m_context->texImage2DDirect(outputTarget, level, internalFormat, width, height, 0, format, type, nullptr);
    193 
    194     // Attach the video texture to the framebuffer.
    195     m_context->framebufferTexture2D(GraphicsContext3D::READ_FRAMEBUFFER, GraphicsContext3D::COLOR_ATTACHMENT0, videoTextureTarget, videoTextureName, level);
    196 
    197     GC3Denum status = m_context->checkFramebufferStatus(GraphicsContext3D::READ_FRAMEBUFFER);
    198     if (status != GraphicsContext3D::FRAMEBUFFER_COMPLETE)
    199         return false;
    200 
    201     // Copy texture from the read framebuffer (and thus the video texture) to the output texture.
    202     m_context->copyTexImage2D(outputTarget, level, internalFormat, 0, 0, width, height, 0);
    203 
    204     // Restore the previous texture and framebuffer bindings.
    205     m_context->bindTexture(outputTarget, boundTexture);
    206     m_context->bindFramebuffer(GraphicsContext3D::READ_FRAMEBUFFER, boundReadFramebuffer);
    207 
    208     return !m_context->getError();
    209 }
    210 
    211 
    212 }
     342
     343    return true;
     344}
     345
     346VideoTextureCopierCV::GC3DStateSaver::GC3DStateSaver(GraphicsContext3D* context)
     347    : m_context(context)
     348{
     349    ASSERT(context);
     350    m_context->getIntegerv(GraphicsContext3D::TEXTURE_BINDING_2D, &m_texture);
     351    m_context->getIntegerv(GraphicsContext3D::FRAMEBUFFER_BINDING, &m_framebuffer);
     352    m_context->getIntegerv(GraphicsContext3D::CURRENT_PROGRAM, &m_program);
     353    m_context->getIntegerv(GraphicsContext3D::ARRAY_BUFFER_BINDING, &m_arrayBuffer);
     354    m_context->getIntegerv(GraphicsContext3D::VIEWPORT, m_viewport);
     355}
     356
     357VideoTextureCopierCV::GC3DStateSaver::~GC3DStateSaver()
     358{
     359    m_context->bindTexture(GraphicsContext3D::TEXTURE_BINDING_2D, m_texture);
     360    m_context->bindFramebuffer(GraphicsContext3D::FRAMEBUFFER, m_framebuffer);
     361    m_context->useProgram(m_program);
     362    m_context->bindBuffer(GraphicsContext3D::ARRAY_BUFFER, m_arrayBuffer);
     363    m_context->viewport(m_viewport[0], m_viewport[1], m_viewport[2], m_viewport[3]);
     364}
     365
     366
     367}
  • trunk/Source/WebCore/platform/graphics/cv/VideoTextureCopierCV.h

    r222069 r222197  
    2929#import "GraphicsContext3D.h"
    3030
    31 typedef struct  __CVBuffer* CVImageBufferRef;
     31typedef struct __CVBuffer* CVImageBufferRef;
    3232typedef CVImageBufferRef CVOpenGLTextureRef;
    3333typedef CVImageBufferRef CVOpenGLESTextureRef;
     
    5151
    5252private:
     53    class GC3DStateSaver {
     54    public:
     55        GC3DStateSaver(GraphicsContext3D*);
     56        ~GC3DStateSaver();
     57
     58    private:
     59        GraphicsContext3D* m_context;
     60        GC3Dint m_texture { 0 };
     61        GC3Dint m_framebuffer { 0 };
     62        GC3Dint m_program { 0 };
     63        GC3Dint m_arrayBuffer { 0 };
     64        GC3Dint m_viewport[4] { 0, 0, 0, 0 };
     65    };
     66
     67    bool initializeContextObjects();
     68
    5369    Ref<GraphicsContext3D> m_context;
    54     Platform3DObject m_readFramebuffer;
     70    Platform3DObject m_framebuffer { 0 };
     71    Platform3DObject m_program { 0 };
     72    Platform3DObject m_vertexBuffer { 0 };
     73    GC3Dint m_textureUniformLocation { -1 };
     74    GC3Dint m_textureDimensionsUniformLocation { -1 };
     75    GC3Dint m_flipYUniformLocation { -1 };
     76    GC3Dint m_premultiplyUniformLocation { -1 };
     77    GC3Dint m_positionAttributeLocation { -1 };
    5578};
    5679
  • trunk/Source/WebCore/platform/graphics/opengl/GraphicsContext3DOpenGLCommon.cpp

    r222069 r222197  
    667667    ::glCompileShader(shader);
    668668   
    669     int GLCompileSuccess;
     669    int compileStatus;
    670670   
    671     ::glGetShaderiv(shader, COMPILE_STATUS, &GLCompileSuccess);
     671    ::glGetShaderiv(shader, COMPILE_STATUS, &compileStatus);
    672672
    673673    ShaderSourceMap::iterator result = m_shaderSourceMap.find(shader);
     
    687687    }
    688688
    689     if (GLCompileSuccess != GL_TRUE) {
     689    if (compileStatus != GL_TRUE) {
    690690        entry.isValid = false;
    691691        LOG(WebGL, "Error: shader translator produced a shader that OpenGL would not compile.");
     692    }
     693}
     694
     695void GraphicsContext3D::compileShaderDirect(Platform3DObject shader)
     696{
     697    ASSERT(shader);
     698    makeContextCurrent();
     699
     700    HashMap<Platform3DObject, ShaderSourceEntry>::iterator result = m_shaderSourceMap.find(shader);
     701
     702    if (result == m_shaderSourceMap.end())
     703        return;
     704
     705    ShaderSourceEntry& entry = result->value;
     706
     707    const CString& shaderSourceCString = entry.source.utf8();
     708    const char* shaderSourcePtr = shaderSourceCString.data();
     709    int shaderSourceLength = shaderSourceCString.length();
     710
     711    LOG(WebGL, "--- begin direct shader source ---\n%s\n--- end direct shader source ---\n", shaderSourcePtr);
     712
     713    ::glShaderSource(shader, 1, &shaderSourcePtr, &shaderSourceLength);
     714
     715    ::glCompileShader(shader);
     716
     717    int compileStatus;
     718
     719    ::glGetShaderiv(shader, COMPILE_STATUS, &compileStatus);
     720
     721    if (compileStatus == GL_TRUE) {
     722        entry.isValid = true;
     723        LOG(WebGL, "Direct compilation of shader succeeded.");
     724    } else {
     725        entry.isValid = false;
     726        LOG(WebGL, "Error: direct compilation of shader failed.");
    692727    }
    693728}
     
    10671102    LOG(WebGL, "::glGetAttribLocation is mapping %s to %s", name.utf8().data(), mappedName.utf8().data());
    10681103    return ::glGetAttribLocation(program, mappedName.utf8().data());
     1104}
     1105
     1106int GraphicsContext3D::getAttribLocationDirect(Platform3DObject program, const String& name)
     1107{
     1108    if (!program)
     1109        return -1;
     1110
     1111    makeContextCurrent();
     1112
     1113    return ::glGetAttribLocation(program, name.utf8().data());
    10691114}
    10701115
Note: See TracChangeset for help on using the changeset viewer.