Changeset 225638 in webkit
- Timestamp:
- Dec 7, 2017 11:56:25 AM (7 years ago)
- Location:
- trunk
- Files:
-
- 13 edited
Legend:
- Unmodified
- Added
- Removed
-
trunk/LayoutTests/ChangeLog
r225637 r225638 1 2017-12-07 Jer Noble <jer.noble@apple.com> 2 3 Creating a second AVPlayerItemVideoOutput causes flakey failures 4 https://bugs.webkit.org/show_bug.cgi?id=177119 5 <rdar://problem/34507977> 6 7 Reviewed by Dean Jackson. 8 9 * platform/mac/TestExpectations: 10 1 11 2017-12-07 Jer Noble <jer.noble@apple.com> 2 12 -
trunk/LayoutTests/platform/mac/TestExpectations
r225637 r225638 1726 1726 [ HighSierra+ ] http/tests/websocket/tests/hybi/deflate-frame-parameter.html [ Failure ] 1727 1727 1728 # <rdar://problem/34507977>1729 webkit.org/b/177119 webgl/1.0.2/conformance/textures/tex-image-and-sub-image-2d-with-video.html [ Pass Failure ]1730 1731 1728 # Release logging not supported 1732 1729 [ ElCapitan ] inspector/console/webcore-logging.html [ Skip ] -
trunk/Source/WebCore/ChangeLog
r225637 r225638 1 2017-12-07 Jer Noble <jer.noble@apple.com> 2 3 Creating a second AVPlayerItemVideoOutput causes flakey failures 4 https://bugs.webkit.org/show_bug.cgi?id=177119 5 <rdar://problem/34507977> 6 7 Reviewed by Dean Jackson. 8 9 Rather than having two concurrent AVPlayerItemVideoOutputs when painting to OpenGL, 10 use only one, but with the OpenGL compatibility flags enabled, for both 3D and 2D 11 canvas painting. 12 13 Of course, once this is fixed, the test breaks again because VideoTextureCopierCV is 14 stomping on top of some WebGL state set by the test. Rather than expand the 15 VideoTextureCopierCV's GraphicsStateSaver, use a totally separate (but shared) GL context to 16 implement the texture copy, so that its own state will not affect the state of the WebGL 17 context. 18 19 This necessitates adding a new createShared() factory for GraphicsContext3D. Since this is only 20 used on Cocoa platforms (for now), make it platform specific, and modify the other implementations 21 only so far as to throw assertions if somehow a shared context pointer is passed into their 22 constructors. 23 24 Drive-by fixes: 25 26 - These changes exposed some test failures, which were off-by-1-or-2 errors in the color 27 output of videos painted into WebGL contexts. The Y' parameter needs to be scaled by 16/255 28 for video-range conversions and the CbCr vales by 128/255 (rather than just 0.5). Make the 29 color transforms a full 4x4 matrix and do the scaling in the matrix rather than tracked by a 30 separate uniform. 31 32 - There was a slight error in WebCoreDecompressionSession that could cause seeks to never 33 complete because incoming frames would be skipped if their startTime was in the past 34 (rather than their startTime + duration being in the past). 35 36 - The matrix name comparison should use CFEqual rather than pointer equality to test. 37 38 - iOS currently has a bug where the red and green color channels will be reversed when 39 uploading BGRA pixel formatted data to a texture; work around this bug for now. 40 41 * platform/graphics/GraphicsContext3D.h: 42 * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h: 43 * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm: 44 (WebCore::MediaPlayerPrivateAVFoundationObjC::destroyContextVideoRenderer): 45 (WebCore::MediaPlayerPrivateAVFoundationObjC::hasAvailableVideoFrame const): 46 (WebCore::MediaPlayerPrivateAVFoundationObjC::updateLastPixelBuffer): 47 (WebCore::MediaPlayerPrivateAVFoundationObjC::videoOutputHasAvailableFrame): 48 (WebCore::MediaPlayerPrivateAVFoundationObjC::updateLastImage): 49 (WebCore::MediaPlayerPrivateAVFoundationObjC::paintWithVideoOutput): 50 (WebCore::MediaPlayerPrivateAVFoundationObjC::copyVideoTextureToPlatformTexture): 51 (WebCore::MediaPlayerPrivateAVFoundationObjC::nativeImageForCurrentTime): 52 (WebCore::MediaPlayerPrivateAVFoundationObjC::waitForVideoOutputMediaDataWillChange): 53 * platform/graphics/cairo/GraphicsContext3DCairo.cpp: 54 (WebCore::GraphicsContext3D::GraphicsContext3D): 55 * platform/graphics/cocoa/GraphicsContext3DCocoa.mm: 56 (WebCore::GraphicsContext3D::createShared): 57 (WebCore::GraphicsContext3D::GraphicsContext3D): 58 * platform/graphics/cv/VideoTextureCopierCV.cpp: 59 (WebCore::VideoTextureCopierCV::VideoTextureCopierCV): 60 (WebCore::VideoTextureCopierCV::~VideoTextureCopierCV): 61 (WebCore::transferFunctionFromString): 62 (WebCore::YCbCrToRGBMatrixForRangeAndTransferFunction): 63 (WebCore::VideoTextureCopierCV::initializeUVContextObjects): 64 (WebCore::VideoTextureCopierCV::copyImageToPlatformTexture): 65 (WebCore::VideoTextureCopierCV::copyVideoTextureToPlatformTexture): 66 * platform/graphics/cv/VideoTextureCopierCV.h: 67 (WebCore::VideoTextureCopierCV::context): 68 * platform/graphics/opengl/GraphicsContext3DOpenGLES.cpp: 69 (WebCore::GraphicsContext3D::GraphicsContext3D): 70 1 71 2017-12-07 Jer Noble <jer.noble@apple.com> 2 72 -
trunk/Source/WebCore/platform/graphics/GraphicsContext3D.h
r225300 r225638 746 746 747 747 #if PLATFORM(COCOA) 748 static Ref<GraphicsContext3D> createShared(GraphicsContext3D& sharedContext); 749 #endif 750 751 #if PLATFORM(COCOA) 748 752 PlatformGraphicsContext3D platformGraphicsContext3D() const { return m_contextObj; } 749 753 Platform3DObject platformTexture() const { return m_texture; } … … 1288 1292 1289 1293 private: 1290 GraphicsContext3D(GraphicsContext3DAttributes, HostWindow*, RenderStyle = RenderOffscreen );1294 GraphicsContext3D(GraphicsContext3DAttributes, HostWindow*, RenderStyle = RenderOffscreen, GraphicsContext3D* sharedContext = nullptr); 1291 1295 1292 1296 // Helper for packImageData/extractImageData/extractTextureData which implement packing of pixel -
trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h
r224297 r225638 59 59 typedef struct CGImage *CGImageRef; 60 60 typedef struct __CVBuffer *CVPixelBufferRef; 61 #if PLATFORM(IOS)62 typedef struct __CVOpenGLESTextureCache *CVOpenGLESTextureCacheRef;63 #else64 typedef struct __CVOpenGLTextureCache* CVOpenGLTextureCacheRef;65 #endif66 61 67 62 namespace WebCore { … … 266 261 void createVideoOutput(); 267 262 void destroyVideoOutput(); 268 RetainPtr<CVPixelBufferRef> createPixelBuffer();263 bool updateLastPixelBuffer(); 269 264 void updateLastImage(); 270 265 bool videoOutputHasAvailableFrame(); … … 272 267 NativeImagePtr nativeImageForCurrentTime() override; 273 268 void waitForVideoOutputMediaDataWillChange(); 274 275 void createOpenGLVideoOutput();276 void destroyOpenGLVideoOutput();277 void updateLastOpenGLImage();278 269 279 270 bool copyVideoTextureToPlatformTexture(GraphicsContext3D*, Platform3DObject, GC3Denum target, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY) override; … … 364 355 RetainPtr<AVPlayerItemVideoOutput> m_videoOutput; 365 356 RetainPtr<WebCoreAVFPullDelegate> m_videoOutputDelegate; 357 RetainPtr<CVPixelBufferRef> m_lastPixelBuffer; 366 358 RetainPtr<CGImageRef> m_lastImage; 367 359 dispatch_semaphore_t m_videoOutputSemaphore; 368 369 RetainPtr<AVPlayerItemVideoOutput> m_openGLVideoOutput;370 std::unique_ptr<TextureCacheCV> m_textureCache;371 360 std::unique_ptr<VideoTextureCopierCV> m_videoTextureCopier; 372 RetainPtr<CVPixelBufferRef> m_lastOpenGLImage;373 361 #endif 374 362 -
trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm
r225618 r225638 649 649 #if HAVE(AVFOUNDATION_VIDEO_OUTPUT) 650 650 destroyVideoOutput(); 651 destroyOpenGLVideoOutput();652 651 #endif 653 652 destroyImageGenerator(); … … 753 752 if (currentRenderingMode() == MediaRenderingToLayer) 754 753 return m_cachedIsReadyForDisplay; 754 755 if (m_videoOutput && (m_lastPixelBuffer || [m_videoOutput hasNewPixelBufferForItemTime:[m_avPlayerItem currentTime]])) 756 return true; 755 757 756 758 return m_videoFrameHasDrawn; … … 2293 2295 } 2294 2296 2295 RetainPtr<CVPixelBufferRef> MediaPlayerPrivateAVFoundationObjC::createPixelBuffer()2297 bool MediaPlayerPrivateAVFoundationObjC::updateLastPixelBuffer() 2296 2298 { 2297 2299 if (!m_videoOutput) … … 2302 2304 2303 2305 if (![m_videoOutput.get() hasNewPixelBufferForItemTime:currentTime]) 2304 return 0; 2305 2306 return adoptCF([m_videoOutput.get() copyPixelBufferForItemTime:currentTime itemTimeForDisplay:nil]); 2306 return false; 2307 2308 m_lastPixelBuffer = adoptCF([m_videoOutput.get() copyPixelBufferForItemTime:currentTime itemTimeForDisplay:nil]); 2309 m_lastImage = nullptr; 2310 return true; 2307 2311 } 2308 2312 … … 2323 2327 void MediaPlayerPrivateAVFoundationObjC::updateLastImage() 2324 2328 { 2325 RetainPtr<CVPixelBufferRef> pixelBuffer = createPixelBuffer();2326 2327 2329 // Calls to copyPixelBufferForItemTime:itemTimeForDisplay: may return nil if the pixel buffer 2328 2330 // for the requested time has already been retrieved. In this case, the last valid image (if any) 2329 2331 // should be displayed. 2330 if (! pixelBuffer)2332 if (!updateLastPixelBuffer() && (m_lastImage || !m_lastPixelBuffer)) 2331 2333 return; 2332 2334 … … 2344 2346 #endif 2345 2347 2346 m_lastImage = m_pixelBufferConformer->createImageFromPixelBuffer( pixelBuffer.get());2348 m_lastImage = m_pixelBufferConformer->createImageFromPixelBuffer(m_lastPixelBuffer.get()); 2347 2349 2348 2350 #if !RELEASE_LOG_DISABLED … … 2382 2384 } 2383 2385 2384 void MediaPlayerPrivateAVFoundationObjC::createOpenGLVideoOutput()2385 {2386 INFO_LOG(LOGIDENTIFIER);2387 2388 if (!m_avPlayerItem || m_openGLVideoOutput)2389 return;2390 2391 #if PLATFORM(IOS)2392 NSDictionary* attributes = @{(NSString *)kCVPixelBufferIOSurfaceOpenGLESFBOCompatibilityKey: @YES};2393 #else2394 NSDictionary* attributes = @{(NSString *)kCVPixelBufferIOSurfaceOpenGLFBOCompatibilityKey: @YES};2395 #endif2396 m_openGLVideoOutput = adoptNS([[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:attributes]);2397 ASSERT(m_openGLVideoOutput);2398 2399 [m_avPlayerItem.get() addOutput:m_openGLVideoOutput.get()];2400 }2401 2402 void MediaPlayerPrivateAVFoundationObjC::destroyOpenGLVideoOutput()2403 {2404 if (!m_openGLVideoOutput)2405 return;2406 2407 INFO_LOG(LOGIDENTIFIER);2408 2409 if (m_avPlayerItem)2410 [m_avPlayerItem.get() removeOutput:m_openGLVideoOutput.get()];2411 2412 m_openGLVideoOutput = 0;2413 }2414 2415 void MediaPlayerPrivateAVFoundationObjC::updateLastOpenGLImage()2416 {2417 if (!m_openGLVideoOutput)2418 return;2419 2420 CMTime currentTime = [m_openGLVideoOutput itemTimeForHostTime:CACurrentMediaTime()];2421 if (![m_openGLVideoOutput hasNewPixelBufferForItemTime:currentTime])2422 return;2423 2424 m_lastOpenGLImage = adoptCF([m_openGLVideoOutput copyPixelBufferForItemTime:currentTime itemTimeForDisplay:nil]);2425 }2426 2427 2386 bool MediaPlayerPrivateAVFoundationObjC::copyVideoTextureToPlatformTexture(GraphicsContext3D* context, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY) 2428 2387 { 2429 2388 ASSERT(context); 2430 2389 2431 if (!m_openGLVideoOutput) 2432 createOpenGLVideoOutput(); 2433 2434 updateLastOpenGLImage(); 2435 2436 if (!m_lastOpenGLImage) 2390 updateLastPixelBuffer(); 2391 if (!m_lastPixelBuffer) 2437 2392 return false; 2438 2393 2439 size_t width = CVPixelBufferGetWidth(m_lastOpenGLImage.get()); 2440 size_t height = CVPixelBufferGetHeight(m_lastOpenGLImage.get()); 2441 2442 if (!m_textureCache) { 2443 m_textureCache = TextureCacheCV::create(*context); 2444 if (!m_textureCache) 2445 return false; 2446 } 2447 2448 RetainPtr<CVOpenGLTextureRef> videoTexture = m_textureCache->textureFromImage(m_lastOpenGLImage.get(), outputTarget, level, internalFormat, format, type); 2394 size_t width = CVPixelBufferGetWidth(m_lastPixelBuffer.get()); 2395 size_t height = CVPixelBufferGetHeight(m_lastPixelBuffer.get()); 2449 2396 2450 2397 if (!m_videoTextureCopier) 2451 2398 m_videoTextureCopier = std::make_unique<VideoTextureCopierCV>(*context); 2452 2399 2453 return m_videoTextureCopier->copy VideoTextureToPlatformTexture(videoTexture.get(), width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY);2400 return m_videoTextureCopier->copyImageToPlatformTexture(m_lastPixelBuffer.get(), width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY); 2454 2401 } 2455 2402 -
trunk/Source/WebCore/platform/graphics/cairo/GraphicsContext3DCairo.cpp
r225428 r225638 114 114 } 115 115 116 GraphicsContext3D::GraphicsContext3D(GraphicsContext3DAttributes attributes, HostWindow*, GraphicsContext3D::RenderStyle renderStyle )116 GraphicsContext3D::GraphicsContext3D(GraphicsContext3DAttributes attributes, HostWindow*, GraphicsContext3D::RenderStyle renderStyle, GraphicsContext3D* sharedContext) 117 117 : m_attrs(attributes) 118 118 { 119 ASSERT_UNUSED(sharedContext, !sharedContext); 119 120 #if USE(TEXTURE_MAPPER) 120 121 m_texmapLayer = std::make_unique<TextureMapperGC3DPlatformLayer>(*this, renderStyle); -
trunk/Source/WebCore/platform/graphics/cocoa/GraphicsContext3DCocoa.mm
r223707 r225638 387 387 } 388 388 389 GraphicsContext3D::GraphicsContext3D(GraphicsContext3DAttributes attrs, HostWindow*, GraphicsContext3D::RenderStyle) 389 Ref<GraphicsContext3D> GraphicsContext3D::createShared(GraphicsContext3D& sharedContext) 390 { 391 auto context = adoptRef(*new GraphicsContext3D(sharedContext.getContextAttributes(), nullptr, sharedContext.m_renderStyle, &sharedContext)); 392 393 manager().addContext(context.ptr()); 394 395 return context; 396 } 397 398 GraphicsContext3D::GraphicsContext3D(GraphicsContext3DAttributes attrs, HostWindow*, GraphicsContext3D::RenderStyle, GraphicsContext3D* sharedContext) 390 399 : m_attrs(attrs) 391 400 #if PLATFORM(IOS) … … 396 405 #if PLATFORM(IOS) 397 406 EAGLRenderingAPI api = m_attrs.useGLES3 ? kEAGLRenderingAPIOpenGLES3 : kEAGLRenderingAPIOpenGLES2; 398 m_contextObj = [[EAGLContext alloc] initWithAPI:api]; 407 if (!sharedContext) 408 m_contextObj = [[EAGLContext alloc] initWithAPI:api]; 409 else 410 m_contextObj = [[EAGLContext alloc] initWithAPI:api sharegroup:sharedContext->m_contextObj.sharegroup]; 399 411 makeContextCurrent(); 400 412 #else … … 443 455 return; 444 456 445 CGLError err = CGLCreateContext(pixelFormatObj, 0, &m_contextObj);457 CGLError err = CGLCreateContext(pixelFormatObj, sharedContext ? sharedContext->m_contextObj : nullptr, &m_contextObj); 446 458 GLint abortOnBlacklist = 0; 447 459 #if PLATFORM(MAC) -
trunk/Source/WebCore/platform/graphics/cocoa/WebCoreDecompressionSession.mm
r223476 r225638 186 186 187 187 auto currentTime = CMTimebaseGetTime(m_timebase.get()); 188 auto presentationTimeStamp = CMSampleBufferGetPresentationTimeStamp(sample); 189 if (CMTimeCompare(presentationTimeStamp, currentTime) >= 0) 188 auto presentationStartTime = CMSampleBufferGetPresentationTimeStamp(sample); 189 auto duration = CMSampleBufferGetDuration(sample); 190 auto presentationEndTime = CMTimeAdd(presentationStartTime, duration); 191 if (CMTimeCompare(presentationEndTime, currentTime) >= 0) 190 192 return true; 191 193 -
trunk/Source/WebCore/platform/graphics/cv/VideoTextureCopierCV.cpp
r224207 r225638 97 97 static TransferFunction transferFunctionFromString(CFStringRef string) 98 98 { 99 if ( string == kCVImageBufferYCbCrMatrix_ITU_R_709_2)99 if (CFEqual(string, kCVImageBufferYCbCrMatrix_ITU_R_709_2)) 100 100 return TransferFunction::kITU_R_709_2; 101 if ( string == kCVImageBufferYCbCrMatrix_ITU_R_601_4)101 if (CFEqual(string, kCVImageBufferYCbCrMatrix_ITU_R_601_4)) 102 102 return TransferFunction::kITU_R_601_4; 103 if ( string == kCVImageBufferYCbCrMatrix_SMPTE_240M_1995)103 if (CFEqual(string, kCVImageBufferYCbCrMatrix_SMPTE_240M_1995)) 104 104 return TransferFunction::kSMPTE_240M_1995; 105 if (canLoad_CoreVideo_kCVImageBufferYCbCrMatrix_DCI_P3() && string == kCVImageBufferYCbCrMatrix_DCI_P3)105 if (canLoad_CoreVideo_kCVImageBufferYCbCrMatrix_DCI_P3() && CFEqual(string, kCVImageBufferYCbCrMatrix_DCI_P3)) 106 106 return TransferFunction::kDCI_P3; 107 if (canLoad_CoreVideo_kCVImageBufferYCbCrMatrix_P3_D65() && string == kCVImageBufferYCbCrMatrix_P3_D65)107 if (canLoad_CoreVideo_kCVImageBufferYCbCrMatrix_P3_D65() && CFEqual(string, kCVImageBufferYCbCrMatrix_P3_D65)) 108 108 return TransferFunction::kP3_D65; 109 if (canLoad_CoreVideo_kCVImageBufferYCbCrMatrix_ITU_R_2020() && string == kCVImageBufferYCbCrMatrix_ITU_R_2020)109 if (canLoad_CoreVideo_kCVImageBufferYCbCrMatrix_ITU_R_2020() && CFEqual(string, kCVImageBufferYCbCrMatrix_ITU_R_2020)) 110 110 return TransferFunction::kITU_R_2020; 111 111 return TransferFunction::Unknown; … … 123 123 // https://www.itu.int/rec/R-REC-BT.601 124 124 matrices.get().emplace(MapKey(PixelRange::Video, TransferFunction::kITU_R_601_4), Vector<GLfloat>({ 125 1.164383562f, 0.0f, 1.596026786f, 126 1.164383562f, -0.3917622901f, -0.8129676472f, 127 1.164383562f, 2.017232143f, 0.0f, 125 1.164383562f, 0.0f, 1.596026786f, -0.874202218f, 126 1.164383562f, -0.3917622901f, -0.8129676472f, 0.531667823f, 127 1.164383562f, 2.017232143f, 0.0f, -1.085630790f, 128 0.0f, 0.0f, 0.0f, 1.0f, 128 129 })); 129 130 matrices.get().emplace(MapKey({PixelRange::Full, TransferFunction::kITU_R_601_4}), Vector<GLfloat>({ 130 1.000000000f, 0.0f, 1.4075196850f, 131 1.000000000f, -0.3454911535f, -0.7169478464f, 132 1.000000000f, 1.7789763780f, 0.0f, 131 1.000000000f, 0.0f, 1.4075196850f, -1.012571428f, 132 1.000000000f, -0.3454911535f, -0.7169478464f, 0.533302714f, 133 1.000000000f, 1.7789763780f, 0.0f, -0.892976378f, 134 0.0f, 0.0f, 0.0f, 1.0f, 133 135 })); 134 136 // Matrices are derived from the components in the ITU R.709 rev 2 specification 135 137 // https://www.itu.int/rec/R-REC-BT.709-2-199510-S 136 138 matrices.get().emplace(MapKey({PixelRange::Video, TransferFunction::kITU_R_709_2}), Vector<GLfloat>({ 137 1.164383562f, 0.0f, 1.792741071f, 138 1.164383562f, -0.2132486143f, -0.5329093286f, 139 1.164383562f, 2.112401786f, 0.0f, 139 1.164383562f, 0.0f, 1.792741071f, -0.972945075f, 140 1.164383562f, -0.2132486143f, -0.5329093286f, 0.301482665f, 141 1.164383562f, 2.112401786f, 0.0f, -1.133402218f, 142 0.0f, 0.0f, 0.0f, 1.0f, 140 143 })); 141 144 matrices.get().emplace(MapKey({PixelRange::Full, TransferFunction::kITU_R_709_2}), Vector<GLfloat>({ 142 1.000000000f, 0.0f, 1.5810000000f, 143 1.000000000f, -0.1880617701f, -0.4699672819f, 144 1.000000000f, 1.8629055118f, 0.0f, 145 1.000000000f, 0.0f, 1.5810000000f, -0.793600000f, 146 1.000000000f, -0.1880617701f, -0.4699672819f, 0.330304779f, 147 1.000000000f, 1.8629055118f, 0.0f, -0.935105512f, 148 0.0f, 0.0f, 0.0f, 1.0f, 145 149 })); 146 150 }); … … 163 167 164 168 VideoTextureCopierCV::VideoTextureCopierCV(GraphicsContext3D& context) 165 : m_context(context) 169 : m_sharedContext(context) 170 , m_context(GraphicsContext3D::createShared(context)) 166 171 , m_framebuffer(context.createFramebuffer()) 167 172 { … … 322 327 fragmentShaderSource.appendLiteral("uniform int u_premultiply;\n"); 323 328 fragmentShaderSource.appendLiteral("uniform vec2 u_textureDimensions;\n"); 329 fragmentShaderSource.appendLiteral("uniform int u_swapColorChannels;\n"); 324 330 fragmentShaderSource.appendLiteral("void main() {\n"); 325 331 fragmentShaderSource.appendLiteral(" vec2 texPos = vec2(v_texturePosition.x * u_textureDimensions.x, v_texturePosition.y * u_textureDimensions.y);\n"); … … 329 335 fragmentShaderSource.appendLiteral(" vec4 color = texture2DRect(u_texture, texPos);\n"); 330 336 #endif 337 fragmentShaderSource.appendLiteral(" if (u_swapColorChannels == 1) {\n"); 338 fragmentShaderSource.appendLiteral(" color.rgba = color.bgra;\n"); 339 fragmentShaderSource.appendLiteral(" }\n"); 331 340 fragmentShaderSource.appendLiteral(" if (u_premultiply == 1) {\n"); 332 341 fragmentShaderSource.appendLiteral(" gl_FragColor = vec4(color.r * color.a, color.g * color.a, color.b * color.a, color.a);\n"); … … 366 375 m_textureDimensionsUniformLocation = m_context->getUniformLocation(m_program, ASCIILiteral("u_textureDimensions")); 367 376 m_flipYUniformLocation = m_context->getUniformLocation(m_program, ASCIILiteral("u_flipY")); 377 m_swapColorChannelsUniformLocation = m_context->getUniformLocation(m_program, ASCIILiteral("u_swapColorChannels")); 368 378 m_premultiplyUniformLocation = m_context->getUniformLocation(m_program, ASCIILiteral("u_premultiply")); 369 379 m_positionAttributeLocation = m_context->getAttribLocationDirect(m_program, ASCIILiteral("a_position")); … … 434 444 "uniform SAMPLERTYPE u_yTexture;\n" 435 445 "uniform SAMPLERTYPE u_uvTexture;\n" 436 "uniform mat 3u_colorMatrix;\n"446 "uniform mat4 u_colorMatrix;\n" 437 447 "varying vec2 v_yTextureCoordinate;\n" 438 448 "varying vec2 v_uvTextureCoordinate;\n" 439 449 "void main() {\n" 440 " vec 3yuv;\n"450 " vec4 yuv;\n" 441 451 " yuv.r = TEXTUREFUNC(u_yTexture, v_yTextureCoordinate).r;\n" 442 " yuv.gb = TEXTUREFUNC(u_uvTexture, v_uvTextureCoordinate).rg - vec2(0.5, 0.5);\n" 443 " gl_FragColor = vec4(yuv * u_colorMatrix, 1);\n" 452 " yuv.gb = TEXTUREFUNC(u_uvTexture, v_uvTextureCoordinate).rg;\n" 453 " yuv.a = 1.0;\n" 454 " gl_FragColor = yuv * u_colorMatrix;\n" 444 455 "}\n" 445 456 ); … … 490 501 m_context->bindBuffer(GraphicsContext3D::ARRAY_BUFFER, m_yuvVertexBuffer); 491 502 m_context->bufferData(GraphicsContext3D::ARRAY_BUFFER, sizeof(vertices), vertices, GraphicsContext3D::STATIC_DRAW); 503 m_context->enableVertexAttribArray(m_yuvPositionAttributeLocation); 504 m_context->vertexAttribPointer(m_yuvPositionAttributeLocation, 2, GraphicsContext3D::FLOAT, false, 0, 0); 492 505 493 506 return true; … … 502 515 } 503 516 504 if (auto texture = m_textureCache->textureFromImage(image, outputTarget, level, internalFormat, format, type)) 505 return copyVideoTextureToPlatformTexture(texture.get(), width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY); 517 if (auto texture = m_textureCache->textureFromImage(image, outputTarget, level, internalFormat, format, type)) { 518 bool swapColorChannels = false; 519 #if PLATFORM(IOS) 520 // FIXME: Remove this workaround once rdar://problem/35834388 is fixed. 521 swapColorChannels = CVPixelBufferGetPixelFormatType(image) == kCVPixelFormatType_32BGRA; 522 #endif 523 return copyVideoTextureToPlatformTexture(texture.get(), width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY, swapColorChannels); 524 } 506 525 507 526 #if USE(IOSURFACE) … … 527 546 } 528 547 529 GC3DStateSaver stateSaver(m_context.get());530 531 548 if (!m_yuvProgram) { 532 549 if (!initializeUVContextObjects()) { … … 535 552 } 536 553 } 537 538 stateSaver.saveVertexAttribState(m_yuvPositionAttributeLocation);539 554 540 555 m_context->bindFramebuffer(GraphicsContext3D::FRAMEBUFFER, m_framebuffer); … … 604 619 auto transferFunction = transferFunctionFromString((CFStringRef)CVBufferGetAttachment(image, kCVImageBufferYCbCrMatrixKey, nil)); 605 620 auto& colorMatrix = YCbCrToRGBMatrixForRangeAndTransferFunction(range, transferFunction); 606 m_context->uniformMatrix 3fv(m_colorMatrixUniformLocation, 1, GL_FALSE, colorMatrix.data());621 m_context->uniformMatrix4fv(m_colorMatrixUniformLocation, 1, GL_FALSE, colorMatrix.data()); 607 622 608 623 // Do the actual drawing. 609 m_context->bindBuffer(GraphicsContext3D::ARRAY_BUFFER, m_yuvVertexBuffer);610 m_context->enableVertexAttribArray(m_yuvPositionAttributeLocation);611 m_context->vertexAttribPointer(m_yuvPositionAttributeLocation, 2, GraphicsContext3D::FLOAT, false, 0, 0);612 624 m_context->drawArrays(GraphicsContext3D::TRIANGLES, 0, 6); 625 626 #if PLATFORM(IOS) 627 // flush() must be called here in order to re-synchronize the output texture's contents across the 628 // two EAGL contexts. 629 m_context->flush(); 630 #endif 613 631 614 632 // Clean-up. 615 633 m_context->deleteTexture(yTexture); 616 634 m_context->deleteTexture(uvTexture); 617 m_context->bindTexture(videoTextureTarget, 0);618 635 619 636 m_lastSurface = surface; … … 628 645 } 629 646 630 bool VideoTextureCopierCV::copyVideoTextureToPlatformTexture(TextureType inputVideoTexture, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY )647 bool VideoTextureCopierCV::copyVideoTextureToPlatformTexture(TextureType inputVideoTexture, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY, bool swapColorChannels) 631 648 { 632 649 if (!inputVideoTexture) … … 650 667 flipY = !flipY; 651 668 652 return copyVideoTextureToPlatformTexture(videoTextureName, videoTextureTarget, width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY );653 } 654 655 bool VideoTextureCopierCV::copyVideoTextureToPlatformTexture(Platform3DObject videoTextureName, GC3Denum videoTextureTarget, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY )669 return copyVideoTextureToPlatformTexture(videoTextureName, videoTextureTarget, width, height, outputTexture, outputTarget, level, internalFormat, format, type, premultiplyAlpha, flipY, swapColorChannels); 670 } 671 672 bool VideoTextureCopierCV::copyVideoTextureToPlatformTexture(Platform3DObject videoTextureName, GC3Denum videoTextureTarget, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY, bool swapColorChannels) 656 673 { 657 674 LOG(WebGL, "VideoTextureCopierCV::copyVideoTextureToPlatformTexture(%p) - internalFormat: %s, format: %s, type: %s flipY: %s, premultiplyAlpha: %s", this, enumToStringMap()[internalFormat], enumToStringMap()[format], enumToStringMap()[type], flipY ? "true" : "false", premultiplyAlpha ? "true" : "false"); 658 659 GC3DStateSaver stateSaver(m_context.get());660 675 661 676 if (!m_program) { … … 665 680 } 666 681 } 667 668 stateSaver.saveVertexAttribState(m_positionAttributeLocation);669 682 670 683 m_context->bindFramebuffer(GraphicsContext3D::FRAMEBUFFER, m_framebuffer); … … 705 718 706 719 m_context->uniform1i(m_flipYUniformLocation, flipY); 720 m_context->uniform1i(m_swapColorChannelsUniformLocation, swapColorChannels); 707 721 m_context->uniform1i(m_premultiplyUniformLocation, premultiplyAlpha); 708 722 … … 713 727 m_context->drawArrays(GraphicsContext3D::TRIANGLES, 0, 6); 714 728 729 #if PLATFORM(IOS) 730 // flush() must be called here in order to re-synchronize the output texture's contents across the 731 // two EAGL contexts. 732 m_context->flush(); 733 #endif 734 715 735 // Clean-up. 716 736 m_context->bindTexture(videoTextureTarget, 0); … … 720 740 } 721 741 722 VideoTextureCopierCV::GC3DStateSaver::GC3DStateSaver(GraphicsContext3D& context) 723 : m_context(context) 724 { 725 m_activeTextureUnit = m_context.activeTextureUnit(); 726 m_boundTarget = m_context.currentBoundTarget(); 727 m_boundTexture = m_context.currentBoundTexture(); 728 m_context.getIntegerv(GraphicsContext3D::FRAMEBUFFER_BINDING, &m_framebuffer); 729 m_context.getIntegerv(GraphicsContext3D::CURRENT_PROGRAM, &m_program); 730 m_context.getIntegerv(GraphicsContext3D::ARRAY_BUFFER_BINDING, &m_arrayBuffer); 731 m_context.getIntegerv(GraphicsContext3D::VIEWPORT, m_viewport); 732 733 } 734 735 VideoTextureCopierCV::GC3DStateSaver::~GC3DStateSaver() 736 { 737 if (m_vertexAttribEnabled) 738 m_context.enableVertexAttribArray(m_vertexAttribIndex); 739 else 740 m_context.disableVertexAttribArray(m_vertexAttribIndex); 741 742 m_context.bindBuffer(GraphicsContext3D::ARRAY_BUFFER, m_arrayBuffer); 743 m_context.vertexAttribPointer(m_vertexAttribIndex, m_vertexAttribSize, m_vertexAttribType, m_vertexAttribNormalized, m_vertexAttribStride, m_vertexAttribPointer); 744 745 m_context.activeTexture(m_activeTextureUnit); 746 m_context.bindTexture(m_boundTarget, m_boundTexture); 747 m_context.bindFramebuffer(GraphicsContext3D::FRAMEBUFFER, m_framebuffer); 748 m_context.useProgram(m_program); 749 m_context.viewport(m_viewport[0], m_viewport[1], m_viewport[2], m_viewport[3]); 750 } 751 752 void VideoTextureCopierCV::GC3DStateSaver::saveVertexAttribState(GC3Duint index) 753 { 754 m_vertexAttribIndex = index; 755 m_context.getVertexAttribiv(index, GraphicsContext3D::VERTEX_ATTRIB_ARRAY_ENABLED, &m_vertexAttribEnabled); 756 m_context.getVertexAttribiv(index, GraphicsContext3D::VERTEX_ATTRIB_ARRAY_SIZE, &m_vertexAttribSize); 757 m_context.getVertexAttribiv(index, GraphicsContext3D::VERTEX_ATTRIB_ARRAY_TYPE, &m_vertexAttribType); 758 m_context.getVertexAttribiv(index, GraphicsContext3D::VERTEX_ATTRIB_ARRAY_NORMALIZED, &m_vertexAttribNormalized); 759 m_context.getVertexAttribiv(index, GraphicsContext3D::VERTEX_ATTRIB_ARRAY_STRIDE, &m_vertexAttribStride); 760 m_vertexAttribPointer = m_context.getVertexAttribOffset(index, GraphicsContext3D::VERTEX_ATTRIB_ARRAY_POINTER); 761 } 762 763 } 742 } -
trunk/Source/WebCore/platform/graphics/cv/VideoTextureCopierCV.h
r224207 r225638 51 51 52 52 bool copyImageToPlatformTexture(CVPixelBufferRef, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY); 53 bool copyVideoTextureToPlatformTexture(TextureType, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY );53 bool copyVideoTextureToPlatformTexture(TextureType, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY, bool swapColorChannels = false); 54 54 55 GraphicsContext3D& context() { return m_context .get(); }55 GraphicsContext3D& context() { return m_context; } 56 56 57 57 private: 58 bool copyVideoTextureToPlatformTexture(Platform3DObject inputTexture, GC3Denum inputTarget, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY); 59 60 class GC3DStateSaver { 61 public: 62 GC3DStateSaver(GraphicsContext3D&); 63 ~GC3DStateSaver(); 64 65 void saveVertexAttribState(GC3Duint index); 66 67 private: 68 GraphicsContext3D& m_context; 69 GC3Denum m_activeTextureUnit { 0 }; 70 GC3Denum m_boundTarget { 0 }; 71 GC3Denum m_boundTexture { 0 }; 72 GC3Dint m_framebuffer { 0 }; 73 GC3Dint m_program { 0 }; 74 GC3Dint m_arrayBuffer { 0 }; 75 GC3Dint m_viewport[4] { 0, 0, 0, 0 }; 76 77 GC3Duint m_vertexAttribIndex { 0 }; 78 GC3Dint m_vertexAttribEnabled { 0 }; 79 GC3Dint m_vertexAttribSize { 0 }; 80 GC3Dint m_vertexAttribType { 0 }; 81 GC3Dint m_vertexAttribNormalized { 0 }; 82 GC3Dint m_vertexAttribStride { 0 }; 83 GC3Dsizeiptr m_vertexAttribPointer { 0 }; 84 }; 58 bool copyVideoTextureToPlatformTexture(Platform3DObject inputTexture, GC3Denum inputTarget, size_t width, size_t height, Platform3DObject outputTexture, GC3Denum outputTarget, GC3Dint level, GC3Denum internalFormat, GC3Denum format, GC3Denum type, bool premultiplyAlpha, bool flipY, bool swapColorChannels); 85 59 86 60 bool initializeContextObjects(); … … 95 69 #endif 96 70 71 Ref<GraphicsContext3D> m_sharedContext; 97 72 Ref<GraphicsContext3D> m_context; 98 73 std::unique_ptr<TextureCacheCV> m_textureCache; … … 103 78 GC3Dint m_textureDimensionsUniformLocation { -1 }; 104 79 GC3Dint m_flipYUniformLocation { -1 }; 80 GC3Dint m_swapColorChannelsUniformLocation { -1 }; 105 81 GC3Dint m_premultiplyUniformLocation { -1 }; 106 82 GC3Dint m_positionAttributeLocation { -1 }; -
trunk/Source/WebCore/platform/graphics/opengl/GraphicsContext3DOpenGLES.cpp
r219553 r225638 275 275 } 276 276 277 GraphicsContext3D::GraphicsContext3D(GraphicsContext3DAttributes attributes, HostWindow*, GraphicsContext3D::RenderStyle renderStyle )277 GraphicsContext3D::GraphicsContext3D(GraphicsContext3DAttributes attributes, HostWindow*, GraphicsContext3D::RenderStyle renderStyle, GraphicsContext3D* sharedContext) 278 278 : m_attrs(attributes) 279 279 , m_compiler(isGLES2Compliant() ? SH_ESSL_OUTPUT : SH_GLSL_COMPATIBILITY_OUTPUT) 280 280 , m_private(std::make_unique<GraphicsContext3DPrivate>(this, renderStyle)) 281 281 { 282 ASSERT_UNUSED(sharedContext, !sharedContext); 282 283 makeContextCurrent(); 283 284
Note: See TracChangeset
for help on using the changeset viewer.